[1.x] Adds Microsoft SQL Server

What?
This brings Microsoft SQL Server (their latest version) to Sail. I did it because I usually deal with clients using Azure ootb, and MSSQL it's very picky.
Because the Docker Container is very non-standard, the whole setup is done by setting a custom entrypoint. If no command is issued, the script initializes the databases (default and testing) and then starts SQL Server again but through exec.
EULA shenanigans
The installation is done by manually accepting Microsoft SQL Server EULA, which is found on their Docker Hub. This is both required to install the database engine and the PHP extensions to connect. If the EULA is not accepted, the installation fails.
This sounds like a hurdle, but the correct thing to do is let the developer accept the EULA.
What adds and changes

Adds sqlsrv service.
Adds the DB_ROOT_PASWORD with a complex password (required by SQL Server)
All Dockerfile have an additional block to install sqlsrv and pdo_sqlsrv only if the EULA is accepted (by default, nope).
Extensions version are fixed. This is because latest versions drop support for old PHP versions.

Pending work

The Sail Server (laravel.build) should add slqsrv to the accepted services array.
The documentation (laravel.com/docs/sail) should mention sqlsrv support with the following caveats:

Microsoft SQL Server is a x64-only container.
Requires EULA Acceptance
Adds a separate "root" strong password due to default SQL Server policies.

Thanks for submitting a PR!
Note that draft PR's are not reviewed. If you would like a review, please mark your pull request as ready for review in the GitHub user interface.
Pull requests that are abandoned in draft may be closed due to inactivity.

Thanks for noting that, dear bot. The PR is ready for review, so this won't get abandoned.

Thanks for your pull request to Laravel!
Unfortunately, I'm going to delay merging this code for now. To preserve our ability to adequately maintain the framework, we need to be very careful regarding the amount of code we include.
If applicable, please consider releasing your code as a package so that the community can still take advantage of your contributions!


func settings ends with System.NullReferenceException

func settings lists out all the related settings commands and then ends with
System.NullReferenceException: Object reference not set to an instance of an object.
   at Azure.Functions.Cli.Actions.HelpAction.DisplayOptions(IEnumerable`1 options)
   at Azure.Functions.Cli.Actions.HelpAction.DisplaySwitches(ActionType actionType)

Functionality still works but the error is displayed in red.

closed with @joescars's PR https://github.com/Azure/azure-functions-cli/pull/120


Deny doesn't call onPermissionResult

I'm using permiso in a fragment within a tablayout Fragment extends ListFragment
In the onResume I am performing the permission request:
@Override
    public void onResume()
    {
        super.onResume();

        Permiso.getInstance().setActivity(getActivity());

        Log.e(TAG, "Requesting Contact Permission");

        Permiso.getInstance().requestPermissions(new Permiso.IOnPermissionResult()
        {
            @Override
            public void onPermissionResult(Permiso.ResultSet resultSet)
            {
                if(resultSet.areAllPermissionsGranted())
                {
                    Log.e(TAG, "Contact Permission Granted");
                }
                else
                {
                    Log.e(TAG, "Contact Permission Denied");
                }
            }

            @Override
            public void onRationaleRequested(Permiso.IOnRationaleProvided callback, String... permissions)
            {
                Permiso.getInstance().showRationaleInDialog(getText(R.string.contacts_string_contactpermissiontitle).toString(), getText(R.string.contacts_string_contactpermissionrationale).toString(), null, callback);
            }
        }, Manifest.permission.WRITE_CONTACTS);
    }

The onPermissionResult is never called if the 'Deny' option is pressed.
It all works correctly if 'Allow' is pressed.
Please note, the contact fragment is not the default tab but it is next to it so it gets loaded anyway.
I have also set:
@Override
    public void onRequestPermissionsResult(int requestCode, String[] permissions, int[] grantResults)
    {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);
        Permiso.getInstance().onRequestPermissionResult(requestCode, permissions, grantResults);
    }

and
@Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);

        // Setup permission library
        Permiso.getInstance().setActivity(getActivity());
    }

Hi @behelit, I'm looking into it! I'm doing my best to replicate your code setup. Here are some questions:

In your logs, do you see anything with the tag "Permiso" that says "onRequestPermissionResult() was given an unrecognized request code." after you deny the permission?
Is the code your showing me in the fragment? If so, you shouldn't be calling Permiso.setActivity(). That only needs to be done at the activity level. The easy way to do it is by having your activity subclass PermisoActivity. If you don't want to do that, you just have to make sure your activity calls the correct methods as specified in the README (short version: call Permiso.setActivity() in onCreate() and onResume(), and forward the results of onRequestPermissionsResult() to Permiso.onRequestPermissionResult(). Nothing needs to be done in the fragment itself.

So, I just created a simple Activity-Fragment setup, and I put this code in the Fragment's onResume(), and I can't replicate your issue. However, I'm guessing your issue is really in the details, something to do with either your activity setup or the tab group.
Also, can I get the details of the phone and android version you're testing on?

Oh, also, when you do get the 'deny' button working, you're going to run into an issue. The user will get stuck in a deny loop. After they deny the permission, onResume() is called, triggering another permission request, showing the rationale dialog, which when dismissed triggers another onResume(), etc, etc getting the user stuck in a loop.

Anyway, this is the app I created to in the process of trying to mimic your setup. Feel free to make changed to try to reproduce the issue. I'll add you as a collaborator.
https://github.com/greysonp/permiso-debug-app

Thanks @greysonp, I will try out your suggestions relating to the parent activity (and onStart) and give an update.
I think that sounds like the most probably cause.

Yes, that was the problem.
I moved the Permiso instance configurations into the tab activity and everything is working as expected.
Thanks for your help. This library is awesome.

That's great to hear, and thank you!


Bump to Wasmtime v0.29.0 and Cranelift 0.76.0.

Release Wasmtime v0.29.0 and Cranelift v0.76.0.

Also, while here, I went ahead and updated the release notes -- https://github.com/bytecodealliance/wasmtime/commit/e9fe64f0c7eb14975f1788a57b097258ad8850d0 -- mind including that here?

Ah, I hadn't done that yet, thanks!

Hi @cfallin @alexcrichton .. Yes there are two more pull requests that are needed to complete SIMD. Specifically what is left is what covers these tests:
    match (testsuite, testname) {
        ("simd", "simd_i16x8_extadd_pairwise_i8x16") => return true,
        ("simd", "simd_i32x4_extadd_pairwise_i16x8") => return true,
        ("simd", "simd_int_to_int_extend") => return true,
        ("simd", _) => return false,
        _ => {}
    }

The first two are taken care of by pr #3031. It actually works fine but I think we want to do a little refactoring. The lowerings that cover int_to_int_extend I haven't pushed a pr for but have started. Let me see how far I get and give an updated estimate later this week if no urgency to finish by this week.

I just updated this PR to rebase on top of latest main with the last x64 SIMD bits (and Alex's just-merged 64-bit memory limit PR). Also updated release notes accordingly. PTAL @alexcrichton for updated note on the latter especially -- thanks!


cmake doesn't build anything

With BUILD_EXAMPLES=OFF BUILD_UNIT_TESTS=OFF I am getting
-- Build files have been written to: /usr/ports/math/ascent/work/.build
===>  Building for Ascent-<IP_ADDRESS>
ninja: no work to do.
===>  Staging for Ascent-<IP_ADDRESS>
===>   Generating temporary packing list
ninja: error: unknown target 'install/strip'
*** Error code 1

This issue hasn't been properly resolved.

usually it is your project's CMake file that will have a line like:
include_directories(../Ascent/include)

The above isn't true. In order to properly create a package the project needs to install its files.
In your case, installing means copying files into ${DESTDIR}${PREFIX}/include/...
cmake has instructions for this.

Pull request #7 should fix this issue.
If it is accepted you should be able install ascent and use it like this:
find_package(ascent REQUIRED)
...Other stuff...
target_link_libraries(${PROJECT_NAME} ascent::ascent)


Watchdog should be passthrough for function process for logging

Expected Behavior
In the cases where invocation is not utilizing the stdout/err pipes for communication with the watchdog the watchdog should not modify or impose structure on the logs passing through from these event streams.  This allows me as an application author to manage how my logs are presented in order to be processed downstream, for example if I wanted to use structured logging such as with JSON or CSV.  Ideally the Watchdog should not mangle the output of my application except to make it available to the platform log driver.
There are some logs that need to be emitted from the watchdog itself:
• Errors parsing config
• Forked process termination
• Issues reading from the pipe files
• Errors communicating with metrics services
These of-watchdog logs should be able to be easily and optionally be skipped or selected on by log parsers/shippers so that I can have an unobscured event trail of logs as emitted by my application.
Current Behavior
There are several undesirable behaviors that exist in the current implementation that highlight the mangling that is currently being done by the Watchdog.
Lines longer than 256 bytes are split
There is a hard limit on the number of bytes that can be in a single line from the wrapped function process.  For instance a message of 300 bytes emitted from the underlying function would be split to two lines of 256 bytes and the remaining 44 bytes below.  Even worse it can be split across output from other go routines (such as the watchdog itself).
Example:
2019/08/09 00:01:42 stderr: 2019/08/09 00:01:42 {"abexi":"yxgdndacmbhgwhadjnba","aclra":"dbgmdiytcwloabveajbe","aczai":"nlqlibaatlyhdaopovfo","aereu":"nuzjqxmzotarlutmygms","afgie":"hlgizmhgzptxtfrgkaqq","ahugb":"qxifgcyvgcazaefizhgw","ajxwk":"afikzruuywsuwkobbuor","akgza":"altlhtuzh
2019/08/09 00:01:42 GET / - 200 OK - ContentLength: 2
2019/08/09 00:01:42 stderr: oidz","zxbqo":"tukdfklasvyafstrdpoj","zynxg":"bfrwcxacobgabksdrjdi","zzawj":"lvncjvkyrwrlixxqcahq","zzhms":"ouyykvnikbudiryewvos","zzttv":"wvtuerakfsxlplgaftry"}

Stderr and Stdout are redirected to Stderr for HttpRunner
These files are being piped into the same output file Stderr with a prefix.  This is somewhat surprising since for the HttpRunner both of these files are written to, however Stdout does not contain any output from my wrapped application.The prefix itself isn't that bad as it can allow me to easily determine if a log line is from the wrapped process or the of-watchdog itself.  However combined with the hard limit of 256 bytes splitting lines up, if I have a structured log line it becomes extremely difficult to parse these structures consistently.
Example:
stdout: My function's original log message to stdout
stderr: My function's original log message to stderr

Null bytes and extra new lines in output.
The current implementation is calling Println and passing a buffer of length 256.  This is resulting in all of the bytes to be written out and causing double new lines in those cases as a newline is being copied from the original buffer and the another one is applied to the end of the byte buffer and if you're using a viewer that doesn't strip null bytes (such as a tool that writes to a file) those null bytes are cluttering up the output.
Example log file:
2019/08/26 23:11:21 stderr: Starting application server
������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������

Possible Solution
I originally proposed a solution involving a line based tokenizer from bufio as a way to attempt to output align the mirrored logs with those of the wrapped function application.  I believe this addresses the problems the best in that log lines are persisted from the underlying wrapped application and re-emitted with the common prefixing done by the watchdog as they were originally written.
Another possible solution is to increase the read buffer to be larger than expected output via making it a tunable parameter to the of-watchdog.  So for applications that are doing verbose structured logging a much larger buffer can be provided versus something closer to the default provided now for a terse application.
Steps to Reproduce (for bugs)
I tested this using a contrived http mode go function named chatty-fn that just logs requests that it receives and a bunch of other extra information.  I built the chatty-fn docker container and included different watchdogs to wrap it.  I then boot the docker container directly, and issue network requests directly to it.
# Within chatty-fn
docker build -t chatty-fn:latest .
docker run -p 8080:8080 chatty-fn:latest > stdout.log 2> stderr.log
curl -X POST localhost:8080 -H 'Content-Type: application/json' --data '{"hello": "world"}'

Context
I have been attempting to add structured logging to an application running in a container.  Using a tool like filebeat to parse, ship, and then extract those structured logs into a searchable system.  Due to the small buffer size currently implemented it makes it difficult to handle these json log lines that extend beyond 256 bytes.
Your Environment

Docker version docker version (e.g. Docker 17.0.05 ):
Docker 19.03.1

Are you using Docker Swarm or Kubernetes (FaaS-netes)?
N/A

Operating System and version (e.g. Linux, Windows, MacOS):
Linux and MacOS

Link to your project or a code example to reproduce issue:
https://github.com/cconger/chatty-fn

I would make the buffer size configurable via an environment variable that can be set in the function definition.

+1 @stefanprodan
I've found the PR I mentioned on #72, #37 attempted to add configurable buffer sizes
I think that this is the mechanics of what we'd want:
https://github.com/openfaas-incubator/of-watchdog/pull/37/files#diff-194c7aaf3d69dbfd1dc1fb6c6837cd18L72
What is confusing about the old PR, is that it looks like I wrote it, but @canerbasat proposed it in a PR.
In any case, I think I would be open to a single buffer value or potentially two different buffer size values for stdin/stdout.
Alex

I propose a single env variable for both stdout/err named log_buffer_size.

the proposal mentions a couple of things

buffering (or not) the message and the buffer size
the merging of the function stderr and stdout by watchdog
watchdog specific logs are interwoven with function logs

for topic (1) I am also in favor of a new env variable log_buffer_size. Increasing the default buffer might also make sense. If a true streaming solution existed, I would prefer that. Also, it is worth noting that io.Copy allocates a 32kb buffer https://golang.org/src/io/io.go?s=12796:12856#L391 so using io.Copy is also a buffered solution.
for (2), I did some digging into the k8s and Swarm docs. Both collect stderr and stdout automatically, so I think it would be fine for watchdog to send the stderr to stderr and stdout to stdout without any prefix.
for (3), in the light of (2), perhaps we can partially address this by having watchdog send all of it's own logs to stderr only

I'm for adding the log_buffer_size param.  I would also enjoy both stderr and stdout to be passed to their appropriate outputs sans prefix.  I think these two changes make it possible for the end user to control their experience with logging largely.
I want to experiment a little bit, but I believe on posix that the reads will be usually aligned to pipe fd flushes so a sufficiently large buffer affords control to the writer for chunking the output.  I think therefore increasing the default buffer size to something like the stdlib uses 32kb (io.Copy) to 64kb (for the Scanner) makes sense in addition to user level control.

I've submitted a new PR that adds this parameter, however while playing with it more I remain a relatively strong proponent of the Scanner approach in #72.  It affords a core library solution to a dynamically sizable buffer, with system agnostic handling of new lines.  I would however want to update it to write to the appropriate out file (Stdout => Stdout / Stderr => Stderr)
The benefit one gets from using the scanner is that a line as generated from the wrapped process will be guaranteed to be written as a line in the resulting log output if that line is less than 32Kb.  You can potentially still get log lines interspersed from the watchdog in your lines from the function process however they'll only ever be interspersed between true lines as generated by my application.
A dichotomy of approach seems to be you can either try to be a true byte passthrough (what I've tried to capture in #77) or you can aggregate the lines which seems more appropriate if the watchdog will be emitting logging during operation.  This affords a guarantee on the contents of any given line up to the golang buffer cap (32Kb) and affords a way to filter contents from the watchdog by log parsers.
It also is more reasonable for me as a consumer to properly size what size a log line might be instead of me guessing what a healthy buffer size might be in the case of a GC pause or latency reading the file descriptor from the watchdog.  So perhaps a Scanner built on top of a tunable buffer size from the param makes more sense.

Hey there, I'd appreciate some insight into whether the initial assertion of the issue matters to this project.  Should the wrapped application function be the responsible for the shape and format of logs?  It seems like an obvious win for portability and flexibility to me but if that view isn't held I understand why I seem to be pushing in a different direction than you all.

Hi @cconger
I am concerned about changing the behaviour in this way, but I need more time to dedicate to your problem and am having to prioritise other work and features.
An alternative to using the of-watchdog, is to use a Dockerfile, you can still use the templating system if you want.
What are your thoughts about this blog post? If you're using the http mode, then it will be relatively straight-forward for you to have complete control over STDIO.
https://www.openfaas.com/blog/stateless-microservices/
Alex

The last comment from Lucas/Stefan/myself was about adding log_buffer_size.
Would you like to send a PR for this? It will be easy to merge, or do you feel that you need the line-scanner for structured logging to work?

for (2), I did some digging into the k8s and Swarm docs. Both collect stderr and stdout automatically, so I think it would be fine for watchdog to send the stderr to stderr and stdout to stdout without any prefix.

@LucasRoesler can you explain the context, or what you mean by this? What has log collection got to do with the use of log.Printf(), which prints a prefix? Pass-through of stdio sounds like it's essential for structured logging.

I have a pr of #77 that does the log_buffer_size.  It however raised interesting concerns about preserving lines from the wrapped process and having a consistent prefix for log lines which @LucasRoesler noted in his comment when I removed log.Println.  Because I do not easily see how you can have both a guaranteed prefix on each new line (ideal) and also have a 1:1 mapping of lines in of-watchdog's output and the source application's (also ideal) unless you do buffering and tokenizing.

Is there a way that it makes sense to preserve the existing behaviour, and add your tokenised scanner, so that the two code paths work separately?

Certainly, it can be flagged behavior.  However, I'm unsure why anyone would prefer the existing behavior.  I say that not out of judgement but I just do not see the use case and would enjoy accounting for it.

I guess for backwards compatibility, because this could be a breaking change. Some functions may not write a new-line, in which case, how would the scanner work?

This approach looks fine to me https://github.com/openfaas-incubator/of-watchdog/pull/73
If we wanted to have some level of backwards compatibility then perhaps you could add something like?
bindLoggingPipe - without scanner
bindScanningLoggingPipe - with \n scanner
Given this rather bold statement, I am starting to wonder if I am not understanding something obvious?

I'm unsure why anyone would prefer the existing (code) behavior.

In the case where no newlines were sent the scanner would flush the entire buffer at process conclusion or once the buffer's limit had been hit 64kb.
While I appreciate that its existing behavior its odd to me that someone would expect to write 256 bytes exactly to an output and count on that getting mapped to a line based textual output.  In the case that its some streaming protocol thats not textual/line delimited it is already getting mangled by the transformation to line delimited 256 byte blocks and therefore difficult to use by anything consuming it without having to re-extract it which was happening in my structured logging example.

I'm happy to decouple the 2 proposals that are inside of this issue if it helps ease concerns about change.

Remapping the output file descriptors (using both stderr and out)
Changing output slurping/tokenization from the wrapped file descriptors.

I think 2 is the most important, largely because 1 gets undone by the Kubernetes log driver.

#73 is merged now. I like the solution and the design. It was very refreshing to see the testing and output too, that makes it a lot easier to merge.
I've asked on the PR if you could do a smoke test with two of the more popular templates. Perhaps if that looks good, you could submit a PR to bump of the version in their respective Dockerfiles?

Could you summarise what remaining changes you feel need to be made?

I feel that the issues I was experiencing have been resolved.  I feel comfortable writing to the log in an application controlled way and easily parsing the resulting output with existing tools.
I believe we've addressed all three of the original issues raised in the body.
What I am working on:
• I'm following up on your comment on #73 to be able to turn off prefixes (and log flags) via an argument.
• I will follow up by bumping the Dockerfile of-watchdog for the tested templates.
Happy to proceed how you would like, either tie those changes back to this issue, or open new issues to cover those individually and close this one.

@cconger I have stumbled upon this issue, and I haven't even realized that the log is being split, so thanks for raising that.
Following your conversation, I have stumbled on alex's comment, and this is exactly the issue we are facing at the moment.
Our functions are deployed on k8s cluster which has it's logs scraped and sent upstream to log parsers to be indexed and become queryable. Of course for that issue, we must have the logs in their original JSON structure without the prefix.
Where does this issue stand at the moment, and can I assist?

@saginadir please can you open a new issue for your use-case? @cconger is currently filtering out the prefix with his log aggregator tool.
Would also be helpful if you could join Slack and maybe tell us a bit about your use-case to help prioritise.
Alex

/msg: slack

@alexellis Will do.
Also I requested invitation to the slack channel.

How are we doing with this now @cconger @saginadir ?

@alexellis For my solution, at the moment I forked the watchdog and made the specific changes I needed

I think that the Netflix guys just used a prefix rule for their logging so that it worked with their ingestion pipeline. Maybe that'd work for others too?
Do you have your fork public for us to compare?

Yes, that's my fork
https://github.com/openfaas-incubator/of-watchdog/compare/master...saginadir:master

Thanks.
On this line, how do you ensure that the code doesn't write plain-text? https://github.com/openfaas-incubator/of-watchdog/compare/master...saginadir:master#diff-42b34d1af16e4b2190c38ad0ec0e0f54R18

The logger that logs the messages from the function simply pipes the message from the function, and I make sure the function logs JSON.
And I print the status strings in JSON instead of plain log, which is done by replacing this line:
log.Printf("%s %s - %s - ContentLength: %d", r.Method, r.RequestURI, res.Status, res.ContentLength)

with this:
log.Printf(`{"method":"%s","request_uri":"%s","status_code":%d,"content_length":%d}`, r.Method, r.RequestURI, res.StatusCode, res.ContentLength)

I avoid using JSON encode for a small performance gain (which is really minor for our current scale, but can matter in larger scales)

In our utilization we were planning on just stripping the included prepended flags, regular expression essentially targeting the :
I could see this as a configurable.  In the simplest case it could just a passthrough to the log.Flags call and the user could set LOG_FLAGS=19 as a config environment variable.  The downside is users would have parse the bit mask constants https://golang.org/pkg/log/#pkg-constants. Another alternative would be to just be able to enable/disable the default.

I think that enable/disable the default is probably the simplest and most common case. Any other customization of the log output could be implemented in the function by the template/handler

Having the ability to pass through logs from the function and letting the template/handler decide  the log format would be super helpful

@Voziv please can you improve the quality of your +1 here by stating what problem you're facing and which of the three outlined solutions would help you?

This is what it looks like without the log prefix for streamed lines
2021/02/08 12:17:39 Forking - node [app.js]
2021/02/08 12:17:39 Attached to: stderr.
2021/02/08 12:17:39 Attached to: stdout.
2021/02/08 12:17:39 OperationalMode: http
2021/02/08 12:17:39 Timeouts: read: 10s, write: 10s hard: 10s.
2021/02/08 12:17:39 Listening on port: 8080
2021/02/08 12:17:39 Writing lock-file to: /tmp/.lock
2021/02/08 12:17:39 Metrics listening on port: 8081
stdout: alexellis.io started on port: http://<IP_ADDRESS>:3000
stderr: Disabled etag and x-powered-by

And without everything removed including the prefix
http://<IP_ADDRESS>:3000 fprocess="node app.js" ./of-watchdog
2021/02/08 12:19:22 Forking - node [app.js]
2021/02/08 12:19:22 Attached to: stderr.
2021/02/08 12:19:22 Attached to: stdout.
2021/02/08 12:19:22 OperationalMode: http
2021/02/08 12:19:22 Timeouts: read: 10s, write: 10s hard: 10s.
2021/02/08 12:19:22 Listening on port: 8080
2021/02/08 12:19:22 Writing lock-file to: /tmp/.lock
2021/02/08 12:19:22 Metrics listening on port: 8081
alexellis.io started on port: http://<IP_ADDRESS>:3000
Disabled etag and x-powered-by
2021/02/08 12:19:27 GET / - 200 OK - ContentLength: 19686
2021/02/08 12:19:27 GET / - 200 OK - ContentLength: 19686

However there are mixed requests here, going as far as asking for the watchdog itself to output structured logs in JSON format:
{"level":"info","ts":1612787360.1393752,"caller":"of-watchdog/main.go:353","msg":"Forking","command":"node","args":["app.js"]}
{"level":"info","ts":1612787360.1394744,"caller":"executor/logging.go:17","msg":"Attached to","name":"stderr"}
{"level":"info","ts":1612787360.1395352,"caller":"executor/logging.go:17","msg":"Attached to","name":"stdout"}
{"level":"info","ts":1612787360.1397042,"caller":"of-watchdog/main.go:64","msg":"OperationalMode\n","OperationalMode":"http"}
{"level":"info","ts":1612787360.1398025,"caller":"of-watchdog/main.go:85","msg":"Timeouts","read":10,"write":10,"exec":10}
{"level":"info","ts":1612787360.1398275,"caller":"of-watchdog/main.go:93","msg":"Listening on\n","port":8080}
{"level":"info","ts":1612787360.2052274,"caller":"executor/logging.go:30","msg":"alexellis.io started on port: http://<IP_ADDRESS>:3000"}
{"level":"info","ts":1612787360.2054331,"caller":"executor/logging.go:30","msg":"Disabled etag and x-powered-by"}

If we move to a structured logger to output JSON and then some users want to see plaintext for debugging (frankly, I find the JSON hard to read), then the results are pretty ugly and still pretty difficult to read at a glance.
1.6127880326700268e+09	info	Forking	{"command": "node", "args": ["app.js"]}
1.6127880326701288e+09	info	Attached to	{"name": "stderr"}
1.6127880326701422e+09	info	Attached to	{"name": "stdout"}
1.612788032670306e+09	info	OperationalMode
	{"OperationalMode": "http"}
1.6127880326704435e+09	info	Timeouts	{"read": 10, "write": 10, "exec": 10}
1.6127880326704571e+09	info	Listening on
	{"port": 8080}
2021/02/08 12:40:32 Writing lock-file to: /tmp/.lock
2021/02/08 12:40:32 Metrics listening on port: 8081
1.6127880327378833e+09	dpanic	Ignored key without a value.	{"ignored": "alexellis.io started on port: http://<IP_ADDRESS>:3000"}
1.6127880327378604e+09	info	name	{"stdout": "msg"}
1.6127880327381146e+09	dpanic	Ignored key without a value.	{"ignored": "Disabled etag and x-powered-by"}
1.6127880327381005e+09	info	name	{"stderr": "msg"}
2021/02/08 12:40:35 GET / - 200 OK - ContentLength: 19686
2021/02/08 12:40:40 GET / - 200 OK - ContentLength: 19686

If we move to a structured logger to output JSON and then some users want to see plaintext for debugging (frankly, I find the JSON hard to read).

Totally agree that json destroys readability if you're looking at the logs directly. Our case we only want to have json where logs are never looked at directly but instead via an aggregator (newrelic-logging in our case).

@Voziv please can you improve the quality of your +1 here by stating what problem you're facing and which of the three outlined solutions would help you?

Absolutely. The main issue I came looking for is the ability to just have template/function logs passed through.
Current State:
2021/02/08 12:30:00 POST / - 200 OK - ContentLength: 16
2021/02/08 13:00:00 stdout: {"message":{} ,"level":"info"}
2021/02/08 13:00:00 stdout: {"message":"Requested function: <snip>", "level":"info"}

So what I can see is we have three different pieces logging

2021/02/08 12:19:22 Forking - node [app.js] - I think this is the of-watchdog image
POST / - 200 OK - ContentLength: 16 - I think this is the node template/express output?
{"message":{} ,"level":"info"} - The function ouput. Could be considered the same as the template output since the user in control of this.

In my current use case we really only care about the function output being pure json when deployed to the cluster. The other lines will get parsed just fine as-is. So we'd prefer the ability to strip the date/time prefix & stdout/stderr portion just leaving the functions raw output being logged.
While it would be nice for all 3 types of logs to be json to keep things consistent in the cluster, it's less important as the main benefit of json is being able to include extra metadata in the log entries that can be index and searched.

So would this line from the watchdog not break your log parser, if it's expecting a strongly typed JSON output?
2021/02/08 12:30:00 POST / - 200 OK - ContentLength: 16

cc @saginadir if you have time to give your 2c here

Cluster logs are injested from the container log files on the node. At least in the case of newrelic-logging they're parsed by individual lines so lines that are pure json (and likely has a requirement to have a date field + message field) will be parsed as json. Other plain text logs should parsed normally so it should be fine.
I could be wrong about this though.

@alexellis From my experience I need a log that looks like that:
{"level":"info","ts":1612788414.757807,"caller":"whatever.go:13", "method": "POST", "path": "/", "statusCode": 200, "ContentLength": 15}

For hobbiest it's a terrible experience to read JSON logs like you mentioned. But working in a big organization with a rigid DevOps team can sometimes make you jump through hoops in order to abide by their guidelines. For us, we were using ElasticSearch to aggregate the stdout output of all production nodes in k8s cluster. So I couldn't invoke the faas-cli logs nodeinfo --since 8h --format json command and needed the process itself to print outputs in pure JSON.
I think it's quite common today to have the entire log be printed in JSON, example with zap:
logger, _ = zap.Config{
    Encoding:    "json",
    Level:       zap.NewAtomicLevelAt(zapcore.DebugLevel),
    OutputPaths: []string{"stdout"},
}.Build()

logger.Info("This is an INFO message with fields", zap.String("region", "us-west"), zap.Int("id", 2))

Output: {"region":"us-west","id":2}
So at the time I would have greatly appreciate some env variable to pass to the watchdog node like LOG_ENCODING=json and would have the log formatter just print JSON instead of the standard log format. But ended up just hardcoding a fork of watchdog to print JSON.

Please see https://github.com/openfaas/of-watchdog/commit/51ea0d642955a011bd1f2f0be64319721de2f2e2 which adds a mechanism to disable prefixes to function logs.
This is option 3 (No prefix at all - pass-through for all lines)


Ctrl-C/Ctrl-V, Ctrl-Ins/Shift-Ins do not work in demo console

It is impossible to copy/paste instructions into Try it online console to try things. Context menu doesn't work for this either.

That's why I made it so you can click on any of the commands in the instruction to have it executed into the terminal session.
The javascript terminal library we're using doesn't do so well with copy/paste and I'm not sure that they'll be making much progress on that.

linuxcontainers.org website uses term.js: https://github.com/chjj/term.js
This is not maintained anymore, and as mentioned on the github page, the official successor is:
https://github.com/sourcelair/xterm.js
In lxd-webgui, i just exchanged term.js with the sourcelair xterm.js. Everything still worked perfectly, and i had paste support with ctrl-shift-v.
https://github.com/dobin/lxd-webgui/commit/14c4b74bbc738192e65cf3064f13ca390365f9fa

So I just tried to replace term.js with xterm.js and rendering is completely broken...

So if either of you want to volunteer to port github.com/lxc/linuxcontainers.org from term.js to xterm.js, I'd certainly appreciate it. Note that validating the setup is always a bit tricky and it took me a couple of weeks to get term.js to render perfectly on all the setups we care about:

Chrome on Linux
Chrome on Windows
Chrome on Android phone
Chrome on Android tablet
Firefox on Linux
Firefox on Windows

Especially making sure the things render properly in both very wide windows (fullscreen) and very narrow ones. Some of the font-sizing logic needed a fair amount of tweaking to convert the available screen estate in pixels to the number of columns for the terminal.

I'll have a look, as I anyway want to use lxd-demo-server for a private project.

I see that https://github.com/sourcelair/xterm.js is pretty active, so there are chances to fix rendering bugs upstream if somebody is going to report that. Are there any tests to check that is renders perfectly?

Well, I'm not sure that it's an actual upstream bug. It's just that @dobin said that it worked as a drop-in replacement for him, which it didn't for me and suggests there are a bunch of javascript and css changes I'd have to do to get things back to working order.
I don't believe that they pretend to be exactly compatible with term.js, though they mostly are since it's effectively a fork.

If it is a terminal, then all you need is a websocket for stdin and stdout of controlling terminal, which is the browser. Right?

Here is the alternative officially tested and maintained by Chromium team https://chromium.googlesource.com/apps/libapps/+/master/hterm/doc/ChangeLog.md
and how to embed
https://chromium.googlesource.com/apps/libapps/+/master/hterm/doc/embed.md


devfile push Integration test on kubernetes cluster

What type of PR is this?
/kind test
What does does this PR do / why we need it:
Runs the devfile delete integration test on kubernetes cluster.
Which issue(s) this PR fixes:
Fixes - Part of #2752
How to test changes / Special notes to the reviewer:
make test-cmd-devfile-push should run on kubernetes cluster in travis CI.

Blocked because of https://github.com/openshift/odo/issues/2877#issuecomment-621117106

Issue has been tracked in https://github.com/openshift/odo/issues/3096
/retest

https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/3041/pull-ci-openshift-odo-master-v4.2-integration-e2e/75#1:build-log.txt%3A748
/retest

I was following the changes since you started working on it. Looks good to me 👍
/lgtm

/retest

Rebased
/lgtm

/retest

prow failure
/retest

/test v4.4-integration-e2e

/refresh

/test v4.4-integration-e2e

/test v4.4-integration-e2e

Prow failure https://deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/3041/pull-ci-openshift-odo-master-v4.4-integration-e2e/397#1:build-log.txt%3A40
/retest

/refresh

Seeing failure on prow : https://deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/3041/pull-ci-openshift-odo-master-v4.2-integration-e2e/408
@amitkrout @mohammedzee1000 As per my change OpenShift CI is failing because we are triggering the test from the test container registry.svc.ci.openshift.org/openshift/release:golang-1.13 where we do not have KUBECONFIG. Can you please suggest how to fix it.

@amitkrout @mohammedzee1000 As per my change OpenShift CI is failing because we are triggering the test from the test container registry.svc.ci.openshift.org/openshift/release:golang-1.13 where we do not have KUBECONFIG. Can you please suggest how to fix it.

The problem is quite opposite. the $KUBECONFIG is set there - https://deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/3041/pull-ci-openshift-odo-master-v4.2-integration-e2e/408#1:build-log.txt%3A43

@priti You can check this function, its check for existence though but consider all  cases. https://github.com/openshift/odo/blob/99f1a2f84fef58c2aafbce9a9bc44e47f7e63fe8/pkg/util/util.go#L976

/retest

@kadel @amitkrout @mohammedzee1000 After copying the [KUBECONFIG](https://deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/3041/pull-ci-openshift-odo-master-v4.2-integration-e2e/440#1:build-log.txt%3A44) file to temporary config I am hitting a weird failure on prow. Though the changes works fine on travis CI.
Not sure why is it happening on openshift CI. Can you please suggest what is the issue about and how to fix it?

Not sure why is it happening on openshift CI. Can you please suggest what is the issue about and how to fix it?

it is happening because you are using wrong KUBECONFIG, you need to use the one that $KUBECONFIG points to

@prietyc123 Can you please revert the implementation you made for syncing the test code to run against prow. BTW test code sync will be the part of #3203.
Let unblock the this pr and yours other devfile prs too. cc_ @kadel

Can you please revert the implementation you made for syncing the test code to run against prow.

I have reverted my changed and skipped setting KUBECONFIG to a temporary config file on prow due to https://github.com/openshift/odo/issues/3203

prow failure
/retest

/retest

/retest

External issue https://deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gcs/origin-ci-test/pr-logs/pull/openshift_odo/3041/pull-ci-openshift-odo-master-v4.4-integration-e2e/865#1:build-log.txt%3A573
/retest

/retest

/approve

/lgtm

https://github.com/openshift/odo/issues/3122
/test v4.3-integration-e2e


add definitions for Ne and LE.

I think it would sense to add NewDefinition LE and NewDefinition Ne to the correct levels along with some DefinitionDoc about them.

cleanup: done.


feat(datepicker): infinite scrolling through months & years

Add ability to scroll through months and years on the calendar

+1

+1


BinaryJedisCluster pexpireAt bug

Source code
BinaryJedisCluster.java：
@Override
  public Long pexpireAt(final byte[] key, final long millisecondsTimestamp) {
    return new JedisClusterCommand<Long>(connectionHandler, maxAttempts) {
      @Override
      public Long execute(Jedis connection) {
        return connection.pexpire(key, millisecondsTimestamp);
      }
    }.runBinary(key);
  }

Expected behavior
It should call connection.pexpireAt,instead of connection.pexpire.
Actual behavior
After call connection.pexpire,the expiration date is very long
Redis / Jedis Configuration
Jedis version: jedis-2.9.0.jar
Spring data redis version:  spring-data-redis-1.8.4-RELEASE.jar
Redis version:Redis 3.2.8
Java version: jdk 1.8.0_144
I'd like to discuss this with you. I think it's a bug.
中国开发者请阅读我的博客一起分析该问题：redis cluster环境下jedis expireAt命令bug探究
Stack Overflow：https://stackoverflow.com/questions/46725457/jedis-binaryjediscluster-pexpireat-bug

It's fixed at #1497


Add prefix to CSS

And maybe also remove the CSS variable support

Done https://github.com/corellm/corellm/commit/920cb758e21fec83acd9f2b0509e379ca7be52d1


sometime soon jsonclient needs to be separated from CIAPI.CS in nuget

We already compile a separate NuGet package for JsonClient - see http://ci.labs.cityindex.com:8080/job/CIAPI.CS/lastSuccessfulBuild/artifact/dist/
I don't think its published to NuGet.org though.


Monument text got stuck when I pressed the keyboard repeatedly really fast to get through quickly

oops, wrong repo! :cold_sweat:


[WIP] baremetal: Explicitly advertise the host IP chosen by kubelet

When using the baremetal infrastructure platform, the networking architecture
includes 3 Virtual IP addresses (VIPs).  For more information about the use
of these VIPs, see this document:
https://github.com/openshift/installer/blob/master/docs/design/baremetal/networking-infrastructure.md
By default, the kube-apiserver will advertise an IP address that it determines
from the default network interface.  When running in IPv4, we have been getting
lucky that the VIPs were never chosen.  They are always listed after the primary
address when you observe with "ip", and the same order is reflected in golang
when it iterates available IP addresses.
When we starting running IPv6 clusters, we observed kube-apiserver often
advertising one of the VIPs instead of the permanent IP address for the
interface.  This is bad because the VIP can move between servers.
These IPv6 VIPs are "deprecated" addresses, and we observed them listed
first in the list of addresses for an interface.  For more info on "deprecated"
IPv6 addresses, see:
http://www.davidc.net/networking/ipv6-source-address-selection-linux
Unfortunately, this attribute is not accessible using golang's net package.
Accessing this attribute from golang would require rewriting the relevant
code for Linux to use the netlink interface directly.  This is far from a
trivial change, so we seek a simpler workaround in the meantime.
This same problem exists with kubelet and which IP address it chooses for
a master Node.  That was resolved with some scripts that filter out VIPs
and then specify the correct IP address directly as a command line argument.
For more information on the related kubelet workaround, see:
https://github.com/openshift/machine-config-operator/pull/1444
This PR builds on what was done for kubelet, and explicitly tells
kube-apiserver to advertise the address that we already determined was
correct for kubelet.
The reason this is marked as WIP is because we probably only want this
workaround for the baremetal infrastructure platform, and some guidance is
requested on how the operator maintainers would like to see that done.

WIP because I haven't figured out how to make this specific to the baremetal platform yet

/retest

Sgtm.

The reason this is marked as WIP is because we probably only want this
workaround for the baremetal infrastructure platform, and some guidance is
requested on how the operator maintainers would like to see that done.

I don't think it needs to be conditional. All kube-apiservers in all OCP clusters should be advertising the IP address that their node declares as its primary IP. This PR should be a no-op on any non-ipv6-bare-metal OCP system.

@

The reason this is marked as WIP is because we probably only want this
workaround for the baremetal infrastructure platform, and some guidance is
requested on how the operator maintainers would like to see that done.

I don't think it needs to be conditional. All kube-apiservers in all OCP clusters should be advertising the IP address that their node declares as its primary IP. This PR should be a no-op on any non-ipv6-bare-metal OCP system.

@sttts you ok with this?  It did pass e2e-aws, at least ...

/hold
this approach will break other operators which need to create a pod definition to run on multiple nodes that requires those pods communicate with each other. The canonical example is etcd using IP based communication.

A solution that determines which value is the correct value based on the node object and creates an observed configuration will allow all the use-cases we know we have to function.

To clarify - are you suggesting that we may hit this problem in other places?  and not that this particular change is going to break something?

To clarify - are you suggesting that we may hit this problem in other places? and not that this particular change is going to break something?

This change is bypassing a symptom of a general problem with a specific solution that isn't suited to all operators.  If you don't have a general solution for determining which IP address is correct, then you will break in other operators like etcd.

To clarify - are you suggesting that we may hit this problem in other places? and not that this particular change is going to break something?

This change is bypassing a symptom of a general problem with a specific solution that isn't suited to all operators. If you don't have a general solution for determining which IP address is correct, then you will break in other operators like etcd.

Our solution right now was to fix it to ensure that the Node address is correct.  Anything that uses the address from the Node will be fine.  It's only things that inspect the host directly that may or may not get it right.
If the etcd operator is getting the IPs from the Node, we're fine.  If it isn't, we may have to fix something there too.

Our solution right now was to fix it to ensure that the Node address is correct. Anything that uses the address from the Node will be fine. It's only things that inspect the host directly that may or may not get it right.
If the etcd operator is getting the IPs from the Node, we're fine. If it isn't, we may have to fix something there too.

Will the node have a single internal IP address for us to choose?  If not, are we safe to choose the first?  If so, then I'm ok releasing this PR because the information is available externally via the resource.

Our solution right now was to fix it to ensure that the Node address is correct. Anything that uses the address from the Node will be fine. It's only things that inspect the host directly that may or may not get it right.
If the etcd operator is getting the IPs from the Node, we're fine. If it isn't, we may have to fix something there too.

Will the node have a single internal IP address for us to choose? If not, are we safe to choose the first? If so, then I'm ok releasing this PR because the information is available externally via the resource.

Yes, the Node has a single InternalIP on this platform now.  Here's a sample from a running cluster:
  status:
    addresses:
    - address: fd2e:6f44:5dd8:c956::14
      type: InternalIP
    - address: master-0.ostest.test.metalkube.org
      type: Hostname

Will the node have a single internal IP address for us to choose? If not, are we safe to choose the first?

In the bare-metal case, there will always be a single Internal IP. (kubelet enforces that.) In the cloud case there may be more than one but we already take steps to ensure that the first one is the one you want.

Will the node have a single internal IP address for us to choose? If not, are we safe to choose the first?

In the bare-metal case, there will always be a single Internal IP. (kubelet enforces that.) In the cloud case there may be more than one but we already take steps to ensure that the first one is the one you want.

thanks
/hold cancel

/lgtm

/approve

/cherry-pick release-4.4


Adding the get_side_acivated function in pnor image flash procedure.

By using this patch, we will get BMC and PNOR Version side info(Golden or Primary) before and after
PNOR FW Flash.

This change is

Review status: 0 of 2 files reviewed at latest revision, 2 unresolved discussions.

ci/source/op_ci_bmc.py, line 234 [r1] (raw file):
function name suggests that you are only getting the active sides and not validating anything

ci/source/op_ci_bmc.py, line 238 [r1] (raw file):
the output is l_bmc_side, l_pnor_side. You need to address 2 outputs from the calling function

Comments from Reviewable

Review status: 0 of 3 files reviewed at latest revision, 2 unresolved discussions.

bvt/op-opal-ci.xml, line 98 [r2] (raw file):

        <test>
            <testcase>
                <cmd>op-ci-bmc-run "op_ci_bmc.validate_side_activated()"</cmd>

wondering why you would need the same test twice?

Comments from Reviewable

We hitted the issue https://github.com/open-power/op-build/issues/470 during flashing of pnor and while running opal tests, so next time we make sure system is in primary side Before and after pnor flash. So that we will acknowledged with side of pnor before and after flashing.

Reviewed 3 of 3 files at r2.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable


Normalised the Tags on the templates page

Hiya
Normalised the Tags on the templates page so there are no dupes, but had to do away with the lowercase and capitalize code in the JS to facilitate the correct naming of tags.

Sorry bit messy - didn't realise my fork was so out of date.

firstly, can you use the old whitespace setting for YAML. No new submitters are going to do the aligned version, so the file will look inconsistent.
second, I don't think we can rely on submitters to use an "approved" casing and naming of tags. Rather, we should be writing code to make sure they match up even if it doesn't match well (de-duping and making case intensitive). We can do this on either the MM or JS side.

Yeah can do either way, sorry my text editor setting just aligns it how I like.
Tabs - are a little inconsistent in the doc, but I think the [ '' , '' ] works better - agreed ?
Yeah the case is a problem, it is broken at the moment unfortunately - for instance
https://directory.middlemanapp.com/#/templates/semantic ui
https://directory.middlemanapp.com/#/templates/pattern library
So thought the fix I did is better than nothing at the moment.


Consider removing enum APIRequest

This could probably be a lot simpler if a Request had an additional constructor and serialisation logic, removing the need for the enum.

Removed in favour of an easier automated system, implemented as of 039f8732953d06163d4afacd6ced316c614c5ca6


Resize produces incorrect results

The relay reshape function produces incorrect results. For scale factors other than 2, the output of relay.resize does not match the testing function.
The test I changed:
--- a/tests/python/relay/test_op_level5.py
+++ b/tests/python/relay/test_op_level5.py
@@ -66,7 +66,7 @@ def test_resize():
tvm.testing.assert_allclose(op_res.asnumpy(), ref_res, rtol=1e-4, atol=1e-6)
for method in ["bilinear", "nearest_neighbor"]:
for layout in ["NHWC", "NCHW"]:
-            verify_resize((1, 4, 4, 4), 2, method, layout)
+            verify_resize((1, 4, 4, 4), 7, method, layout)
In this particular case, we get this error:
   AssertionError:
   Not equal to tolerance rtol=0.0001, atol=1e-06

   Mismatched elements: 832 / 3136 (26.5%)
   Max absolute difference: 0.8489792
   Max relative difference: 711.5619
    x: array([[[[0.507647, 0.792792, 0.245963, 0.439271],
            [0.507647, 0.792792, 0.245963, 0.439271],
            [0.507647, 0.792792, 0.245963, 0.439271],...
    y: array([[[[0.507647, 0.792792, 0.245963, 0.439271],
            [0.507647, 0.792792, 0.245963, 0.439271],
            [0.507647, 0.792792, 0.245963, 0.439271],...

cc @mbrookhart

It looks like it's failing with NHWC `nearest_neighbor. Looking at git blame, it seems @masahi worked on the test utility and @yongwww worked on the topi op. Could one or both of you guys take a look at this?

Using align_corner option with nearest is a mistake: https://github.com/apache/incubator-tvm/blob/master/tests/python/relay/test_op_level5.py#L56
For nearest, I think the coordinate transform option should always be assymetric. The following should fix the test. The same fix should be applied to the dynamic one added in https://github.com/apache/incubator-tvm/pull/6198.
coord_trans = "align_corners"
if method == "nearest_neighbor":
    coord_trans = "asymmetric"

z = relay.image.resize(x, size, layout, method,
                       coordinate_transformation_mode=coord_trans)

Proposed fix is in #6298


feat!: require @angular/compiler v15

Codecov Report
Base: 100.00% // Head: 100.00% // No change to project coverage :thumbsup:

Coverage data is based on head (47b2be7) compared to base (ac870f0).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #236   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            5         5
  Lines          330       330
  Branches        87        87
=========================================
  Hits           330       330

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Will userDidProvideCreditCardInfo method returns nil CardIOCreditCardInfo object?

Hi,
Will below delegate method returns nil CardIOCreditCardInfo object?

(void)userDidProvideCreditCardInfo:(CardIOCreditCardInfo *)cardInfo inPaymentViewController:(CardIOPaymentViewController *)paymentViewController;

I haven't encountered any scenarios.

@vijayviswas that delegate method is called only after a successful scan. cardInfo will never be nil.


feat: prototype SQL API

This is a basic prototype for SQL queries.  I'm a bit torn if it's useful.  I think we can soon support datafusion-python.  Once we can then users will have the choice of both datafusion-python and duckdb.  I'm not sure if there's value in also maintaining a simple no-dependency endpoint for SQL queries.

Codecov Report
Attention: Patch coverage is 20.00000% with 20 lines in your changes missing coverage. Please review.

Project coverage is 77.89%. Comparing base (f257489) to head (faedd7e).
Report is 2 commits behind head on main.

Files with missing lines
Patch %
Lines

rust/lance/src/datafusion/sql.rs
0.00%
20 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3132      +/-   ##
==========================================
- Coverage   77.91%   77.89%   -0.03%
==========================================
  Files         240      241       +1
  Lines       81564    81479      -85
  Branches    81564    81479      -85
==========================================
- Hits        63550    63465      -85
- Misses      14806    14838      +32
+ Partials     3208     3176      -32

Flag
Coverage Δ

unittests
77.89% <20.00%> (-0.03%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[aws mac1.metal] buildkite agents not getting ec2 metadata tags

Environment

using buildkite-agent v.3.29
have tags-from-ec2-tags=true and tags-from-ec2 both set to true and wait-for-ec2-tags-timeout=300s in config.
launching the agent via launchctl on mac1.metal ec2 instance (Big Sur).

Issue
buildkite-agent is set to launch when machine boot. it however does not get the ec2 metadata tags (aws:instance-id, aws:instance-type...etc). Looking at the log I see that it did indeed wait for ec2-tags (the tag I set) but not the metadata tags. fetch retried 5 times then gave up
2021-04-24 00:37:23 WARN   RequestError: send request failed
caused by: Get "http://<IP_ADDRESS>/latest/meta-data/instance-id": dial tcp <IP_ADDRESS>:80: connect: network is unreachable (Attempt 5/5 Retrying in 1.465s)
2021-04-24 00:37:24 ERROR  Failed to fetch EC2 meta-data: RequestError: send request failed

If I unload the daemon and reload it (restart the agent) then I get all the tags. I am suspecting ec2-metadata tags was not ready yet during fetch)
Ask
Is there an equivalent of  wait-for-ec2-tags-timeout for metadata tags? seems like the timeout only applies to ec2 tags

Sorry @OliverKoo, it looks like we only use that timeout for the resource tags:
https://github.com/buildkite/agent/blob/693b319b4b892e84e2e66daece8a84acd4d1c053/agent/tags.go#L150
I think it'd be totally reasonable to add an equivalent setting for the ec2 meta-data too. If you're keen, you could follow the same pattern and apply it here and submit a pull request:
https://github.com/buildkite/agent/blob/693b319b4b892e84e2e66daece8a84acd4d1c053/agent/tags.go#L103
Otherwise we'll get to it when we can. :-)

Here ya go - https://github.com/buildkite/agent/pull/1425


Add clean_text_or_reply util.

Closes #100.
Does what the title says.
This util function will clean text if provided, else try and clean the referenced message, or if text isn't provided and was unable to get a reference message will return None.
This is useful for commands such as lancebot's .uwu and .randomcase where users can either provide a text argument, or reply to a message and have the bot use the replied message's content.
NB: Currently a draft as untested

This has now been tested and it works :+1:
Sample code usage:

Result of said sample:

This is now ready for review :smile:


DoNotDisturb.isEnabled isn't working on monterey

Hello,
It looks like DoNotDisturb.isEnabled always returns false on Monterey, it seems like a bug.

I am getting getDoNotDisturb as FALSE all the time for Ventura as well.

I am getting getDoNotDisturb as FALSE all the time for Ventura as well,
      log.info(require('macos-notification-state').getDoNotDisturb())
      log.info(require('macos-notification-state').getNotificationState())
      log.info(require('macos-notification-state').getSessionState())

Always returns
11:49:00.722 › false
11:49:00.724 › SESSION_ON_CONSOLE_KEY
11:49:00.726 › SESSION_ON_CONSOLE_KEY

Readme doesn't mention any requirements but in PR I can see, which is unclear whether it's for apps using this or for the this package?
// Requirements for make it work:
// 1. add entitlement <key>com.apple.developer.usernotifications.communication</key><true/>
// 2. create ProvisioningProfile (if have not yet) - allow Communication Notification service access
// 3. "NSFocusStatusUsageDescription": "some description", - in order to show Request access to FocusStatus
// note: for reset use "tccutil reset FocusStatus".

@nbashkankov could you please shed a light as you created the PR? It would be much appreaciated :)

sure. as written in comments - you need to add several things to your application (Electron based) in order to support MacOS > 12

add entitlement com.apple.developer.usernotifications.communication into plist file - this file mentioned in one of electron-builder config "mac" sections "entitlements" or "entitlementsInherit" (depends on what you use)
add NSFocusStatusUsageDescription key with some text at extendInfo section in electron-builder config
in developer.apple.com your bundleId should have access to Communication Notification service
and also you should create provisioning profile for that bundleId
include created provisioning profile as well in electron-builder config

so after that your application should able to access
For Monterey - it will check values in this location "~/Library/DoNotDisturb/DB/" - there is several jsons which updated by Os

Thanks, this should be in readme somewhere.
Many people don't have Apple Developer Account so this won't work for us :(

you dont need many Account - if your application is somehow distributed you should have at least one where you define bundleId and created Certificate for codesign - without it you cannot normally distribute app

@nbashkankov not sure if I understand "cannot distribute normally", I guess you mean Apple Store,  but:

I don't have Apple developer account and I don't sign the app. I build on github actions
I redistribute my app via brew cask https://formulae.brew.sh/cask/stretchly#default
people that don't use brew, need to do some manual steps to run the app https://github.com/hovancik/stretchly#application-signing

There are many apps like that


problems with spring boot project using gradle

I use intelliJ to create springboot project，the created project run with no problems when i choose build with maven.  If i choose build with gradle, the created project can run , but the following problems occur:

application.properties where i change the http port seems doesn't work (\spring\boot\src\main\resources\application.properties)
properties file does work when i move it to the root （\spring\boot\application.properties）
the build directory only has classes but no resources，and the application doesn't work with static resources.

I compare the maven project and the gradle project which both are generated originally by the intelliJ, the gradle project has a submodule where the problems may be, something relaed to the resource folder setting. I can't figure it out now。

I use:
intelliJ 2019.2.1
springboot 2.2.0.RELEASE

I'm not an IDEA user but isn't that expected? You have to ask IDEA to "build" the project before it changes the classpath.

Updated from 2018 to 2020 IntelliJ and it resolved it for me


hyphen in nodir clues?

Relates to #40 (well actually was part 3 in that)
I realized it is a bit strange question for the generic "nodir", but for my only use of "nodir" it makes sense -- the "turning words".
So ideal this should be my suggested behaviour:
The same way of notation for hyphens Example (9-1-3), but the hyphen should only be placed if the next square is beside the "current" square.
So in my 9x10: https://fripp.changeip.org/melodikryss/melodikryss2/lost2.html
#a7 #b7 #c7 #d7 #e7 #f7 #g7 #h7 #i7 #j7 #j6 #j5 #j4 [] (9-1-3)
(The long turning word with number 8 if you check the link.)
I would like to have two hyphens -- one across and one down.
I understand it would perhaps give a lot of extra work I am the only one asking for... but it would be elegant :-)

An alternative implementation. Not at all that elegant, but it should do for me!
It is that it should be possible to add hyphens in "deleted" clues.
The same example (https://fripp.changeip.org/melodikryss/melodikryss2/lost2.html)
I use
exolve-down:
#j7 [] *
To skip "down only-part" of my long turning word to get pinkish...
If I write  #j7 [] (1-3) *
the hyphens works perfect, (because it is not handled "deleted").
So if you are in no mood for the real implementation of this esoteric demand perhaps you could make #j7 [] (1-3) *
place the hyphes, but still act as deleted

Not sure I understand fully with a quick read, but I will try to do something for this bug as a top priority, in the next version that I put out :-). Thanks for your patience!

I added the ability to force hyphens/bars to the right of or below any cell, with v0.95.
Eg.:
  exolve-force-hyphen-right: a5 c4

Please see for more details.
https://github.com/viresh-ratnakar/exolve/blob/master/README.md#exolve-force-hyphen-right-exolve-force-hyphen-below-exolve-force-bar-right-exolve-force-bar-below
Further, the way to link nodir clues is like this (this was already the case):
   exolve-nodir:
    #a7 #b7 #c7 #d7 #e7 #f7 #g7 #h7 #i7 #j7 #j6 #j5 #j4 [8] See 9.
    #d4 #e4 #f4 #g4 #h4 #h3 #h2 #h1 [13] See 9.
    #a5 #a4 #a3 #a2 #a1 #b1 #c1 [9], [13], [8] Clue

Fantastico! Thanks. Embarrasing that I didn't  understand  the "See-concept"


Fix unwrapping peft models

Fixes issues discussed here.
cc @younesbelkada

Hi @kkteru
Can you merge this branch with main branch to make sure tests pass?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Interesting, somehow the test_sft_trainer.py tests failed on latest main branch (without any changes from this branch) in my local. Investigating further.

@kkteru what are the failed tests? might be unrelated to your changes I think - you can just merge your branch with upstream main and see here if the tests pass

These ones fail both on the main branch and this branch (merged with main) on my system. I just rebased on top of latest main and pushed. I dont see the tests running here?
=================================================================== short test summary info ====================================================================
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer - AssertionError: False is not true
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_infinite_with_model - AssertionError: False is not true
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_infinite_with_model_epochs - AssertionError: False is not true
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_with_model - AssertionError: False is not true
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_with_model_num_train_epochs - AssertionError: False is not true

The tests were failing because you had an old version of transformers, installing the 4.35.0 will fix your isses! As you can see, CI is green now :D


Fixes #1670

This pull requests fixes issue #1670 and returns an object of following format, when using the list_things method on the AWS IoT service:
{
"things": [
        {
            "thingName": "string",
            "thingTypeName": "string",
            "thingArn": "string",
            "attributes": {
                "string": "string"
            },
            "version": 123
        }
    ]
}

Coverage decreased (-0.2%) to 92.118% when pulling afa6ae288d7f0d8865c8050c436cc1e2f0c2da72 on sthuber90:master into<PHONE_NUMBER>4ee78affc07643d16fae6bb31b4014 on spulec:master.

I don't see why the build fails as here everything was build correctly

Ah, looks like we aren't freezing time for some test. I'll get that fixed separately.
Thanks for your help!


No way to get status_code or error

There is no way to get the status_code of a response or the error string because the response given only includes the subresponses

Replacing These lines with something like the code below helps, but I was running into some issues when I tested it and am busy right now
@resonse = @response.to_hash.delete('returns')
@response = decoded_resp.merge(@response)

I'll add the status_code and error in, going to leave out the authentication information unless there's demand for it (it's in bytes anyhow).

Okay, thanks

Commit 8f3caa91adb07c1e636dd9e9b71cfec4ae0bde5b fixes this issue, new tag is 0.0.4 with updated readme/gpsoauth and response.
You can access the status code and error using call.response.status_code and call.response.error (if the error is empty then assume there's no error).


Merge develop add scenario

The new branch merge-develop-add_scenario is the result of resolving all the conflicts of merging add_scenario into develop.

this should be a merge back to add_scenario


Sampling Strategies settings not updating in Collector without pod restart

I am deploying Jaeger via the Operator using the following Jaeger CR with sampling options shown below:
# https://www.jaegertracing.io/docs/1.21/operator/#external-elasticsearch
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: simple-prod
  namespace: monitoring
spec:
  strategy: production
  ingress:
    enabled: true
    hosts:
      - jaeger-query.monitoring-container-dev
  sampling:
    options:
      service_strategies:
      - service: samplewebapplication
        type: probabilistic
        param: 0.5
      default_strategy:
        type: probabilistic
        param: 0.5
        operation_strategies:
        - operation: "/health"
          type: probabilistic
          param: 0
        - operation: "/metrics"
          type: probabilistic
          param: 0
        - operation: "/healthz"
          type: probabilistic
          param: 0
  storage:
    type: elasticsearch
    options:
      es:
        server-urls: https://tracing-es-http.monitoring.svc:9200
        index-prefix: monitoring
        tls:
          ca: /es/certificates/ca.crt
    secretName: jaeger-secret  # Secret which defines environment variables ES_PASSWORD and ES_USERNAME. Created by: kubectl create secret generic jaeger-secret --from-literal=ES_PASSWORD=changeme --from-literal=ES_USERNAME=elastic
    esIndexCleaner:
      enabled: true            # turn the cron job deployment on and off
      numberOfDays: 7          # number of days to wait before deleting a record
      schedule: "55 23 * * *"  # cron expression for it to run
  volumeMounts:
    - name: certificates
      mountPath: /es/certificates/
      readOnly: true
  volumes:
    - name: certificates
      secret:
        secretName: tracing-es-http-certs-public

My Jaeger Agent then picks up the sampling strategies and I can confirm the sampling strategies are being read correctly by hitting the Agent's sampling strategy service via an HTTP GET to http://localhost:5778/sampling?service=samplewebapplication:
{
    "strategyType": "PROBABILISTIC",
    "probabilisticSampling": {"samplingRate": 0.5},
    "operationSampling": {
        "defaultSamplingProbability": 0.5,
        "defaultLowerBoundTracesPerSecond": 0,
        "perOperationStrategies": [
            {
                "operation": "/health",
                "probabilisticSampling": {"samplingRate": 0}
            },
            {
                "operation": "/metrics",
                "probabilisticSampling": {"samplingRate": 0}
            },
            {
                "operation": "/healthz",
                "probabilisticSampling": {"samplingRate": 0}
            }
        ],
        "defaultUpperBoundTracesPerSecond": 0
    }
}

However, if I update the Jaeger CR spec to add or remove one of my sampling strategy settings, the Collector and/or Agent don't seem to pick up the new settings until I restart the Collector pod (I've done so via kubectl delete pod simple-prod-collector-asdf for example).
Is this expected or am I just not being patient enough for some refresh time interval to expire and the Collector re-reads the sampling strategy settings?

Is this expected or am I just not being patient enough for some refresh time interval to expire and the Collector re-reads the sampling strategy settings?

There are a couple of things that might be needed:

[ ] check that the collector will reload the strategies file when it's changed. I remember seeing a PR bringing this to the collector, just not sure if it's already there.
[ ] check that the config map gets updated with the changed configuration
[ ] check that the file mounted in the container based on the config map has changed

Those three pieces need to be working for the reload to happen. If you are able to confirm that, it would speed up a resolution/implementation of this. Otherwise, I can check it myself in a couple of weeks.

I just tested to check some of those things:

[x]  check that the config map gets updated with the changed configuration - The config map is indeed updated when I kubectl apply an updated Jaeger CR spec
[x]  check that the file mounted in the container based on the config map has changed - The file associated with the config map (/etc/jaeger/sampling/sampling.json) is indeed updated when I kubectl apply an updated Jaeger CR spec

Also, my version info for my Jaeger CR (via kubectl describe jaeger simple-prod):
Status:
  Phase:    Running
  Version:  1.21.0

I found the PR I was thinking about: https://github.com/jaegertracing/jaeger/pull/2188 . Looks like, a flag is needed in order to reload the config file: --strategies-reload-interval. You should be able to set this in the CR, under the collector's options.

a flag is needed in order to reload the config file: --strategies-reload-interval. You should be able to set this in the CR, under the collector's options.

Any idea of the syntax on that? The reference for the CR options is scant. My guess (probably not accurate and not sure how to confirm):
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: simple-prod
  namespace: monitoring
spec:
  strategy: production
  ingress:
    enabled: true
    hosts:
      - jaeger-query.monitoring-container-dev
  collector:
    options:
      strategies_reload_interval: 50ms

I figured it out through some trial and error:
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: simple-prod
  namespace: monitoring
spec:
  strategy: production
  ingress:
    enabled: true
    hosts:
      - jaeger-query.monitoring-container-dev
  collector:
    options:
      sampling:
        strategies-reload-interval: 30s

Also confirmed the reload works. Thanks! I'll close this issue.


Deploy of Nextjs app creates CloudFront Distribution, subsequent sst dev deletes it

Getting a cycle of creating of the CloudFront Distribution when sst deploying and then it gets deleted by a subsequent sst dev next dev.  This seems to also turnover all of the related resources, such as the S3 bucket and logs.
Relevant log lines are:

During deploy, this is logged as Creating: DashboardCDNWaiter... and takes ~4 minutes to complete
During dev, this is logged as Deleting: DashboardCDN... and takes ~6 minutes to complete

The churn isn't helpful because I would have expected the distribution resources to be retained as they shouldn't conflict with the resources needed for dev, but the duration of these updates, even if necessary for the underlying CF resource creation/deletion, severely impacts DX.
IMPORTANT: Because the CF distribution gets recreated, the <random> component of the the automatically generated domain name changes each time, which would affect things like Auth where the provider uses a hardcoded callback URL.
This is a fresh Nextjs app with a single page.  I have been encountering this problem in my other in progress app, but it seemed to stabilise after I added a custom domain for the distribution, so this problem might only be for auto-generated domains, e.g. <random>.cloudfront.net.

Initial Deploy phase
/Users/dmeehan/.nvm/versions/node/v20.15.1/bin/npm run deploy

><EMAIL_ADDRESS>deploy
> sst deploy

SST ❍ ion 0.1.4  ready!

➜  App:        EasyL8
   Stage:      dmeehan

~  Deploying

|  Log         Running "npx --yes<EMAIL_ADDRESS>build" script
|  Log         Next.js version : 14.2.5
|  Log         OpenNext v3.0.2
|  Log         ┌─────────────────────────────────┐
|  Log         │ OpenNext — Building Next.js app │
|  Log         └─────────────────────────────────┘
|  Log         ><EMAIL_ADDRESS>build
|  Log         > next build
|  Log           ▲ Next.js 14.2.5
|  Log            Creating an optimized production build ...
|  Log          ✓ Compiled successfully
|  Log            Linting and checking validity of types ...
|  Log            Collecting page data ...
|  Log            Generating static pages (0/5) ...
|  Log            Generating static pages (1/5)
|  Log            Generating static pages (2/5)
|  Log            Generating static pages (3/5)
|  Log          ✓ Generating static pages (5/5)
|  Log            Finalizing page optimization ...
|  Log            Collecting build traces ...
|  Log         Route (app)                              Size     First Load JS
|  Log         ┌ ○ /_not-found                          871 B          87.9 kB
|  Log         └ ○ /dashboard/applications              141 B          87.2 kB
|  Log         + First Load JS shared by all            87 kB
|  Log           ├ chunks/23-b75664ace61c0abb.js        31.5 kB
|  Log           ├ chunks/fd9d1056-2821b0f0cabcd8bd.js  53.6 kB
|  Log           └ other shared chunks (total)          1.86 kB
|  Log         ○  (Static)  prerendered as static content
|  Log         ┌──────────────────────────────┐
|  Log         │ OpenNext — Generating bundle │
|  Log         └──────────────────────────────┘
|  Log         Bundling middleware function...
|  Log         Bundling static assets...
|  Log         Bundling cache assets...
|  Log         Building server function: default...
|  Log         Bundling revalidation function...
|  Log         Bundling image optimization function...
|  Log         Bundling warmer function...
|  Log         OpenNext build complete.
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEvents sst:aws:Queue
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeeder sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardAssets sst:aws:Bucket
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsId random:index:RandomId
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederLogGroup aws:cloudwatch:LogGroup
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederCode aws:s3:BucketObjectv2
|  Created     Dashboard sst:aws:Nextjs → DashboardOriginAccessIdentity sst:aws:OriginAccessIdentity (1.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsBucket aws:s3:BucketV2 (3.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsPublicAccessBlock aws:s3:BucketPublicAccessBlock
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsCors aws:s3:BucketCorsConfigurationV2
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsPolicy aws:s3:BucketPolicy (4.9s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationTable aws:dynamodb:Table (24.5s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsQueue aws:sqs:Queue (26.5s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederRole aws:iam:Role (2.3s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefault sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouue sst:aws:QueueLambdaSubscriber
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizer sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunction sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultLogGroup aws:cloudwatch:LogGroup
|  Created     Dashboard sst:aws:Nextjs → DashboardServerCachePolicy aws:cloudfront:CachePolicy (1.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionLogGroup aws:cloudwatch:LogGroup
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerLogGroup aws:cloudwatch:LogGroup (1.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionCode aws:s3:BucketObjectv2 (2.6s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerRole aws:iam:Role (3.6s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultRole aws:iam:Role (4.0s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionRole aws:iam:Role (4.1s)
|  Created     Dashboard sst:aws:Nextjs → DashboardCloudfrontFunctionServerCfFunction aws:cloudfront:Function (7.9s)
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetFiles sst:aws:BucketFiles (8.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardCdn sst:aws:CDN
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederFunction aws:lambda:Function (15.3s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionFunction aws:lambda:Function (11.9s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultFunction aws:lambda:Function (12.3s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerFunction aws:lambda:Function (12.7s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederCodeUpdater sst:aws:FunctionCodeUpdater (2.4s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultUrl aws:lambda:FunctionUrl (1.9s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerUrl aws:lambda:FunctionUrl (1.9s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionCodeUpdater sst:aws:FunctionCodeUpdater (2.7s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeed aws:lambda:Invocation (2.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardCdnDistribution aws:cloudfront:Distribution (6.1s)
|  Created     Dashboard sst:aws:Nextjs → DashboardInvalidationPolicy aws:iam:Policy (1.3s)
|  Created     Dashboard sst:aws:Nextjs → DashboardInvalidation sst:aws:DistributionInvalidation (1.3s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueEventSourceMapping aws:lambda:EventSourceMapping (18.4s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerCode aws:s3:BucketObjectv2 (36.5s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerCodeUpdater sst:aws:FunctionCodeUpdater (2.6s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultCode aws:s3:BucketObjectv2 (42.5s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultCodeUpdater sst:aws:FunctionCodeUpdater (2.0s)
|  Created     Dashboard sst:aws:Nextjs → DashboardInvalidationPolicyAttachment70e6 aws:iam:RolePolicyAttachment
|  Created     Dashboard sst:aws:Nextjs → DashboardCdnWaiter sst:aws:DistributionDeploymentWaiter (236.6s)
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerCodeUpdater sst:aws:FunctionCodeUpdater
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerFunction aws:lambda:Function
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerCode aws:s3:BucketObjectv2
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerLogGroup aws:cloudwatch:LogGroup
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerRole aws:iam:Role (1.6s)
|  Deleted     Dashboard sst:aws:Nextjs → DashboardDevServer sst:aws:Function

✓  Complete
   Dashboard: https://<REDACTED>eeg54u.cloudfront.net

Process finished with exit code 0

Dev phase
/Users/dmeehan/.nvm/versions/node/v20.15.1/bin/npm run dev

><EMAIL_ADDRESS>dev
> sst dev next dev

SST ❍ ion 0.1.4  ready!

➜  App:        EasyL8
   Stage:      dmeehan
   Console:    https://console.sst.dev/local/EasyL8/dmeehan

~  Deploying

|  Created     Dashboard sst:aws:Nextjs → DashboardDevServer sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardDevServerLogGroup aws:cloudwatch:LogGroup
|  Created     Dashboard sst:aws:Nextjs → DashboardDevServerCode aws:s3:BucketObjectv2
|  Created     Dashboard sst:aws:Nextjs → DashboardDevServerRole aws:iam:Role (1.8s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDevServerFunction aws:lambda:Function (14.3s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDevServerCodeUpdater sst:aws:FunctionCodeUpdater (2.1s)
|  Deleted     Dashboard sst:aws:Nextjs → DashboardInvalidationPolicyAttachment70e6 aws:iam:RolePolicyAttachment
|  Deleted     DashboardCdn sst:aws:CDN → DashboardCdnWaiter sst:aws:DistributionDeploymentWaiter
|  Deleted     Dashboard sst:aws:Nextjs → DashboardInvalidation sst:aws:DistributionInvalidation
|  Deleted     Dashboard sst:aws:Nextjs → DashboardInvalidationPolicy aws:iam:Policy
|  Deleted     DashboardCdn sst:aws:CDN → DashboardCdnDistribution aws:cloudfront:Distribution (345.7s)
|  Deleted     DashboardImageOptimizer sst:aws:Function → DashboardImageOptimizerCodeUpdater sst:aws:FunctionCodeUpdater
|  Deleted     DashboardImageOptimizer sst:aws:Function → DashboardImageOptimizerUrl aws:lambda:FunctionUrl
|  Deleted     DashboardRevalidationEventsSubscriberBaouue sst:aws:QueueLambdaSubscriber → DashboardRevalidationEventsSubscriberBaouueEventSourceMapping aws:lambda:EventSourceMapping (14.3s)
|  Deleted     Dashboard sst:aws:Nextjs → DashboardCdn sst:aws:CDN
|  Deleted     DashboardRevalidationEventsSubscriberBaouueFunction sst:aws:Function → DashboardRevalidationEventsSubscriberBaouueFunctionCodeUpdater sst:aws:FunctionCodeUpdater
|  Deleted     Dashboard sst:aws:Nextjs → DashboardRevalidationSeed aws:lambda:Invocation
|  Deleted     DashboardImageOptimizer sst:aws:Function → DashboardImageOptimizerFunction aws:lambda:Function
|  Deleted     DashboardAssets sst:aws:Bucket → DashboardAssetsPolicy aws:s3:BucketPolicy
|  Deleted     DashboardRevalidationSeeder sst:aws:Function → DashboardRevalidationSeederCodeUpdater sst:aws:FunctionCodeUpdater
|  Deleted     DashboardDefault sst:aws:Function → DashboardDefaultCodeUpdater sst:aws:FunctionCodeUpdater
|  Deleted     Dashboard sst:aws:Nextjs → DashboardAssetFiles sst:aws:BucketFiles
|  Deleted     DashboardDefault sst:aws:Function → DashboardDefaultUrl aws:lambda:FunctionUrl
|  Deleted     DashboardRevalidationEventsSubscriberBaouueFunction sst:aws:Function → DashboardRevalidationEventsSubscriberBaouueFunctionFunction aws:lambda:Function
|  Deleted     DashboardAssets sst:aws:Bucket → DashboardAssetsPublicAccessBlock aws:s3:BucketPublicAccessBlock
|  Deleted     DashboardAssets sst:aws:Bucket → DashboardAssetsCors aws:s3:BucketCorsConfigurationV2
|  Deleted     DashboardImageOptimizer sst:aws:Function → DashboardImageOptimizerRole aws:iam:Role (1.7s)
|  Deleted     DashboardRevalidationEventsSubscriberBaouueFunction sst:aws:Function → DashboardRevalidationEventsSubscriberBaouueFunctionCode aws:s3:BucketObjectv2
|  Deleted     DashboardDefault sst:aws:Function → DashboardDefaultFunction aws:lambda:Function
|  Deleted     DashboardRevalidationSeeder sst:aws:Function → DashboardRevalidationSeederFunction aws:lambda:Function
|  Deleted     DashboardRevalidationEventsSubscriberBaouueFunction sst:aws:Function → DashboardRevalidationEventsSubscriberBaouueFunctionLogGroup aws:cloudwatch:LogGroup
|  Deleted     DashboardRevalidationEventsSubscriberBaouueFunction sst:aws:Function → DashboardRevalidationEventsSubscriberBaouueFunctionRole aws:iam:Role (1.1s)
|  Deleted     DashboardAssets sst:aws:Bucket → DashboardAssetsBucket aws:s3:BucketV2 (1.3s)
|  Deleted     DashboardRevalidationEventsSubscriberBaouue sst:aws:QueueLambdaSubscriber → DashboardRevalidationEventsSubscriberBaouueFunction sst:aws:Function
|  Deleted     DashboardAssets sst:aws:Bucket → DashboardAssetsId random:index:RandomId
|  Deleted     DashboardRevalidationSeeder sst:aws:Function → DashboardRevalidationSeederCode aws:s3:BucketObjectv2
|  Deleted     DashboardDefault sst:aws:Function → DashboardDefaultCode aws:s3:BucketObjectv2
|  Deleted     DashboardImageOptimizer sst:aws:Function → DashboardImageOptimizerCode aws:s3:BucketObjectv2
|  Deleted     DashboardRevalidationSeeder sst:aws:Function → DashboardRevalidationSeederLogGroup aws:cloudwatch:LogGroup
|  Deleted     DashboardImageOptimizer sst:aws:Function → DashboardImageOptimizerLogGroup aws:cloudwatch:LogGroup
|  Deleted     DashboardDefault sst:aws:Function → DashboardDefaultLogGroup aws:cloudwatch:LogGroup
|  Deleted     DashboardDefault sst:aws:Function → DashboardDefaultRole aws:iam:Role (1.3s)
|  Deleted     DashboardRevalidationSeeder sst:aws:Function → DashboardRevalidationSeederRole aws:iam:Role (1.4s)
|  Deleted     DashboardRevalidationEvents sst:aws:Queue → DashboardRevalidationEventsQueue aws:sqs:Queue (3.2s)
|  Deleted     Dashboard sst:aws:Nextjs → DashboardDefault sst:aws:Function
|  Deleted     Dashboard sst:aws:Nextjs → DashboardImageOptimizer sst:aws:Function
|  Deleted     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouue sst:aws:QueueLambdaSubscriber
|  Deleted     Dashboard sst:aws:Nextjs → DashboardRevalidationSeeder sst:aws:Function
|  Deleted     Dashboard sst:aws:Nextjs → DashboardRevalidationEvents sst:aws:Queue
|  Deleted     Dashboard sst:aws:Nextjs → DashboardAssets sst:aws:Bucket
|  Deleted     Dashboard sst:aws:Nextjs → DashboardServerCachePolicy aws:cloudfront:CachePolicy
|  Deleted     Dashboard sst:aws:Nextjs → DashboardOriginAccessIdentity sst:aws:OriginAccessIdentity (1.1s)
|  Deleted     Dashboard sst:aws:Nextjs → DashboardCloudfrontFunctionServerCfFunction aws:cloudfront:Function (2.9s)
|  Deleted     Dashboard sst:aws:Nextjs → DashboardRevalidationTable aws:dynamodb:Table (3.9s)

✓  Complete

  ▲ Next.js 14.2.5
  - Local:        http://localhost:3000

 ✓ Starting...
 ✓ Ready in 1641ms

Subsequent Deploy phase
/Users/dmeehan/.nvm/versions/node/v20.15.1/bin/npm run deploy

><EMAIL_ADDRESS>deploy
> sst deploy

SST ❍ ion 0.1.4  ready!

➜  App:        EasyL8
   Stage:      dmeehan

~  Deploying

|  Log         Running "npx --yes<EMAIL_ADDRESS>build" script
|  Log         Next.js version : 14.2.5
|  Log         OpenNext v3.0.2
|  Log         ┌─────────────────────────────────┐
|  Log         │ OpenNext — Building Next.js app │
|  Log         └─────────────────────────────────┘
|  Log         ><EMAIL_ADDRESS>build
|  Log         > next build
|  Log           ▲ Next.js 14.2.5
|  Log            Creating an optimized production build ...
|  Log          ✓ Compiled successfully
|  Log            Linting and checking validity of types ...
|  Log            Collecting page data ...
|  Log            Generating static pages (0/5) ...
|  Log            Generating static pages (1/5)
|  Log            Generating static pages (2/5)
|  Log            Generating static pages (3/5)
|  Log          ✓ Generating static pages (5/5)
|  Log            Finalizing page optimization ...
|  Log            Collecting build traces ...
|  Log         Route (app)                              Size     First Load JS
|  Log         ┌ ○ /_not-found                          871 B          87.9 kB
|  Log         └ ○ /dashboard/applications              141 B          87.2 kB
|  Log         + First Load JS shared by all            87 kB
|  Log           ├ chunks/23-b75664ace61c0abb.js        31.5 kB
|  Log           ├ chunks/fd9d1056-2821b0f0cabcd8bd.js  53.6 kB
|  Log           └ other shared chunks (total)          1.86 kB
|  Log         ○  (Static)  prerendered as static content
|  Log         ┌──────────────────────────────┐
|  Log         │ OpenNext — Generating bundle │
|  Log         └──────────────────────────────┘
|  Log         Bundling middleware function...
|  Log         Bundling static assets...
|  Log         Bundling cache assets...
|  Log         Building server function: default...
|  Log         Bundling revalidation function...
|  Log         Bundling image optimization function...
|  Log         Bundling warmer function...
|  Log         OpenNext build complete.
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeeder sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEvents sst:aws:Queue
|  Created     Dashboard sst:aws:Nextjs → DashboardAssets sst:aws:Bucket
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsId random:index:RandomId
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederLogGroup aws:cloudwatch:LogGroup
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederCode aws:s3:BucketObjectv2
|  Created     Dashboard sst:aws:Nextjs → DashboardOriginAccessIdentity sst:aws:OriginAccessIdentity (2.3s)
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsBucket aws:s3:BucketV2 (2.8s)
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsPublicAccessBlock aws:s3:BucketPublicAccessBlock
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsCors aws:s3:BucketCorsConfigurationV2
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetsPolicy aws:s3:BucketPolicy (9.0s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationTable aws:dynamodb:Table (24.6s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsQueue aws:sqs:Queue (26.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederRole aws:iam:Role (1.9s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefault sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizer sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouue sst:aws:QueueLambdaSubscriber
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunction sst:aws:Function
|  Created     Dashboard sst:aws:Nextjs → DashboardServerCachePolicy aws:cloudfront:CachePolicy (1.4s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionLogGroup aws:cloudwatch:LogGroup (2.8s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerLogGroup aws:cloudwatch:LogGroup (2.9s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionCode aws:s3:BucketObjectv2 (2.8s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultLogGroup aws:cloudwatch:LogGroup (4.4s)
|  Created     Dashboard sst:aws:Nextjs → DashboardAssetFiles sst:aws:BucketFiles (5.0s)
|  Created     Dashboard sst:aws:Nextjs → DashboardCdn sst:aws:CDN
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerRole aws:iam:Role (5.0s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultRole aws:iam:Role (5.1s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionRole aws:iam:Role (5.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardCloudfrontFunctionServerCfFunction aws:cloudfront:Function (5.5s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederFunction aws:lambda:Function (14.5s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerFunction aws:lambda:Function (11.1s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultFunction aws:lambda:Function (11.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionFunction aws:lambda:Function (11.2s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeederCodeUpdater sst:aws:FunctionCodeUpdater (2.8s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerUrl aws:lambda:FunctionUrl (1.7s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultUrl aws:lambda:FunctionUrl (1.7s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationSeed aws:lambda:Invocation (2.0s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueFunctionCodeUpdater sst:aws:FunctionCodeUpdater (2.7s)
|  Created     Dashboard sst:aws:Nextjs → DashboardCdnDistribution aws:cloudfront:Distribution (7.5s)
|  Created     Dashboard sst:aws:Nextjs → DashboardInvalidation sst:aws:DistributionInvalidation (2.4s)
|  Created     Dashboard sst:aws:Nextjs → DashboardInvalidationPolicy aws:iam:Policy (2.9s)
|  Created     Dashboard sst:aws:Nextjs → DashboardRevalidationEventsSubscriberBaouueEventSourceMapping aws:lambda:EventSourceMapping (19.4s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultCode aws:s3:BucketObjectv2 (38.7s)
|  Created     Dashboard sst:aws:Nextjs → DashboardDefaultCodeUpdater sst:aws:FunctionCodeUpdater (3.3s)
|  Created     Dashboard sst:aws:Nextjs → DashboardInvalidationPolicyAttachment70e6 aws:iam:RolePolicyAttachment
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerCode aws:s3:BucketObjectv2 (44.5s)
|  Created     Dashboard sst:aws:Nextjs → DashboardImageOptimizerCodeUpdater sst:aws:FunctionCodeUpdater (2.0s)
|  Created     Dashboard sst:aws:Nextjs → DashboardCdnWaiter sst:aws:DistributionDeploymentWaiter (275.9s)
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerCodeUpdater sst:aws:FunctionCodeUpdater
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerFunction aws:lambda:Function
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerCode aws:s3:BucketObjectv2
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerLogGroup aws:cloudwatch:LogGroup
|  Deleted     DashboardDevServer sst:aws:Function → DashboardDevServerRole aws:iam:Role (1.7s)
|  Deleted     Dashboard sst:aws:Nextjs → DashboardDevServer sst:aws:Function

✓  Complete
   Dashboard: https://<REDACTED>h1s3nq.cloudfront.net

Process finished with exit code 0

sst.config.ts
/// <reference path="./.sst/platform/config.d.ts" />

export default $config({
  app(input) {
    return {
      name: "<REDACTED>",
      removal: input?.stage === "production" ? "retain" : "remove",
      home: "aws",
    };
  },
  async run() {

    const entitiesArn = 'arn:aws:dynamodb:eu-west-1:<REDACTED>:table/<REDACTED>'
    const { tableName } = entitiesArn.match(/table\/(?<tableName>.*)$/)?.groups ?? {}

    new sst.aws.Nextjs("Dashboard", {
      permissions: [
        { actions: ['dynamodb:DescribeEndpoints'], resources: ['*'] },
        {
          actions: [
            'dynamodb:GetItem',
            'dynamodb:PutItem',
            'dynamodb:DeleteItem',
            'dynamodb:Query',
          ],
          resources: [entitiesArn]
        },
      ],
      environment: {
        ENTITIES_TABLE_NAME: tableName,
      }
    });
  },
});

Oh are you using sst dev and sst deploy on the same stage?

@jayair That's what I was doing, but I realized it should be on separate stages. I think a doc update around this might be helpful, since I didn't understand that the first time around and got confused similar to @dmeehan1968.

Yeah we updated all our quick starts with this.

I also added this workflow doc: https://ion.sst.dev/docs/workflow/

Oh are you using sst dev and sst deploy on the same stage?

Yes, but that's not been a problem up till somewhere around 0.1 release.
Now that's clear I can avoid that.