ENH Enable use of AWS spot instances

Reference Issue

What does this implement/fix? Explain your changes.
Use request_spot_instances.
Any other comments?
~Problem with request_spot_instances is that it returns a dictionary. Currently we use create_instances which returns a list(ec2.Instance), where ec2.instance is a class.~
Hopefully list(resource.Instance(instance_id)) is the same type of object as returned by create_instances which returns a list(ec2.Instance).
All the current code is written for ec2.instance
cc @agramfort @glemaitre

Hello @lucyleeow! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file ramp-engine/ramp_engine/aws/api.py:

Line 171:59: E231 missing whitespace after ','
Line 177:51: E231 missing whitespace after ','
Line 189:39: E231 missing whitespace after ','
Line 196:34: E231 missing whitespace after ','

@agramfort @glemaitre this is ready for review if either of you want to look at it. After this is merged I still need to make a patch release and update ramp-board on the server. Should probably be done before the course on Friday?
I've tried this on my local RAMP server a few times, sending submissions to AWS and it works fine.
I was not able to test:

when the spot instance is not fulfilled (e.g., due to lack of capacity etc)
when a spot instance is terminated (manually terminating an instance does not occur in the same way as AWS terminating a spot instance)

FYI @rth I have added a abstractmethod _is_submission_terminated to the BaseWorker, which returns whether or not a spot instance as been terminated for AWS worker, and for CondaWorker always returns False.

Thanks for the review @rth. Suggestions added

It might be useful to add a dispatcher test that a CondaEnvWorker with a _status = 'retry' (or 'reset') is indeed restarted, to test at least some of it locally?

I'm not good with RAMP and it's tests, and it doesn't seem to like me either. I tried adding a test (see recent commits) but I couldn't test locally because I got this error:
    @pytest.fixture(scope='session')
    def database_connection():
        """
        Create a Postgres database for the tests, and drop it when the tests are
        done.
        """
        config = safe_load(open("db_engine.yml"))
        dbowner = config.get('db_owner')

        engine = create_engine(f'postgresql://{dbowner}:@localhost/postgres',
                               isolation_level='AUTOCOMMIT')

        connection = engine.connect()

        database_config = read_config(database_config_template())
        username = database_config['sqlalchemy']['username']
        database_name = database_config['sqlalchemy']['database']
        try:
            connection.execute(f"""CREATE USER {username}
                                  WITH PASSWORD '{username}';
                                  ALTER USER {username} WITH SUPERUSER""")
        except exc.ProgrammingError as e:
>           raise ValueError(f'user {username} already exists') from e
E           ValueError: user mrramp already exists

but when I tried to delete mrramp with ramp-database delete-user --login mrramp, i got:
Traceback (most recent call last):
  File "/home/lucy/miniconda3/envs/ramp/bin/ramp-database", line 11, in <module>
    load_entry_point('ramp-database', 'console_scripts', 'ramp-database')()
  File "/home/lucy/Documents/RAMP/ramp-board/ramp-database/ramp_database/cli.py", line 365, in start
    main()
  File "/home/lucy/miniconda3/envs/ramp/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/lucy/miniconda3/envs/ramp/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/lucy/miniconda3/envs/ramp/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lucy/miniconda3/envs/ramp/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/lucy/miniconda3/envs/ramp/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/lucy/Documents/RAMP/ramp-board/ramp-database/ramp_database/cli.py", line 59, in delete_user
    user_module.delete_user(session, login)
  File "/home/lucy/Documents/RAMP/ramp-board/ramp-database/ramp_database/tools/user.py", line 117, in delete_user
    user = session.query(User).filter(User.name == name).one()
  File "/home/lucy/miniconda3/envs/ramp/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3289, in one
    raise orm_exc.NoResultFound("No row was found for one()")
sqlalchemy.orm.exc.NoResultFound: No row was found for one()

The tests seemed to have passed but I think the failure is due to time out?
================== 87 passed, 3 warnings in 116.35s (0:01:56) ==================

and
ramp-engine/ramp_engine/tests/test_aws.py ss                             [  7%]

ramp-engine/ramp_engine/tests/test_cli.py ......                         [ 28%]

ramp-engine/ramp_engine/tests/test_conda_worker.py ...........           [ 67%]

ramp-engine/ramp_engine/tests/test_daemon.py ..                          [ 75%]

ramp-engine/ramp_engine/tests/test_dispatcher.py ......

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received

The build has been terminated

I've restarted the build....

The frontend tests seemed to have passed but then I think it times out?

@lucyleeow I can reproduce locally, so it looks like it's a genuine issue with the test (that deadlocks at some point) rather than a CI issue. I'll try to have a look later today.

I haven't tested the test I added because https://github.com/paris-saclay-cds/ramp-board/pull/455#issuecomment-722753373
so that's likely to be the reason

Yeah, that error is strange..

It's probably a problem with the test - not know about ramp tests and not being able to test it means it's highly likely the test I wrote is incorrect...

commented out test in latest commit

Hi @lucyleeow as for the test
I think the problem is that if you call: dispatcher._awaiting_worker_queue.get() and there is nothing in that queue it will wait forever.
That said, you do put submissions in the queue when calling dispatcher.launch() but before you run dispatcher._awaiting_worker_queue.get() they are already processed and the queue is empty.
Non-optimal solution could be to create a submission manually right before getting worker from the queue:
    submissions = get_submissions(
        session_toy, event_config['ramp']['event_name'], 'training_error'
        )[0]
    submission_id, submission_name, _ = submissions

    # create the worker
    worker = dispatcher.worker(dispatcher._worker_config, submission_name)

    dispatcher._awaiting_worker_queue.put_nowait((worker, (submission_id,
                                                 submission_name)))
    worker, (_, submission_name) = dispatcher._awaiting_worker_queue.get()

But this still would not be sufficient because before you get to the worker status the submission is already killed.
Perhaps the submission would have to be set to something which takes longer to run? Is it possible within our test settings?

Also tried:
(ramp) lucy@:ramp-board(aws_spot)$ psql postgres lucy
psql (11.2)
Type "help" for help.

postgres=# DROP USER mrramp
postgres-#

to remove user mrramp but did not work either.

Travis isn't building anymore

Builds have been temporarily disabled for public repositories due to a negative credit balance. Please go to the Plan page to replenish your credit balance or alter your Consume paid credits for OSS setting.

This test works - tested locally. It may or may not test

It might be useful to add a dispatcher test that a CondaEnvWorker with a _status = 'retry' (or 'reset') is indeed restarted, to test at least some of it locally?

adequately.


[Request] RogueKiller

 I've looked at the list of issues to see if there is already a request for this but no [CONFIRMED BY XEL]
Package Request
Information
Name: RogueKiller
Description: [clear and concise details of what it is]
Quoting from BleepingComputer:
RogueKiller is a security tool that can be used to terminate and remove malicious processes and programs from your computer.  RogueKiller has the ability to remove infections such as ZeroAccess, TDSS, rogue anti-spyware programs, and Ransomwares.

I also mentioned this back in here: Rkill
Homepage: [a URL/link]
https://www.bleepingcomputer.com/download/roguekiller/
Download link(s): [URL(s)/link(s)]
https://www.bleepingcomputer.com/download/roguekiller/dl/121/
Some indication of popularity/repute: [GitHub stars/software reviews etc.]
https://download.cnet.com/RogueKiller/3000-8022_4-75764640.html - review from cnet.com
https://www.snapfiles.com/userreviews/112945/roguekiller.html  - review from users

Created a PR to close this issue #9311


Trim Async suffix on SignalR Hub method names

See https://github.com/aspnet/AspNetCore/pull/7420
@muratg We'll probably want to do this sooner than later since it's breaking.

Thanks @halter73.  @bradygaster FYI.

Should we do this in SignalR? I don't think it makes the same kind of sense here, since SignalR is RPC, but MVC is a URL pattern.
In MVC, it makes a lot of sense, because you may have an action like:
public class CustomersController
{
    public async Task GetAllAsync()
    {
    }
}

And you want to access it at /Customers/GetAll, not /Customers/GetAllAsync. This makes total sense to me.
However, in SignalR you are calling a method, rather than sending a request to a URL. So if you have a Hub method:
public class CustomersHub
{
    public async Task GetAllAsync()
    {
    }
}

The code you want to write on the .NET client would definitely be:
var customers = /* Get Hub Connection */;
// Assume we had perfect client generation, etc.
var customers = await customers.GetAllAsync();

Stripping Async from the method name doesn't make as much sense there for me.
I realize it's not a perfect analogy, since JavaScript, Java and C++ are all viable clients and don't have this pattern. However, in this case, I think we should not be stripping suffixes from method names since the user is defining a method which will be called like a method rather than defining a method that ends up generating a URL pattern.

Good point @anurse about RPC. The real benefit of this change seems to be related to the default way MVC does routing based on action names.

If we were to do this at all, the Async suffix probably should be optional, but I'd rather keep things simple and have one true name for every hub method.

Triage Decision: This doesn't make sense for SignalR, let's close.


Strange boolean-discriminant narrowing with strictNullChecks off

TypeScript Version:  nightly (2.1.0-dev.20160826)
Code
type Result = { success: true }
            | { success: false, error: string }

function handleError(res: Result) {
    if (res.success === true) {
        return;
    }

    res.error; // OK
}

function handleError2(res: Result) {
    if (res.success !== false) {
        return;
    }

    res.error; // OK
}

function handleError3(res: Result) {
    if (res.success) {
        return;
    }

    res.error; // Property 'error' does not exist on type 'Result'
               // but should be OK
}

Expected behavior:
All three cases should behave the same.

This is correct. We don't know that res.success isn't undefined or null, which would mean it's possibly not false and therefore not { success: false, error: string }

@RyanCavanaugh While you are technically right, when strictNullChecks is off null and undefined are not modelled at all and such differences don't surface in other situations. For example handleError has exactly the same semantics but compiles just fine. I think the behaviour should be consistent or at least the error message should be more descriptive.

what is the expected behavior?

@DanielRosenwasser I have the same question

This is correct. We don't know that res.success isn't undefined or null, which would mean it's possibly not false and therefore not { success: false, error: string }

I'm wondering why the if (res.success === true) { example above should compile then, because couldn't success still be undefined or null?

Hey, I just wanted to point out that this issue now has additional relevance in TypeScript 3.6 due to the addition of the Generator type and the new IteratorResult type.  I was writing some code with generators, expecting I could use while(!result.done) to loop over the case where the generator was not done, but found in fact I had to use while(result.done === false).  Might be a reason to revisit this?

I have experienced similar behaviour, and the following code helped me to work around this limitation:
(Assuming we have foo with the discriminating boolean-property "discBool" and the optional property "optionalProperty" which is only present in one of the two sub-types of foo's super-type)
switch (foo.discBool) {
  case false:
    doSomething(foo.optionalProperty);
}

while this code didn't work:
if (!foo.discBool) {
  doSomething(foo.optionalProperty); // Error here
}


Display feature image on record detail page

Using rs:image custom_field. Use semantic_ui item for aligning the optional feature image and description.

Implemented via 82d4732.


First version - no tests

First version builds a docker image with the adapter and octactl, including the necessary credentials.
No tests yet.

Removed sensitive text.  Proceeding with merge for alpha version.


What happens if you set a monthly schedule on say, the 31st, and it's February?

Would the Patch Orchestration App just not check for updates? Would it shift it to the next month? Are you simply not able to put in a number for the date exceeding 28? I apologize if this was answered in the document.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a3a16900-c967-ccf9-2d87-1c994013bf0e
Version Independent ID: 474287be-47ca-50b3-1240-8cca745eca9e
Content: Azure Service Fabric patch orchestration application
Content Source: articles/service-fabric/service-fabric-patch-orchestration-application.md
Service: service-fabric
Product: unspecified
GitHub Login: @novino
Microsoft Alias: nachandr

@montanamott Thanks for the question! We are investigating and will update you shortly.

@montanamott I believe it will default to the 28th in this case if there is a month that does not have 31 days.
@novino can you confirm on this?

@montanamott I am working with @novino offline to figure this out. Will provide you with an update shortly.

@montanamott For monthly frequency, the permissible values for day field are 1 - 28 as well as "last"
So if you want to update on last day of every month, you'd have to set eg: "Monthly, last, 12:34:56"
Realized this special case isn't covered in our documentation, will update the same shortly.

Thanks @Novino! I will assign this to you for now while you work on that update.
@montanamott let us know if you have additional questions.


Update free-courses-hi.md

The course is no longer available in youtube. Removed the link to improve remaining suggestions in the android course.
Android App Development Full Course in Hindi - Geeky Hub
What does this PR do?
It removes the removed playlist link from the youtube to optimize the suggestions of the android course.
For resources
Android App Development Full Course in Hindi - Geeky Hub
link of the removed playlist

Thanks, and welcome to free-programming-books!


[Backport 1.x] Add flat_skew setting to node overload decider

Add flat_skew setting to node overload decider

Signed-off-by: Rishab Nahata<EMAIL_ADDRESS>Description
Backport 836a9c4910f5eb4e7fcaca5573bd3bf47073d416 from #3563

What's the story behind backporting it to 1.x? That ship has long sailed ;)


More motion

Hi, according to another issue, I increase the system.loss.lambda_sds_video towards 1.0. Because based on the results displayed in the gallery, "a dog riding a skateboard" doesn't seem to have much motion, so I tried "a panda dancing".
When training for 50,000 epochs with system.loss.lambda_sds_video=0.1, I got,
https://github.com/sherwinbahmani/4dfy/assets/67221921/af148f19-03ad-49e5-9581-a957c921904b
When training for 50,000 epochs with system.loss.lambda_sds_video=1.0, I got,
https://github.com/sherwinbahmani/4dfy/assets/67221921/8389ef21-1dbd-4c75-b985-b38065b22e39
And I found that, when training for 10,000 epochs with system.loss.lambda_sds_video=0.1,
https://github.com/sherwinbahmani/4dfy/assets/67221921/c56a434b-3eed-4f41-8404-59bba98304c3
when training for 10,000 epochs with system.loss.lambda_sds_video=1.0,
https://github.com/sherwinbahmani/4dfy/assets/67221921/63dcec1d-d7f1-40ed-aa56-c8e3b60022cd
Thus, I guess that perhaps when using a small system.loss.lambda_sds_video, it requires longer training time to obtain more motion. However, when the training time is long enough, the impact of system.loss.lambda_sds_video seems not to be significant?

Hi,
Yes for the case you are showing it does not seem to have a huge difference, but I remember it made differences for some prompt when testing it. I am currently experimenting with Videocrafter2 as a video model guidance, this should improve motion a lot. I will get back to that, as soon as I have first results.
I still think one bottleneck is the 4D representation, some better disentangled approach there could help there instead of using the two hashgrids.

Thanks a lot for the reply and I'm looking forward to the results~
Additionally, what do you think are the limitations of 4D representations? Is it that they cannot capture more motion? AYG is a disentangled approach, but I don't see their motion being significantly more than 4dfy, and their method also includes "extended autoregressive generation" (which extends their dynamic time) and "motion amplification", which could be helpful in increasing motion. Of course, I'm not suggesting that hashgrids are the best representation method; I'm just considering aspects in relation to the issue of motion.
I hope my expression is clear. Thank you again for the discussion！

That makes sense and thanks for your insight! I'll run some tests to explore more.


Dynamic workspace shapefile sample not working outside Sample Viewer

I downloaded the Sample Viewer 100.5 source code from here, built it using VS2017 and ran the Dynamic workspace shapefile sample and it worked as expected. Then I manually downloaded the empty map package and the shape files and changed the sample to use the local copies instead of using the DataManger and it still worked fine.
But when I try to create my own sample app based on that, it does not work. The Local Server is started, I get no exceptions, but no shape file data is displayed in the map. The only difference between the samples is that I put the MapView in my MainWindow.xaml instead of in a UserControl.
I created my sample application in VS2017 by creating a new WPF App (.NET Framework) project using .NET Framework 4.8 to which I added the NuGet packages Esri.ArcGISRuntime.WPF v100.5 and Esri.ArcGISRuntime.LocalServices v100.5 and then copying the code from DynamicWorkspaceShapefile.xaml.cs in your sample to MainWindow.xaml.cs in my project.
I tried changing to .NET Framework 4.6.1 which Sample Viewer 100.5 uses, but that didn't help.
Any ideas on why it does not work?

One more difference I found after a bit more digging is that the downloaded Sample Viewer uses the Local Server from the installed Local Server SDK (C:\Program Files (x86)\ArcGIS SDKs\LocalServer100.5\64\bin\RuntimeLocalServer.exe), while my sample project uses the Local Server copied to the project (C:...\TestLocalServerShp2\bin\Debug\LocalServer100.5\64\bin\RuntimeLocalServer.exe).
What controls which Local Server is used?

@marhja By default the API will look for the folder LocalServer100.X alongside your app executable, and if not present it will fall back to looking for the installed SDK location.
The creation of the folder LocalServer100.X is controlled by the config file in your project folder.
For more information see https://developers.arcgis.com/net/latest/wpf/guide/create-a-local-server-deployment.htm.

@marhja Also ensure you set an appropriate Temp folder - we may not have highlighted that point sufficiently in the viewer because it's handled in the app initialization code.
https://github.com/Esri/arcgis-runtime-samples-dotnet/blob/99c19153a72fff496fac3a6a416247ea5a70bbe6/src/WPF/ArcGISRuntime.WPF.Viewer/App.xaml.cs#L25

@mbranscomb ArcGIS Runtime Temp folder was already initialized to my user's Temp folder, so setting that wasn't necessary.
Deleting the bin\Debug\LocalServer100.5 folder and start the app without debugging (to not trigger a build which would create the folder again) made it use the instance in the installed Local Server SDK as you explained, which made my sample app work.
It turned out I had to enable the child packages ArcMapAdditionalDataFormats and ArcMapVector of the ArcMap Compatible Server package in the ArcGISLocalServer_100.5.AGSDeployment file to make my sample app work with the Local Server instance in bin\Debug\LocalServer100.5.
Thanks for the help!


Reverting PR# 13684

Coded needed by bulk assignments/unassignments of tags
and possible future policies.

Merging now because this was causing a failed build on master.

@abellotti in the future, please do not merge PRs you author.  /cc @gtanzillo @Fryguy


Unicode error - german umlaut

I try to create a tkinter app with python.
I am german and write my instructions in german. That seems to work more or less, but gpte seems to struggle with german umlauts, like ä, ö or ü.
When it writes code with these symbols, the files it creates are encoded in ansi, not utf-8. Therefore the python-interpreter struggles when processessing them:
File "C:\Users\Arnd\Desktop 3\GPT Engineer\stundenplan\tabs.py", line 33
self.teacher_table = Treeview(self, columns=("Langname", "K�rzel"))
^
SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0xfc in position 1: invalid start byte
File "C:\Users\Arnd\Desktop 3\GPT Engineer\stundenplan\app.py", line 21
tab_control.add(self.plaene_tab, text='Pl�ne')
^
SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0xe4 in position 2: invalid continuation byte
When I set the file encoding to utf-8 and correct the problematic symbols by hand, the errors go away and the application runs.

I believe the easiest fix to this is to tell the LLM to only respond in ASCII characters, no matter what language it is prompted in. With that, you should still be able to prompt in German, but the software, including strings and variable names should all be ASCII. Does this at least partly solve your problem?

I didn't try it out, but I guess it would.
But there are quite a lot of languages on this planet and since utf-8 there should be no reason to fall back on ascii. I am writing python-code in german for many years now and never had any problems with this issue. Such a regression to the 80's oder 90's (I am old enough to remember that) seems weird.
In the end it is not a big deal for me, because I am just toying around, but I thought to mention it to add it to your to-do list.
Thanks
Arnd


JSON parse fails while trying to display some ballots.

The code assumes all ballots come with a valid JSON string in the 'content' field. However, there are some that come with the text "Telos Amend Proposal" with cause the parser to break, and the popup just gets stuck.

I close this issue because is fixed and merged

I'm reopening this issue because is not solved.
Somehow I didn't push the try-catch fix that I mention here in the comments, so the problems still remain.

I close this issue is fixed and merged


Generate by looking into dream_log.txt instead of last image in the directory

This also fixes a bug where it keeps overwriting last file when run as dream.py --from_file path/input.txt -o output/path/

If you run again after removing the last generated image because you didn't like it, it will save with the name of file you just deleted. Now you have two entries in log file with same name. You can't reliably lookup in log with this.
That's why using log file for next name is better in my opinion.

Ouch. This unwanted behavior is an indirect consequence of the decision to make pngwriter stateless in PR #231. Originally, I had pngwriter keep track of the last file name written during the session rather than going back to the filesystem. Of course, the bug you've identified would still have bit us if the user deleted the last file between sessions.
Before I review your solution, we need to discuss what to do in the event that the user deletes or moves the log file. Here are some options:

Give a big fat warning if the directory contains png files and no log file.
Always check whether the generated filename exists and never clobber an existing file (I might be missing it, but I don't think this check is implemented yet)
Instead of using the log file, read the last written filename from a hidden file in the directory. On Linux and Mac, this would be a file starting with a ".". I think on Winodws use set the "h" attribute?

I'm leaning a bit towards (3), but there's also the risk that the user finds the hidden file and deletes it, not knowing that it will bring calamity upon them. In this event, maybe we fall back on tallying the files in the directory (or the log) and recreate the hidden file.
Bringing in @bakkot on this conversation, cause he's worked on the filenaming issue in the past.

The refactor in https://github.com/lstein/stable-diffusion/pull/231 didn't change this behavior. The old pngwriter would also look for a new "prefix" from the filesystem every time the class was instantiated, which happened every time you entered a new prompt at dream.py.
Anyway, we should figure out what the actual behavior we want is. Personally I would be inclined to:

keep track of the last prefix used during this session
during initialization (i.e. when the PngWriter class is instantiated), start that counter by reading the last entry in the log file
whenever we go to generate a new prefix, read the output directory and ensure the candidate prefix is not already in use

Not a fan of introducing a new hidden file - as a user hidden files tend to provide a pretty annoying experience, because it means I can't just reset everything by deleting all the files I can see.

I have a file "counter.txt". Inside it there is a number. This number will be used for generation of next image. Then number inside "counter.txt" will be increased by one. I have not experienced any problems with this approach.

keep track of the last prefix used during this session
I think this is a much better approach. We shouldn't be needing to go back to log file or whole directory to get next prefix.

Since we are reading opt.outdir in loop for some reason, i didn't find a simple way to put last prefix in session.
Updated to get prefix from png if log was not found for some reason.

keep track of the last prefix used during this session

I think this is a much better approach. We shouldn't be needing to go back to log file or whole directory to get next prefix in the same session.

I like the "counter.txt" approach. It means we only have to read one file and it helps the user understand exactly what is happening. There still needs to be a check that we don't overwrite existing files.

I like the "counter.txt" approach. It means we only have to read one file and it helps the user understand exactly what is happening. There still needs to be a check that we don't overwrite existing files.

Another advantage of "counter.txt" is that user can open this file to input any number they want. I do it myself occasionally when software starts generating 5 digits numbers that are too difficult to remember, so I reset it to "12" :)

Adding one more file for count means one more moving part. I believe, from your comment somewhere else @lstein that you see log.txt as a way to quickly lookup command used to create an image. It's nice and simple approach and we should retain that.
If log has reached the 50.png and counter.txt says 10, then 11.png and subsequent ones will be entered twice in log.txt and you will have to read the command from png itself.

How about we do this?

at pngwriter initialization time, it looks for the log file (once) and gets the latest prefix
if log file is missing, then read the directory (once) and get the latest prefix
if no files in directory, then reinitialize to prefix 000001
prefix stored in self and incremented for each generation step so we don't have additional filesystem overhead
there is always a check before clobbering a file, regardless of what the prefix says

I think this will work. Did I miss something?

there is always a check before clobbering a file, regardless of what the prefix says

and in case we do find an existing file with same name, we repeat the first three steps.

there is always a check before clobbering a file, regardless of what the prefix says

Presumably with that check disabled for upscaling, which deliberately overwrites the original unless --save_original is passed?
Also, re "if log file is missing, then read the directory (once) and get the latest prefix" - I would do this unconditionally, and choose the prefix to be the maximum of the two strategies (directory enumeration and log file). A single directory enumeration shouldn't be costly, and it means that you still get incrementing prefixes as long as you keep either the files in the directory or the log, which seems nicer.
Other than those two points, that sounds good to me. It might be a little annoying if, for example, you were generating 5 images, and the first 2 didn't conflict but then the third one did - your prefix would change in the middle of the batch. But I expect this will basically never happen - given this design you have to come up with something pretty contrived for that to be possible - so I'm not worried about it.

This sounds on target. Who wants to take a crack at implementation?
(against the development branch, please)

Again, couldn't just store prefix in session because we can read --outputdir on command any time. Made a few changes anyway.

Get next prefix from both log.txt and weblog.txt
if its 0, get by finding last png in the directory

A few notes:

this logic should live in pngwriter.py, not dream.py
the pngwriter instance can keep a dictionary of outdirs that it knows the prefix for, to avoid re-initializing it every time even in the presence of --outdir (that is, you only need to re-initialize when you encounter a directory you haven't seen before)
I haven't tested it, but from reading the code, it doesn't look like this correctly handles the upscaling case: in that case we actually do intend to overwrite the original image
there are other consumers of pngwriter (in particular in server.py), and this logic should be applied to them as well

A few notes:

this logic should live in pngwriter.py, not dream.py
the pngwriter instance can keep a dictionary of outdirs that it knows the prefix for, to avoid re-initializing it every time even in the presence of --outdir (that is, you only need to re-initialize when you encounter a directory you haven't seen before)
I haven't tested it, but from reading the code, it doesn't look like this correctly handles the upscaling case: in that case we actually do intend to overwrite the original image
there are other consumers of pngwriter (in particular in server.py), and this logic should be applied to them as well

I agree with @bakkot 's comments. I'd add that we should have a unified logger to create and format the log entries. Since both logging and pngwriting need to generate a uniform prompt, some refactoring is required so that pngwriter isn't responsible for prompt generation.
This is all tied up with @bakkot 's proposal to create an object class that acts as a centralized location to store all the gathered parameter values. As I understand the vision, this thing will act like a dictionary for getting/setting values, and be able to produce serialized representations of the parameter state using JSON and command-line formats. See discussion at #351

Could you have a look at @bakkot 's comments and see if your implementation needs to change?

Updated again. Not sure how upscale case overwrites the images since it still asks for unique prefix first. I hope I got that right.

Thanks for the changes. I'll review this as soon as I complete the 1.14 release, either today or tomorrow.

Hello, I have a question - you agreed to create a counter.txt file to keep track of last file prefix. I agree its good idea. But I suppose in near future we will want to save more user related setting / params. Eg the lastly used parameters settings from the webUI could be saved in-between sessions, etc. Wouldnt it be better to create a more generic settings file eg in json format, where among other things would be saved the last prefix? Just an idea.

This PR seems stale, closing for now.


feat(jan 25, 2024)

 Submitted with StackBlitz.

@Reed-Broadhead
Would you review my code to ensure it passes the Acceptable Solution Standards for #2 ?


Does not like all frequencies

Don't know why, but It does not like all frequencies.
Maybe a BUG.
So, if the sound quality is not good, try an other frequency. Make sure to use a free frequency.

Not sure where this should be noted but quick scan of source saw the note on tau.  These two lines work the same in US:
//AudioOutputFM            audioOutput(88, 50e-6f); //Frequency in MHz, preemphasis AudioOutputFM            audioOutput(88, 75e-6f); //Frequency in MHz, preemphasis
107 and 88 work
Using this :: AudioOutputFM            audioOutput(88.7, 75e-6f); //Frequency in MHz, preemphasis
Still registers as 88.0 on a digital FM radio.
But : AudioOutputFM            audioOutput(87.7, 75e-6f); //Frequency in MHz, preemphasis
Doesn't show up anywhere
Lots of float storage for math often done in double ...  should it all stay double?

Not sure where this should be noted but quick scan of source saw the note on tau.  These two lines work the same in US:
//AudioOutputFM            audioOutput(88, 50e-6f); //Frequency in MHz, preemphasis AudioOutputFM            audioOutput(88, 75e-6f); //Frequency in MHz, preemphasis
107 and 88 work
Using this :: AudioOutputFM            audioOutput(88.7, 75e-6f); //Frequency in MHz, preemphasis
Still registers as 88.0 on a digital FM radio.
But : AudioOutputFM            audioOutput(87.7, 75e-6f); //Frequency in MHz, preemphasis
Doesn't show up anywhere
Lots of float storage for math often done in double ...  should it all stay double?

Just updated from github, freq 91.0 okay as well.
AudioOutputFM            audioOutput(33, 91.0, 75e-6f); //Pin (23= I2S1, 30= I2S3, 33= I2S2) , Frequency in MHz, preemphasis

Just updated from github, freq 91.0 okay as well.
AudioOutputFM            audioOutput(33, 91.0, 75e-6f); //Pin (23= I2S1, 30= I2S3, 33= I2S2) , Frequency in MHz, preemphasis

Thank you for all the testing, Tim.

Thank you for all the testing, Tim.

No problem - always fun to 'egg you on'
Paul posted some short 'number' WAV files for mult or fast cycling sounds.  Should find those and add them - for SD 1.54 Beta test?  Not found them yet ...
Those music files are a bit repetitive - not always easy to know the sound isn't skipping.  First run after github update it seemed the buffer was in a cyclic loop with the same notes over and over

No problem - always fun to 'egg you on'
Paul posted some short 'number' WAV files for mult or fast cycling sounds.  Should find those and add them - for SD 1.54 Beta test?  Not found them yet ...
Those music files are a bit repetitive - not always easy to know the sound isn't skipping.  First run after github update it seemed the buffer was in a cyclic loop with the same notes over and over

https://forum.pjrc.com/threads/64235-Teensyduino-1-54-Beta-3?p=258095&viewfull=1#post258095

https://forum.pjrc.com/threads/64235-Teensyduino-1-54-Beta-3?p=258095&viewfull=1#post258095

Another edit for faster play of those short sounds ....
`void playFile(const char *filename)
{
Serial.print("Playing file: ");
Serial.println(filename);
playWav1.play(filename);
delay(25);
while (playWav1.isPlaying()) {
Serial.print("Diagnostics: ");
Serial.print(AudioProcessorUsage());
Serial.print("% ");

Serial.println(AudioMemoryUsageMax());
//delay(500);

}
}
void loop() {
playFile("num1.WAV");  // filenames are always uppercase 8.3 format
// delay(5);
playFile("num2.WAV");  // filenames are always uppercase 8.3 format
// delay(5);
playFile("num3.WAV");  // filenames are always uppercase 8.3 format
delay(5);
playFile("num4.WAV");  // filenames are always uppercase 8.3 format
delay(5);
playFile("num5.WAV");  // filenames are always uppercase 8.3 format
delay(5);
playFile("num6.WAV");  // filenames are always uppercase 8.3 format
delay(5);
playFile("SDTEST2.WAV");  // filenames are always uppercase 8.3 format
delay(500);
playFile("SDTEST3.WAV");
delay(500);
playFile("SDTEST4.WAV");
delay(500);
playFile("SDTEST1.WAV");
delay(500);
}`

Another edit for faster play of those short sounds ....
`void playFile(const char *filename)
{
Serial.print("Playing file: ");
Serial.println(filename);
playWav1.play(filename);
delay(25);
while (playWav1.isPlaying()) {
Serial.print("Diagnostics: ");
Serial.print(AudioProcessorUsage());
Serial.print("% ");

Serial.println(AudioMemoryUsageMax());
//delay(500);

}
}
void loop() {
playFile("num1.WAV");  // filenames are always uppercase 8.3 format
// delay(5);
playFile("num2.WAV");  // filenames are always uppercase 8.3 format
// delay(5);
playFile("num3.WAV");  // filenames are always uppercase 8.3 format
delay(5);
playFile("num4.WAV");  // filenames are always uppercase 8.3 format
delay(5);
playFile("num5.WAV");  // filenames are always uppercase 8.3 format
delay(5);
playFile("num6.WAV");  // filenames are always uppercase 8.3 format
delay(5);
playFile("SDTEST2.WAV");  // filenames are always uppercase 8.3 format
delay(500);
playFile("SDTEST3.WAV");
delay(500);
playFile("SDTEST4.WAV");
delay(500);
playFile("SDTEST1.WAV");
delay(500);
}`

Should work now with ever frequency between 88 and 108 (and way less or more)

Should work now with ever frequency between 88 and 108 (and way less or more)

now fixed ;)


NavigationExperimental anim

when using ToolBarAndroid, the NavigationExperimental default push and pop anim in android have a white space, how to fix?

Thanks for your question!
We want to make sure to keep signal strong in the GitHub issue tracker – to make sure that it remains the best place to track issues that affect the development of React Native.
Questions like yours deserve a purpose-built Q&A forum. Would you like to post this question to Stack Overflow with the tag #react-native? We'll be happy to answer there. Post a link to your Stack Overflow question here, so that we don't lose track of it.


Global Filtered Topic (Suppressed Alarms)

It would be useful to have a global mask topic that is a filtered version of the active-alarms topic, but filtered by global filter rules.   A Kafka Streams app could do the filtering.  A command topic would also likely be needed to instruct the Streams app what to filter.  Currently each consumer (client) can do local filtering based on criteria such as category or location, but this applies only to the local client (not all clients).

We may actually want a separate global topic per each control room (as each control room may want to filter out things differently).  For example, we may want a mask topic for the following control rooms:
CEBAF
LERF
CHL
UITF

In progress: https://github.com/JeffersonLab/alarms-filter
Streams app that provides ability to configure a set of output topics with custom filters applied to the active-alarms topic.  This allows any consumer to share the filtered topic (as opposed to a consumer local filter, which isn't shared)

We might want to consider having a isMaskable attribute on alarms.  A scenario I'm thinking of is something like the radcon tritium alarms that were in Hall A a while back.  Michele had to put them into "SITE" so that they wouldn't be disabled when Hall A wasn't running because they had to be armed all the time, not just when Hall A was taking beam.   Maybe we can up with other options, but the ones I see now are.    1)  Put things "out-of-place" to avoid them being masked or 2) allow items to be excluded from the facility mask.

Terminology note:
ANSI/ISA 18.2-2016 defines three alarm states in which an alarm is "turned off"

Suppressed (by design): "prevent the annunciation of the alarm to the operator when the alarm is active"; "EXAMPLE: shelve, suppress by design, remove from service"
Shelved: "temporarily suppress an alarm, initiated by the operator, with engineering controls (e.g., time-limited) that unsuppress the alarm"
Out-of-Service: "state of an alarm during which the alarm indication is indefinitely suppressed, typically manually, for reasons such as maintenance"

I believe we use term shelve and out-of-service as indented by the standard, but we casually use many names for "suppressed by design" including:

Filtered
Masked
Disabled
Off

This is likely because the word "suppressed" is fairly generic and in fact occurs in the definition of shelved and out-of-service (the distinction there is one is shelved is temporary / time limited whereas out-of-service is indefinite.   The third category "suppressed by design" is a sort of catch-all that we're using for the scenario where a portion of the machine is turned off because we're not using it so we don't want to see alarms from that part of the machine.  We may need to clarify which terms we are going to use, though I think we do agree there are at least three use cases here.

Note: it is possible for an alarm to be in all two states at once: (1) flagged as out-of-service (broken) OR shelved,  (2) filtered out of view because it is located in a portion of the machine that isn't part of the program
It is fine that these combinations are possible, even desirable as what is part of the program probably should be tracked separately from what is broken (and definitely separately from what is shelved).
Separating what is broken from what is shelved is not as useful, but not a problem either.  Since an alarm should rarely be out for maintenance we could add a boolean to the registered-alarms topic.  Alternatively, we currently have an indefinite option on shelving, which could be used as "out-of-service".  The GUI calls this disabled now.   We could rename the shelved-alarms topic to suppressed-alarms to clarify that both indefinite and timed suppressed alarms are captured there if we take the definition of shelved to mean only timed suppressions.   Or just document clearly that we use an indefinite shelving to mean out-of-service.

I forgot to mention "suppressed by", which is a reference to a parent alarm that suppresses a given alarm in a hierarchy.  Given these four suppression modes how about we use the following definitions:
Alarm Suppression States

Name
Duration
Definition

Shelved
Short with expiration
A nuisance alarm can be temporarily shelved with a short expiration date

Disabled
Indefinite
A broken alarm can be flagged as out-of-service

Filtered
Indefinite
An alarm can be "suppressed by design" - generally a group of alarms are filtered out when not needed for the current machine program

Masked
Only while parent alarm is active
An alarm can be suppressed by a parent alarm to minimize confusion during an alarm flood and build an alarm hierarchy

We need to determine if we must independently track each of these alarm suppression states (should they be mutually exclusive?) because they may overlap in all sorts of ways and at transitions the correct effective suppression state must be determined ideally without coupling the various mechanisms that suppress alarms.  There is a calculated "effective" suppression state, but it might be best to calculate it by combining the independent states to avoid confusion and provide a clear audit trail.
If any item on the list changes then all suppression rules must be re-applied in order of precedence (starting at 1).  For example if an alarm is removed from a filter all the other suppression states must be re-evaluated, meaning we need to store state information about what other rules would have been in effect if the filter hadn't been in effect - such as continuous shelving.
It is possible to create a "suppressed-alarms" topic, with message key of both alarm name and suppression reason such that we can store all of the possible suppression state in one topic.   We could create a new topic alarm_state that stores the calculated effective state (could even compute final effective state considering acknowledgements and active-alarms too).  Or clients can do the calculation.   If we have a "suppressed-alarms" topic, our current filter-app prototype will need to change as it writes to a separate topic.

Filter flow might look like:
Source -> [active-alarms] -> disabled-app -> [disabled-alarms] -> filter-app -> [filtered-alarms] -> mask-app -> [masked-alarms] -> delay-app -> [delayed-alarms] -> state-calculator -> [operator-alarms]

Each suppression app would update suppressed-alarms topic

state-calculator computes effective state

honors acknowledgements
honors suppression precedence
honors active state
honors registered alarms (optional) - do you want to see state "Normal" alarms listed?

States:
NORMAL <-> ABNORMAL (active)
ACKNOWLEDGED <-> UNACKNOWLEDGED
SUPPRESSED <-> UNSUPPRESSED

plus combinations and variants of suppressed

I think you're on the right track...layering the various "suppressions"  That way, if there is any other type comes, up it would be easier to add.
Can this also apply to "calc/rules based alarms?" Isn't it just another form of alarm/don't alarm?
I think you're definitions are good.

Yeah, CALC alarms would likely go in that flow as well - probably before the disabled-app.   The delay-app would also likely handle off-delays as well (not just on-delays).

Layering these things makes more and more sense...
Will it be onerous?

Especially since we're not only talking about global suppression, but instead individual and groups of alarms...

If we create a bunch of separate apps it certainly is flexible, but it'll be costly and unwieldy as well (lots of moving parts and lots of duplicate work).   It might make more sense to divide suppression in general into two pieces:

Suppression Rule Processor - App responsible for keeping the suppressed-alarms topic up-to-date based on registered-alarms
Active Alarm Suppressor - App responsible for actually creating a new output topic with suppressed active-alarms

Probably consolidate all of this into the alarm-filters project and rename it alarm-suppressors instead.  See:  https://github.com/JeffersonLab/alarms-filter/issues/1


Create a new image where plugins cannot be built at runtime

The current release image of prestd is using an image whose purpose is to compile go code. That image contains a lot of build dependencies that may contain security vulnerabilities.
This pull request fixes that issue by creating a new pRest image using a debian image as the runtime image. The version of the debian image is the same version used for the base of the golang:1.22.2 image. Users of that image will not be able to dynamically compile plugins at runtime (they will need to create a multi-stage build and copy the plugins in the prest-noplugins image).

Summary by CodeRabbit

New Features

Introduced Dockerfile configuration for building container images without plugin support.
Added functionality to build and push Docker images tagged as beta-noplugins, latest-noplugins, v1-noplugins, and $DOCKER_TAG-noplugins.

Chores

Expanded Docker scripts to support additional tagging and pushing of images without plugins.

Hi @bouchardmathieu-qc thanks for the contribution, looks good to me, could you just address the suggestions made by coderabbit?


Bug fix: wrong reading information

The app will have a NullPointerException if we add the last person to a team and delete a person before him, then close the app and restart it. Person's Identity code will not be read correctly.
The exception will cause the whole app to crash.
Let's fix this bug by making the Persons' Identity codes be the same as what is stored in the file when reading information from the file.
In this way, there will not be a NullPointerException in this situation.

In summary, persons' ID will never change unless he/she is deleted and added back again. This minimizes possibilities of bugs


Consider adding Codetta to the list of coding fonts.

Codetta is a font with special ligatures for C#.
It was developed to have a clear and crisp appearance with typically used font sizes of 9Pt and 10Pt but also to scale well on high dpi monitors.
It was taken care that it doesn't use much space and has a better readabillity than the commonly used Consolas in VisualStudio.

@Proeller Thank you for you good advice for the font Codetta.I'll add it later.


fetcher proxy not take effect when fetch_type='js'

看了下代码，貌似fetcher的proxy没有传给phantomjs_fetcher.js使用

phantomjs doesn't support set proxy on-the-fly.


cannot access AWS eks

Describe the bug
Cannot access AWS EKS. It doesn't want to connect on a newly installed maces Sequoia.
To Reproduce
Steps to reproduce the behavior:

Install lens from dmg downloaded from the website.
Get access to some AWS EKS clusters.
export the KUBECONFIG variable to the path of the kubeconfig you wanna store the kubeconfig at.
run aws eks update-kubeconfig --name  --region
open Lens and browse the cluster and click on it to connect.

Expected behavior
the application would connect to the cluster and show all resources

Screenshots

Environment (please complete the following information):

Lens Version: 2024.9.300059-latest
OS: macOS Sequoia 15.0.1
Installation method (e.g. snap or AppImage in Linux): dmg

Logs:
When you run the application executable from command line you will see some logging output. Please paste them here:
2024-10-14T16:58:08.617Z info: 	[INTERNAL-PROXY-MAIN]: Connection Update
2024-10-14T16:58:08.622Z error: 	[INTERNAL-PROXY-MAIN]: Connection Update
2024-10-14T16:58:08.623Z warn: 	[INTERNAL-PROXY-MAIN]: Proxy request for clusterId="7c713451c33e0926551505d9d7c47470" to url="/version" has statusCode=500
2024-10-14T16:58:08.623Z error: 	[CLUSTER-MANAGER-MAIN]: Failed to connect to clusterId="7c713451c33e0926551505d9d7c47470": Error: Failed to GET "https://<IP_ADDRESS>:52756/7c713451c33e0926551505d9d7c47470/version"
2024-10-14T16:58:08.623Z error: 	[INTERNAL-PROXY-MAIN]: Connection Update
2024-10-14T16:58:10.209Z info: 	[CLUSTER-MANAGER-MAIN]: Ensuring cluster connection is active
2024-10-14T16:58:10.210Z info: 	[INTERNAL-PROXY-MAIN]: Connection Update
2024-10-14T16:58:10.220Z error: 	[INTERNAL-PROXY-MAIN]: Connection Update
2024-10-14T16:58:10.221Z warn: 	[INTERNAL-PROXY-MAIN]: Proxy request for clusterId="7c713451c33e0926551505d9d7c47470" to url="/version" has statusCode=500
2024-10-14T16:58:10.222Z error: 	[CLUSTER-MANAGER-MAIN]: Failed to connect to clusterId="7c713451c33e0926551505d9d7c47470": Error: Failed to GET "https://<IP_ADDRESS>:52756/7c713451c33e0926551505d9d7c47470/version"
2024-10-14T16:58:10.222Z error: 	[INTERNAL-PROXY-MAIN]: Connection Update
2024-10-14T16:58:11.341Z info: 	[CLUSTER-MANAGER-MAIN]: Ensuring cluster connection is active
2024-10-14T16:58:11.341Z info: 	[INTERNAL-PROXY-MAIN]: Connection Update
2024-10-14T16:58:11.353Z error: 	[INTERNAL-PROXY-MAIN]: Connection Update
2024-10-14T16:58:11.354Z warn: 	[INTERNAL-PROXY-MAIN]: Proxy request for clusterId="7c713451c33e0926551505d9d7c47470" to url="/version" has statusCode=500
2024-10-14T16:58:11.355Z error: 	[CLUSTER-MANAGER-MAIN]: Failed to connect to clusterId="7c713451c33e0926551505d9d7c47470": Error: Failed to GET "https://<IP_ADDRESS>:52756/7c713451c33e0926551505d9d7c47470/version"
2024-10-14T16:58:11.355Z error: 	[INTERNAL-PROXY-MAIN]: Connection Update

Kubeconfig:
Quite often the problems are caused by malformed kubeconfig which the application tries to load. Please share your kubeconfig, remember to remove any secret and sensitive information.
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <certificate>
    server: https://<randomstring>.gr7.us-east-2.eks.amazonaws.com
  name: <clustername>
contexts:
- context:
    cluster: qa-eks
    user: qa-eks
  name: qa-eks
current-context: qa-eks
kind: Config
preferences: {}
users:
- name: <arn>
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - --region
      - us-east-2
      - eks
      - get-token
      - --cluster-name
      - qa-eks
      - --output
      - json
      command: aws
      env:
      - name: AWS_PROFILE
        value: <profilename>
      interactiveMode: IfAvailable
      provideClusterInfo: false

@zozo6015  Thank you for reporting the issue and providing detailed steps to reproduce it.
To help us better understand the problem, could you please try the following workaround? Modify the generated kubeconfig file so that the command:
command: aws is replaced with the absolute path to the AWS CLI
If this resolves the issue, it might be related to your shell environment. Please let us know if this helps or if the problem persists.
Looking forward to your feedback, and thank you for your patience!


tailscale: 1.68.1 -> 1.68.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/project/tailscale/versions.
meta.description for tailscale is: Node agent for Tailscale, a mesh VPN built on WireGuard
meta.homepage for tailscale is: https://tailscale.com
Updates performed

Golang update

To inspect upstream changes

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.68.2 with grep in /nix/store/vsznwv028zcsqayzzjywcp4pcm5r4dyj-tailscale-1.68.2
found 1.68.2 in filename of file in /nix/store/vsznwv028zcsqayzzjywcp4pcm5r4dyj-tailscale-1.68.2

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

3 package rebuild(s)

First fifty rebuilds by attrpath

tailscale
trayscale

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/vsznwv028zcsqayzzjywcp4pcm5r4dyj-tailscale-1.68.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A tailscale https://github.com/r-ryantm/nixpkgs/archive/203fb14ff3032c0fd64018f479bc6c3975e1bfa2.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/203fb14ff3032c0fd64018f479bc6c3975e1bfa2#tailscale

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/vsznwv028zcsqayzzjywcp4pcm5r4dyj-tailscale-1.68.2
ls -la /nix/store/vsznwv028zcsqayzzjywcp4pcm5r4dyj-tailscale-1.68.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

tailscale
trayscale

Maintainer pings
cc @martinbaillie @06kellyjac @mfrw for testing.

Result of nixpkgs-review pr 324226 run on x86_64-linux 1

2 packages built:

tailscale
trayscale

Tested Locally


[WIP] Remove Imports Microsoft.VisualBasic, part 6

part 1: #1369
part 2: #1370
part 3: #1371
part 4: #1372
part 5: #1373
Directories:
snippets/visualbasic/VS_Snippets_VBCSharp
snippets/visualbasic/VS_Snippets_WebNet
snippets/visualbasic/VS_Snippets_WindowsServer

This had too much conflicts from the removal of Imports System. I think I'll re-open a new one for that later today.

w


Problem of Table 2

The Table 2 seems confused. Why L_{margin} + MSP are not in green on SST2 dataset and in green on IMDb dataset?
You write 'Results where the contrastive loss improves OOD detection on both evaluation metrics are highlighted in green.' And lower FAR95 is better.

The SST2 cell should be in green, and the IMDB cell shouldn't. Thank you for pointing this out.

OK. Thank you for your reply.


Introduce fake symbols that were present with the old concurrency ABI.

The concurrency ABI has shifted considerably, including renaming many
symbols. However, there remain some existing libraries that link
against these symbols and aren't necessarily immediately rebuilt. Add
some "fake" symbols for the ones that we need, so those existing
libraries can link against a new _Concurrency library... even though
the resulting binary would not be able to execute. This is a grotesque
hack and a stop-gap.
Speculative fix for rdar://79298287.

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - f566c07e76810fa935ee6fbef2c74bfcc5ccf6cb

@swift-ci please test

We only care about this on 5.5, so I'll close this out


Crash in linux smoke tests

Found a crash dump attached to the smoke test: https://dev.azure.com/vscode/VSCode/_build/results?buildId=91614&view=results
Dump:
crash-dump-linux-x64-smoke-1.zip

Symbolicated file:
symbolicated-87.log

Merging to https://github.com/microsoft/vscode-internalbacklog/issues/3454


refactor: Consolidate d3 rendering

This PR aims to:

consolidate D3 rendering across charts and axes by adding renderContainer and maybeTransition methods,
properly disable transitions with the help of maybeTransition function. This function makes it possible to specify the rendering logic when transitions are enabled (t) and disabled (s). I also shared the logic to base this decision on transitionStore,
fix brush disabling the transitions when metadata panel is being toggled,
improve performance of using Zustand stores (atomic state picks),
fix jumping charts during interactions with interactive filters,
introduce minor style adjustments.

Hi @adintegra, thanks for reviewing! I also noticed and fixed this in #1115 👌


Linux samsung-s10 builds pending for over 90 minutes

The tree is currently red because LInux samsung-s10 builds have been pending over 90 minutes. Maybe we have no live bots? cc infra on-call @godofredoc

Bots were quarantined because they failed tasks several times in a row( 6+ times) which in this case was expected because of the bad roll in: https://github.com/flutter/flutter/commit/1add0d790d78918a560387c23802afd2979d15d7

Manually recovering the devices.

Tests are now running.

Tests are now running.

Thanks! We're now green again.


Roll Flutter Engine from 10a1f9cb74c9 to d5f6b007c79b (1 revision)

https://github.com/flutter/engine/compare/10a1f9cb74c9...d5f6b007c79b
2023-08-01<EMAIL_ADDRESS>[iOS] Add darwin_extension_safe flag and use UIScene api when building for extensions (flutter/engine#43449)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC jacksongardner@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


integration tests should not use 3rd party services

integration tests should use mocks for al 3rd party services

This will require some refactoring to be able to mock / implement a dry run version of the external api service calls.
Since the dependency is injected further down from the main entry point, it is not straightforward to pass a mock service instead. If we are able to instantiate all the external service dependencies closer to the main entry, it would be easier to mock.
I propose we do it for one service at a time and test.

[ ] Biostudies
[ ] Biosamples
[ ] ENA Webin API

Break down into tickets for the different services, under this epic. If there is a small ticket, achievable this sprint, please let's have it here/


[logging] Introducing structured log API

This PR introduces structured log API for libra.
Summary
In a nutshell, API introduces two things

StructuredLogEntry class and send_struct_log! macro for directly composing structured log
Bridge between traditional log! macro and structured logging API

Direct API
pub struct StructuredLogEntry {
    text: Option<String>,
    pattern: Option<&'static str>,
    name: Option<&'static str>,
    module: Option<&'static str>,
    location: Option<&'static str>,
    git_rev: Option<&'static str>,
    data: HashMap<&'static str, Value>,
}

impl StructuredLogEntry {
    pub fn new_unnamed() -> Self { /* ... */ }
    pub fn new_named(name: &'static str) -> Self { /* ... */ }
    /* ..Bunch of builder style setters for chained initialization such as
        entry.data(a, b).data(x, y).. */
}

// Usage:
send_struct_log!(StructuredLogEntry::new_named("Committed")
   .data("block", &block)
   .data("autor", &author))

Arguments passed to .data will be serialized into json, and as such should implement Serialize.
Only static strings are allowed as field names.
send_struct_log! should be used to send structured log entry.
This macro populates metadata such as git_rev, location and module, and also skips evaluation of StructuredLogEntry entirely, if structured logging is disabled
Log macro bridge
Crate owners are not required to rewrite their code right away to support new structured logging.
Importing logger crate will automatically emit structured logging on every log(debug!, info!, etc) macro invocation, unless no_struct_log feature was enabled on this crate.
Note: Only enable 'no_struct_log' on leaf binary crates, never on library, as it might disable structured logging for other crates, due to how cargo handles features at this moment.
So
info!("Committing {}", block);

Will emit(in addition to regular text log) structured log such as
{
  pattern: "Committing {}",
  data: {
    block: "<id>"
  },
  ...metadata...
}

There are few caveats to automatic structured logging generation

Argument values for structured logging will be serialized with Debug implementation(vs proper json serialization if send_struct_log! macro is used), regardless of what formatter is used for textual log. As a consequence, this means, that every log argument must implement Debug. In theory, this is extra limitation, but in practice currently in libra node crate and it's dependencies there was no single log argument that would not satisfy this criteria.

Field names will be automatically evaluated if expression is a single identifier, as in example above field block will get human readable name in json. However, if more complex expression is passed, structured log field name will be based on position of argument: "_0", "_1", etc:

info!("Committing {}", block.id());
//->
{
  data: {
    "_0": "<id>"
  },
  ...metadata...
}

Another way to set proper name to json fields is to use named format arguments:

info!("Committing {id}", id=block.id());
//->
{
  data: {
    "id": "<id>"
  },
  ...metadata...
}

Structured log sink
Application must define implementation of StructLogSink interface in order to direct structured logs emitted by send_struct_log and other macros. This sink can be only initialized once, by calling set_struct_logger function.
Currently 3 implementations for StructLogSink exist:

NopStructLog ignores structured logs
PrintStructLog immediately prints structured logs to stdout
FileStructLog prints structured logs into provided file. Using this logger creates separate thread for writing files and structured logging itself is asynchronous in this case.

:umbrella: The latest upstream changes (presumably d6a2f89527f1843b36780141679796fed6f76a2f) made this pull request unmergeable. Please resolve the merge conflicts.

Upd: Added FileStructLog

So my current thought is that structured logs would have to go to separate file, and we would need separate logstash config to pull them
Motivation for that is that we would have something in stdout/err anyway, like panic messages, etc, so mixing in structured logs there would make things break from time to time.
@sherry-x @davidiw  What do you guys think?

Added crate level documentation(majority of it duplicates this PR's description) and fixed few nits(network crate cargo and pub use Writer)

This is awesome!! Logstash works pretty well with json format, so this should be good!
How do we make the structure log store in a separate file? Currently we just direct all the log stream from docker to a local file, and I don't think we can direct output based on log type.

Split implementations into struct_log.rs and text_log.rs. Macros are kept in lib.rs, which is normal practice
Removed all new pub use re-exports

Addressed feedback
@bors-libra r=davidiw

:pushpin: Commit 66431b9 has been approved by davidiw

@bors-libra r=davidiw

:pushpin: Commit 5018961 has been approved by davidiw

:hourglass: Testing commit 5018961a6ab3bba766ca741b2e7e61491dcb005c with merge 9e658575cd3d87a7f4ecf16b416b40a38a268e43...

@bors-libra r=davidiw

:pushpin: Commit c4a418d has been approved by davidiw

@bors-libra r=davidiw

:pushpin: Commit 5f39b73 has been approved by davidiw

:hourglass: Testing commit 5f39b731cbd1eaf00fa514fbc75535a096dfae7d with merge 4c017aa74295d2a5f6bf15a0f14d71928c6297ca...

:sunny: Test successful - checks-actions_land_blocking_test, checks-circle_commit_workflow
Approved by: davidiw
Pushing 4c017aa74295d2a5f6bf15a0f14d71928c6297ca to master...


MuscledAnt-v0 becomes unstable.

See http://www.mujoco.org/forum/index.php?threads/debugging-instability.3909/

By chance I saw the behavior which was causing this "instability" it was extreme. Somehow every tendon was fully activated and the model was vibrating at high speed. I think it's safe to say that the simulation never had a chance under these circumstances and that the model is not at fault.


Switch to use uPheno base and import phenotype ontologies separately

Fixes #24 #34 #36 #37 #82 #88 #91 #96
But really is a refactoring of the full import system.
This ensures that we always have the latest phenotype ontologies in phenio, independent of any particular phenio release. I have eyeballed the resulting ontology and it has still waaaay too many dangling classes, I will try to clean those in subsequent commits.
@caufieldjh I made this a draft because I need to do quite a bit of clean up of phenio which has become quite messy in terms of dangling classes. Will come back to you updates, but feel free to look at the PR and tell me your general opinion of my suggestion

makes sense to me conceptually - seems like it could become an issue if there's discrepancies between the most recent version of a phenotype ontology and upheno, but I will trust you on this

What remains to be done on this PR before it's ready to merge? Just reviewing?

@caufieldjh It is ready from my perspective!

Great - thanks @matentzn and @gouttegd !


Guard is not redirecting where page was before after login

Guard is not redirecting where page was before after login
for example :
I am on page : http://laravel.dev/lists
which shows me list of item
Item 1
Item 1
Item 1

but above item can only accessible to authorized user
Issue
when i click on Item 1 so if user is not logged in then it goes to http://laravel.dev/login then user proceed to login but after login it must redirect to http://laravel.dev/lists/1 but it is redirecting to http://laravel.dev/home.
I have tried
public function showLoginForm()
{
    session()->put('url.intended',url()->previous());

    // or

    session()->put('url.intended', url()->current());

    return view('user.auth.login');
}

but above is not working, it is redirecting to http://laravel.dev/lists
laravel own auth is working fine.
but hesto multi-auth is not working
is there any solution for this

Laravel 5.5?

using Laravel 5.4

same problem laravel 5.5

same issue with Laravel 5.5. Any have solution?

I guess this has something to do with the Login controller code as we mention redirect page there. Changing that should fix the issue. I'm not sure, i'll try and let you know if it works.


Pass framesInfo to the CallBack

Get frames Info from the GifReader object and pass to the CB as additional param.

@Snelius30 this looks good! Could you:

Document in the README that framesInfo gets returned as an additional argument in the GIF case
Remove semicolons for style consistency

Thank you!

OK, will do that.

Please check.

@Snelius30 this looks great! Sorry, one more request. I think the ordering of the docs is a bit funny now... it might be better if it looked like this:
...

**Returns** An ndarray of pixels in raster order having shape equal to `[width, height, channels]` and `frameInfo` param if available (for animated GIFs).

**Note** For animated GIFs, a 4D array is returned with shape `[numFrames, width, height, 4]`, where each frame is a slice of the final array.

**Note** `frameInfo` is an Array of Objects with these fields:
 Name|Type|Description
----|-----|-----------
x | Integer | Image Left Position
y | Integer | Image Top Position
width | Integer | Image Width
height | Integer | Image Height
has_local_palette | Boolean | Image local palette presentation flag
palette_offset | Integer | Image palette offset
palette_size | Integer | Image palette size
data_offset | Integer | Image data offset
data_length | Integer | Image data length
transparent_index | Integer | Transparent Color Index
interlaced | Boolean | Interlace Flag
delay | Integer | Delay Time (1/100ths of a second)
disposal | Integer | Disposal method
 See GIF spec for details. Summary http://www.onicos.com/staff/iz/formats/gif.html

I think the note about the 4d array is more important than the frameInfo description, so this way it doesn't get pushed to the bottom.
I also think it might be better to contribute the frameInfo table to the omggif documentation, but since omggif has no documentation at the moment, I think it's ok to commit it to this README. I'd suggest opening a pull request over there with some markdown documentation though, and updating this README to link to that doc once it's accepted.

I opened an issue in omggif because I want to make sure it's safe to consume this API.

OK, we have corrected ordering now. I just thought that we need to explain frameInfo as mention but u right this one correction is better.

Aawesome, thanks! Looks great. :heavy_check_mark:
I want to wait a couple of days to see if there's a response in the omggif issue thread to make sure we're doing the right thing.. Otherwise I think this is pretty good to go. I also want to make sure we're doing the right thing adding an extra argument.. are there other arguments we might also want to add in the future?

Sure, you can extend CB params as u wish in future.


[SvgIcon] Merge all children to provided component

Close #33421
before: https://codesandbox.io/s/svgicon-titleaccess-before-w11hgv?file=/src/App.tsx
after: https://codesandbox.io/s/svgicon-titleaccess-after-jnskxf?file=/src/App.tsx

[x] I have followed (at least) the PR section of the contributing guide.

First of all, take a look at the Emotion source code.
return (
  <>
    <Insertion
      cache={cache}
      serialized={serialized}
      isStringTag={typeof FinalTag === 'string'}
    />
    <FinalTag {...newProps} />
  </>
)

Insetion renders a style element and FinalTag renders content elements.
Emotion doesn't naturally support rendering other children in newProps  passed to the FinalTag when FinalTag isn't a single string tag.

So MUI tries to render these children in a limited way which doesn't work when the provided component isn't a single string tag.
const otherChildren = Children from others;
const props = {children: otherChildren};

const ICanRenderOtherChildren = (componentProps: any) =>
;
<ICanRenderOtherChildren {...props} />

const ICantRenderOtherChildren = (componentProps: any) => (

    {`I have my own children so I don't need to render you.`}

);
<ICantRenderOtherChildren {...props} />

In order to render these children, we have to code in a weird way which also intrudes into the provided component. There is an example in the before link which can solve the related issue.
// A weird way
const ProvidedComponent = (componentProps) => (
  <svg {...componentProps}>
    <path d="M10 20v-6h4v6h5v-8h3L12 3 2 12h3v8z" />
    {componentProps.children} // Intrude into this component.
  </svg>
);

This PR can solve the above problem and discard the weird use case.
The new logic merges other children (if there are) and my own children (if there are) to the provided component since Emotion render FinalTag through as. The PR also provides inheriting and overriding titleAccess from provided component.

@siriwatknp Have a look. I've avoided creating new component in render phase.

Two things rate a mention:

Same as before, extracted elements from a provided component still have both origin classes(from the provided component) and current classes(from SvgIcon).
Different from before, event handlers bound to these extracted elements will be missing.

Examples are given in the provided code sandboxes.


Add Color Picker  component

@workingxx92 why not using native color picker?
<input type="color">

@saadeghi
I cannot close dialog color when click outside

@workingxx92 I don't see any problem 🤔
https://play.tailwindcss.com/dTrEqcgUNE
https://user-images.githubusercontent.com/7342023/127782302-7f646172-7e28-4a1e-a98b-14f4f3a4b600.mp4

@saadeghi .
Please run on firefox. I cannot close dialog color when click outside on firefox

Firefox opens OS color picker as a popup window. Some people like it this way, some don't.
Anyway, it's not possible to make a custom color picker in a CSS library like daisyUI, so if you need a custom color picker, you need to use a JS plugins for that.
A few to mention:

https://github.com/jaames/iro.js
https://github.com/Simonwep/pickr
https://github.com/casesandberg/react-color
https://github.com/efeskucuk/svelte-color-picker
https://github.com/codeosemeke/vue-colour-picker


Add eglot example for Emacs.

I don't think just adding the server to eglot will work. You need custom config to get it to work. I have this in my dotfiles which was stolen from some reddit post.
https://github.com/meain/dotfiles/blob/3090261e76b5f4bd2dfaf6dc7f67a743b4ccb42c/emacs/.config/emacs/init.el#L1060-L1104
lsp-mode I believe has support for it though: https://github.com/emacs-lsp/lsp-mode/blob/master/clients/lsp-sqls.el

I don't think just adding the server to eglot will work for code actions. You need custom config to get it to work. I have this in my dotfiles which was stolen from some reddit post. https://github.com/meain/dotfiles/blob/3090261e76b5f4bd2dfaf6dc7f67a743b4ccb42c/emacs/.config/emacs/init.el#L1060-L1104
lsp-mode I believe has support for it though: https://github.com/emacs-lsp/lsp-mode/blob/master/clients/lsp-sqls.el

Thanks, I didn't try code-actions, just test auto completions, it enough for me.


[Failing Test]: Python Github actions tests are failing due to update of pip

What happened?
Pip was updated to 23.3 on October 15th https://pypi.org/project/pip/#history and there are some failures observed in the Python test suites in GitHub actions that use pip 23.3.  If multiple virtual environments are installing the same package at the same time one of them can/will fail with a cache error similar to the one below.
For example, https://github.com/apache/beam/actions/runs/6538469207/job/17754466856?pr=29016 and https://github.com/apache/beam/actions/runs/6538469213/job/17754476104?pr=29016

Related issued I found on pypa/pip: https://github.com/pypa/pip/issues/12361.
Issue Failure
Failure: Test is flaky
Issue Priority
Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)
Issue Components

[X] Component: Python SDK
[ ] Component: Java SDK
[ ] Component: Go SDK
[ ] Component: Typescript SDK
[ ] Component: IO connector
[ ] Component: Beam YAML
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[ ] Component: Google Cloud Dataflow Runner

Looks like this is no longer occurring


Decrease the unmarshall error severity from err to warn

SUMMARY
This PR only decreases the unmarshall error severity from err to warn. To make it a bit better, we can contribute to the slack lib and replace:
err := fmt.Errorf("RTM Error: Received unmapped event %q", typeStr)

(source)
with error type UnmappedError. This won't be a breaking change for them, so we should be able to merge that.
As a result, we will know the details about the error and could implement sth like:
// handlingEvents holds event names which have corresponding handling logic.
var handlingEvents = sets.NewString("message", "connected")

// trimmed

      case *slack.UnmarshallingErrorEvent:
        err, ok := ev.ErrorObj.(slack.UnmappedError)
        if ok && !handlingEvents.Has(err.EventType()) { // it's not an err for us, as we don't care about this event type.
          b.log.Debugf("Slack unmarshalling error: %+v", ev.Error())
          continue
        }

        b.log.Errorf("Slack unmarshalling error: %+v", ev.Error())

In that way even if someone will configure custom Slack bot with wrong event subscription we will be able to “ignore them”.
WDYT? Is it worth it? If we stay with RTM, then it's the only option.
FINDINGS
I wanted to describe the workaround for this issue by creating the own Bot. I found that the current Bot is based on RTM. This means that it issues the old API.

In this case, we are not able to:

reduce the app permission
select events to which we want to be subscribed

The best option would be changing the implementation from RTM to WebSocket mode. Should I  create an issue?
Related issue:

#619

The best option would be changing the implementation from RTM to WebSocket mode. Should I create an issue?

Can we switch to WebSocket mode? Is everything what we use supported (I mean in the Go Slack library)?
If yes, please create issue 👍  And then I would skip the contribution when it comes to the RTM method.

Issue created: https://github.com/infracloudio/botkube/issues/631 👍


[Extensions] Comment Notifications

Current Behavior
Users receive email notifications of their own comments, but shouldn't.
Desired Behavior
Rules:

Only be concerned with one Comment (Component) at-at-time. Keep the scope of your logic to a single Comment at-a-time, irrespective of how many other Comments are in the Design.

Always notify the design owner, unless the design owner is the individual making the comment. See #2.

First process: Gather all implicated users.

Automatically include the design owner.
Include all other users who have been “@mentioned”.

Second process: Filter out people who should not be notified.

Remove the user who is sending the message.

this includes the design owner
this includes the current user if they were previously “@mentioned”

Remove any user who’s preference is set to not be notified.

Contributor Guides and Handbook

📚 Meshery Build and Release Strategy
🎨 UI Design For Meshery Extensions Figma in Figma (open invite)
📚 Meshery Extensibility Providers
⌨ Meshery API Docs
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

@vaibhavpnimkar, are you ready for this, yet?

yes!

I already have issues assigned, so I will start working after resolving those issues first.

@leecalcote can I try this out while my other PR is getting reviewed?

Removing myself from the assignee as Yash and Akshay are working on it.

Go ahead and assign them.

I can't assign issues to users atm. I don't think I have the permission for it.

I can't assign issues to users atm. I don't think I have the permission for it.

@AjaxAueleke, let's get that changed. 😉

This issue is fixed


Dismiss button  tag is not properly formatted to work for Enter key

I saw the reply on a previous issue and I have a simple fix for this:
https://github.com/insites/cookieconsent/issues/453#issue-355999452
All you need to do is add href="#" to the dismiss button  tag and the Enter key will automatically work --- this is very important in order to be WCAG 2.0 / WCAG 2.1 compliant and make this Dismiss button accessible to all users and it will work with the "Enter" key.
Current  tag:
Accept
New  tag:
Accept
I'm a software engineer and I can help and contribute to making this plugin even better - it's already awesome!
Please contact me if you like me to help in any way.

@alexadevoe Alexa, we don't want to capture the enter key by default in the library. There are numerous reasons for this one of which a lot of Web Apps will have their own focus and we don't want our plugin to introduce breaking changes into current apps. The issue you linked below would be the proper way of handling this especially because there is no link, and we don't want google to start indexing that link as a part of people's site map. I could go on, but I hope this helps you understand why adding it as a part of the library isn't something we would do by default. However we may wrap our own even listener around it later and add a property to the config like allowOnKeyPress( keyCode ) as a feature in the future, so I'll mark this as a feature request.

Issue #571 is for the feature request and allows this to be closed.


ISSUE-132 Allow customising key/value suffix by configuration

Improve on https://github.com/hortonworks/registry/issues/99
Make the suffix configurable, as such reducing the need for custom concrete extensions of the serializer.

@michaelandrepearce Do you use prefix/suffix for schema names in your scenarios?

Closes #132

@michaelandrepearce We are following PR description and commits to start with respective issue  like "ISSUE-XXX "
I will take care of the current PRs and commits for now, you can follow this convention for next PRs.

@satishd sure. re use case, it was actually inline with the general work I'm trying to make migration / integration from confluent easier. In confluent they use -key and -value suffix's.
So this paired with the confluent api, and then the protocol changes would mean you could inter-op where people have confluent serdes without changing code, just can redirect the schema registry url.


refactor(tests): Prepare for wait-for-publish test changes

In #11062, we are updating cargo publish to wait until a package is published.   The problem is a lot of our tests will block until the timeout.  In finding the tests to update, I was originally relying on test failures from the extra output when timing out.  The problem is not all tests verify the test output so they don't fail.
This tries to update the tests to make the introduction of a timeout more obvious.

Adding with_stderr where it wasn't before
Moving away from with_stderr_contains for publish tests

To help with that, I made the predicates on cargo commands more consistent.
I also moved descriptions of tests to be outside of the test so I can more easily document the registry::init calls with what we are doing.

@bros r+

Oops I accidentally tagged somebody 😅
@bors r+

:pushpin: Commit 08df53ba486c61b77d2580fd1315cd3ea7ed2359 has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit 08df53ba486c61b77d2580fd1315cd3ea7ed2359 with merge 83a1f66d115506a98ce1d92d36a4abdd6ec46ac5...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing 83a1f66d115506a98ce1d92d36a4abdd6ec46ac5 to master...


Making wast parse results serializable

I'd love to use the parser and encoder from wast in JavaScript. The easiest way for that to work would be for the parsed module to be serializable and deserializable as JSON, so that the string representations can be passed to JS via wasm-bindgen. Could the various AST types be made serializable?
I know adding serde adds some heft, but I think it could be done conditionally with an off-by-default feature.
I tried doing this myself by sprinkling #[derive(Serialize, Deserialize)] everywhere, but got stuck making the lifetimes work inside the instructions macro. I don't have enough experience with rust to figure out how to get that working.

Thanks for the report! I think this is related to https://github.com/bytecodealliance/wasm-tools/issues/1395 in that the end result may be similar. I think this would be good to add, but if we could match the preexisting structure that wabt emits I think that'd be neat to avoid having multiple verisons floating around.

Oh and I should mention that a PR would be most welcome!
Do you need library support for this feature? If not I think it'd be best to just do it in the CLI by transforming the AST to a custom structure with #[derive]. I would hope that you won't need to apply #[derive] to all types in the wast crate but only the top-level ones dealing with *.wast directives, and that'll be easier when it's modeled as a different structure rather than directly #[derive]'d on all types.

Do you need library support for this feature?

That was my hope - my goal is to use this in a JS package, by making a crate which exposes two functions:

one which takes a .wat string and gives back a serialized representation of the parse result from the wast crate, and
one which takes such a serialized representation and gives me the compiled wasm binary

and then compiling that crate to wasm and calling it from JS. That would allow me to easily write transforms of wasm programs in a JS library. Doing it in the CLI wouldn't get there as easily. (I guess I could probably compile the CLI to WASI and then use it as a library by using an appropriate virtual filesystem in WASI, but that's a lot more painful.)

I would hope that you won't need to apply #[derive] to all types in the wast crate but only the top-level ones

Not everything, certainly, but you do need to represent instructions, and there's a lot of those.

Re: using wast2json (assuming that's the preexisting structure that wabt emits you're referring to), is there a way to go back from the JSON to .wat? If not it's not would only accomplish one of the directions, though I guess I could write the other one.

Ah ok, I'm erroneously conflating these two use cases then. Your use case is quite different where you want a JSON representation of the entire AST of a module. The wast2json command only returns a JSON representation of the testing directives in a *.wast file, the actual wasm files themselves are all represented as binary wasm.
I'd be a little hesitant taknig an entire module through a JSON AST but if it works for your use case it's something we can support. I suspect though that it's going to be a lot of #[derive] and, yes, we'll want them to be conditionally turned on via a feature.
For your use case I don't think the CLI would work well yeah, your use case would want #[derive] on all types for sure.