Long user input results in 400 Bad Request

Describe the bug
On long user input the bot responds with an error "Response status code does not indicate success: 400 (Bad Request).", validations for the input length don't prevent this error to show.
Version
Composer  1.4.1
Electron: 8.2.4
Chrome: 80.0.3987.165
NodeJS: 12.13.0
V8: 8.0.426.27-electron.0
Browser

[* ] Electron distribution
[ ] Chrome
[ ] Safari
[ ] Firefox
[ ] Edge

OS

[ ] macOS
[ *] Windows
[ ] Ubuntu

To Reproduce
Steps to reproduce the behavior:

Create a bot application and add a text input.
On the validatation tab add the condition "length(string(this.value))<255" and as a response on invalid prompt "the text is too long".
Add a trigger with some some phrases ie. "the machine is broken"
Put the following text on the user input "Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate".
Bot responds "Response status code does not indicate success: 400 (Bad Request)."

Expected behavior
After giving the input the bot should respond "the text is too long"
Screenshots

Additional context
The issue only affects dialogs with  LUIS recognition triggers

To clarify;

This happens (fails) before validation can be done.
It fails/repros with dotnet runtime, but works with node runtime.
Simple repro steps:

Create text prompt asking for text (using dotnet runtime)

Send text with > 500 characters.

After setting up a demo in dotnet sdk, I found the utterance with more than 500 char will get an error from luis portal.

Here is the doc for luis limits:
https://docs.microsoft.com/en-us/azure/cognitive-services/LUIS/luis-limits

Thanks @cosmicshuai for the diagnosis.  @munozemilio , @mrivera-ms , how should we proceed here?  Should we validate input length in SDK before going to the LUIS service?  Or is this expected?

Hi @munozemilio, can you respond to @taicchoumsft's question?
Thanks

@mrivera-ms / @munozemilio are you able to provide an update on this?

As per conversation with @mrivera-ms given this is a Luis limitation we won't handle it in the SDK.


Fix default session timeout

Due to a bug in Pax Web, the default session timeout for Opencast was set to 4 minutes instead of 4 hours. This caused users to be logged out automatically if they do not have an additional remember-me cookie (e.g. LTI users don't have that). This basically means that users are unable to watch videos longer than four minutes for which they need authentication.
This is a temporary fix for #6260
How to test this patch
To test this (check that the value is indeed seconds):

Set org.ops4j.pax.web.session.timeout=20
Go to http://localhost:8080, make sure to remove the check mark for remember me and log-in
Wait for 21 seconds
Go to http://localhost:8080 again

You will be logged out and asked to log in again

Your pull request should…

[x] have a concise title
[x] close an accompanying issue if one exists
[x] be against the correct branch
[x] include migration scripts and documentation, if appropriate
[x] pass automated tests
[x] have a clean commit history
[x] have proper commit messages (title and body) for all commits
[x] explain why it needs to be merged into the legacy branch, if it is targeting the legacy branch

Even if they fix it, we have to upgrade Pax Web (which for us essentially means updating Karaf) to get this fix. In that case, yes, we should update this configuration file again. But that's not happening in 16 or 17.

I opened #6295 to track the revert for probably 18.


Default route issue http://localhost:port/swagger 404 error since VS2017 15.5.1

I recently upgraded Visual Studio 2017 15.5.1.
So far everything was working fine developing .NET Core (2.0) web apis but after the updates the problems started. The problems occur on existing projects as well as newly created ones.
When I now configure the App URL settings in the project properties to start at http://localhost:50000/swagger the browser opens and shows the wrong url, namely http://localhost:50000/swagger/api/values with the correct response of the ValuesController but not the swagger ui. At this point it shouldn't have the extension /api/values since it is not configured and I don't know why or where this is added (it's not in the configuration).
If I now enter (without stopping the app) http://localhost:50000/swagger I start receiving 404 errors an urls, even on http://localhost:50000/api/values.
If I now enter (without stopping the app) http://localhost:50000/swagger/swagger !!! UI shows but with issue about swagger.json of course.
Firing end points at this step seems to work but with strange 500.19 for somes. At this step no breakpoint in the debug are active and app is not stopping anymore on any breakpoints.
Nothing in configuration have changed only upgrade to VS2017 15.5.1.
Any idea to fix ?
Someone else have issue ?

You could try to use this in Startup class
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
	app.UseSwaggerUI(options =>
	{
		options.RoutePrefix = "swagger/ui";
	});
}

Thanks for reply but it has no effect.
I tried all what is possible around RoutePrefix and at the end UI shows with cascading 3 /swagger : http://localhost:50000/swagger/swagger/swagger !!!!!
There is an issue swagger / vs2017 15.5.1 or vs2017 15.5.1 / swagger as nothing changed but it stoped working.
Yes terday evening was at 4  http://localhost:50000/swagger/swagger/swagger/swagger when decide delete .vs directory of my solution to back at least at 2 http://localhost:50000/swagger/swagger....

To let you know i found a temporary fix :
1 - keep starting url in project option : http://localhost:50000/swagger.
2 - in startup.cs :
app.UseSwaggerUI(c =>
{
// force to add another /swagger to fix issue
c.SwaggerEndpoint("/swagger/swagger/v1/swagger.json", "My API V1");
});
3 : manually enter URL in chrome when app launched : http://localhost:50000/swagger/swagger.
Seems to work fine at this step.
App stpp on breakpoint as before.
But in my idea there is something to fix.

+1
SwaggerUI suddenly stopped working after upgrading Visual Studio.
http://localhost:50000/swagger/swagger does the job as a temp workaround.

Fast fix:

Go to project->properties->debug
change App URL from http://localhost:52120/swagger to http://localhost:52120
Run
In browser go to http://localhost:52120/swagger
this fix wont change swagger urls, it is important if you have oauth authentication.

For those still experiencing this issue:

Change the application URL back to your default path without '/swagger'.
Add 'swagger' to the 'Launch Browser' property (right click project, properties -> Debug).
Open .vs/config/applicationhost.config and remove all application entries with the '/swagger' path on your site.

Should now work as expected
What was happening:
Swagger was appending the route to the end of your application path, if your application path is set as localhost:53250/swagger for example, then the swagger application will be {appUrl}/swagger which in this case would be localhost:53250/swagger/swagger.
VS have a property 'Launch Browser' which will leave the default application URL and just append the path which you want to browse, which in this case you want as swagger or your API docs.
VS also saves each and every application path you enter, so if you had once entered the path with a /swagger route, it will still be saved in the .config and hence each time you try to go to {app}/swagger it is going to the apps route, not to the route with /swagger appended.

Is this an issue with SB, and if so what needs to change because it’s not clear to me? Or, can we close the issue now?

As per Greg post 👍 here is a FUNCTIONAL launchSettings.json
{
"iisSettings": {
"windowsAuthentication": false,
"anonymousAuthentication": true,
"iisExpress": {
"applicationUrl": "http://localhost:51564/",
"sslPort": 0
}
},
"profiles": {
"IIS Express": {
"commandName": "IISExpress",
"launchBrowser": true,
"launchUrl": "swagger",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
},
"Ocuco.Web.Application": {
"commandName": "Project",
"launchBrowser": true,
"launchUrl": "api/values",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"applicationUrl": "http://localhost:51565/"
}
}
}
backed to normal declaration in startup.cs
..... configureapp
app.UseSwaggerUI(c =>
{
c.SwaggerEndpoint("/swagger/v1/swagger.json", "My API V1");
});
....
Delete ".vs" directory locally to ensure VS recreate it correctly and it works...
Then yes issue can be close on my side

The issues is with appliacationhost.config after changing project properties > debug > App url. In my case I had to remove the following node from  appliacationhost.config:
 <application path="/swagger" applicationPool="Clr4IntegratedAppPool">            <virtualDirectory path="/" physicalPath="C:\src\LM\LM.Web" />  </application>

In swagger 2.4.0 this is still a bug.
Repro steps:

Set App URL to http://localhost:xxxxx/Swagger
Run the app
Navigate to http://localhost:xxxxx/Swagger/Swagger, and find Swagger there, unable to load the swagger json file

Recovery:

Remove Swagger from the App URL
Delete the .vs folder, and allow it to be rebuilt
All is right with the world

you must deploy .Net Core application to IIS.
follow this guide.
https://neelbhatt.com/2018/01/30/deploy-net-core-application-to-iis-step-by-step-guide/
I was successful

Try use this tutorial:
Enabling Swagger In Your .NET Core 2.0 Application - A Step By Step Guide

I just installed Swashbuckle.AspNetCore 3.0.0. It worked for the first time. I was able to access the documentation via http://localhost:58030/swagger Then I removed the ValuesController (default controller in the VS template) and updated the LaunchUrl in launchSettings.json. Since then I have this same issue. No other changes were made and I use Visual Studio 2017.

I have the same problem, unable to use swagger wtih VS2017. Chrome return ERR_CONNECTION_REFUSED
And yet I followed different tutorials and I created 5 test projects. I feel like I have tried everything.

I had the same problem, Delete vs folder and it is working again. I am sure this is the perfect solution.

The issue seems to be still actual. I have just updated VS to 15.9.3 and Swagger stopped working showing 404.
Configuration as follows:
`services.AddSwaggerGen(c =>
{
c.SwaggerDoc("v1", new Info { Title = "Travel Miser API", Version = "v1" });
});
app.UseSwagger();
        app.UseSwaggerUI(c =>
        {
            c.SwaggerEndpoint("/swagger/v1/swagger.json", "Travel Miser API V1");
        });

        app.UseMvc();

`
Steps proceeded:

.vs removed
libraries reinstalled
adding RoutePrefix did not help
but all above unsuccessful.

Does anyone have any idea?

I am having an issue with swashbuckle(4.0.1) on my .net core web api hosted on kubernetes...
after some time some endpoins on swagger UI disappears, even though they still work when called externally.
Any ideas?

I'm having the same problem with version 4.0.1 and  VS2019. Was anyone able to fix it or has any workaround?

For me, I happened after I moved my project to a different directory. Based upon one of the posts above I inspected by applicationhost.config file  and under the  section one of the references was not updated to the new location. Updated the value and viola swagger was back.

Mine was a simple delete of the .vs folder. Thanks for all the tips on this page.

even if the issue is in a closed state I would add to the previous answers that the main probleme resides in the configuration od IIS Express associated with visual studio 17.x and up,
You have just to create a new virtual directory with a new port number for the webApi Application after you have installed and configured Swagger on it.

In my case, the following code in the Startup.cs file "if (env.IsDevelopment())" results to false because the IWebHostEnvironment was not recognized as 'Development'.  In debug mode with a breakpoint set on the if statement, I can easily see the code inside the if statement being skipped.  If I manually force it to execute the code inside the if statement, all works fine.
So, I realize that I was missing the environment variable:  "environmentVariables": {"ASPNETCORE_ENVIRONMENT": "Development"} As PhilippeCamou mentioned in his previous link .
Once I added the environment variable, as mentioned below, all works fine.
"profiles": {
"IIS Express": {
"commandName": "IISExpress",
"launchBrowser": true,
"launchUrl": "swagger",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
},

In my case, the following code in the Startup.cs file "if (env.IsDevelopment())" results to false because the IWebHostEnvironment was not recognized as 'Development'.  In debug mode with a breakpoint set on the if statement, I can easily see the code inside the if statement being skipped.  If I manually force it to execute the code inside the if statement, all works fine.
So, I realize that I was missing the environment variable:  "environmentVariables": {"ASPNETCORE_ENVIRONMENT": "Development"} As PhilippeCamou mentioned in his previous link .
Once I added the environment variable, as mentioned below, all works fine.
"profiles": {
"IIS Express": {
"commandName": "IISExpress",
"launchBrowser": true,
"launchUrl": "swagger",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
},


Update KingfisherWebP.podspec

fix: change kingfisher dependence version

@VanQuvideo Thanks for this! I forgot that CocoaPods' version constraints are not the same as other languages. I'll merge this PR once the comment issue is addressed.


Update AutoMLSearch with actions parameter

Closes #1883

Updated _make_component_list_from_actions to return a list of (component_class, parameters) tuples because our ComponentGraph only takes classes, not instances, and we have no good way of keeping track of the parameters otherwise.

After some discussion with @dsherry, we opted to not combine multiple of the same components into one. Unless we see strong indication that this is the better user interface, this is good for now as it helps with model understandability (why was each component added?)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Improve the people invite screens

The following changes must be applied on the people invite screen presented when the user decides to start a chat, add a new room member or process a people search:

[x]  A default list is displayed when no search session is in progress - see comment below for detailed description

[x] During a search we consider:

all the local contacts with at least one contact method (email, phone number...) which can be used to invite them.

all the matrix users who share a room with the current user.

[x] The search result lists only the contacts who have the search pattern as prefix in their display name, their matrix identifiers and/or their contact methods (emails, phones).

[ ] The search result is displayed in 2 sections:

"Local Contacts" section (use the same verbiage as the OS, iOS call them Contacts, Android?): Matrix-enabled contacts and non-Matrix enabled contacts are all listed together in alphabetical order.
[Previously; "Contact book" section (local contacts): the Matrix-enabled contacts are listed first by last active, with "active now" first, and then alphabetically, then the non-Matrix-enabled contacts are listed in alphabetical order.]

"Known Contacts" section: Sort the Matrix contacts by last active, with "active now" first, and then alphabetically. Blank in the Default screen, only populates when a search is engaged

Add filter tabs at the top allowing to filter "Matrix contacts" | "All contacts" in the local address book

[x] @ara4n to provide a sketch for the filter tab

[x] Remove a contact from the "Known Contacts" section if a local contact matches with his Matrix identifier.

[x] Local Contacts section: local contact with several contact methods should be split in several contacts. Only one method (email, phone number) by cell.

if a local contact matches thanks to his display name, a cell is displayed for each contact methods even if they don't match the search pattern.

if a local contact matches thanks to his matrix ids or contact methods, a cell is displayed only for the methods which have the searched prefix.

[ ] Local Contacts section:

The cell for a Matrix-enabled contact should display the matrix id, and add the Riot icon on the avatar. Lookup should be done on both email and MSISDN from the initial version.
[Note to @ara4n: we get a Riot icon on the avatar as reachable from Riot, but then it's a Matrix enabled contact... Should we use a Matrix icon? To be discussed. Use Riot icon for now]
See the following picture (display name font: medium 17, matrix id font: regular 17, email font: regular 15):

The cell for a non-Matrix-enabled contact displays the associated contact method (use '@' for email, '#' for phone number):

[ ] A contact details screen should be designed to display the contact information - #906 To be confirmed. Postponed for now, as splitting the contact by contact method might be enough.
(Note: this contact details should be available from a room member screen)

Notes:

Multiselection is out of scope for now and will be done as a second step in order to get the new flow out of the door ASAP.
Clicking on a contact only invites it, it doesn't open the Contact Details page

Summary of the user flows in following picture:

#905 is open to track the new button in the Room Member list

Default List Screen

2 sections:

Local Contacts (use the same verbiage as the OS, iOS call them Contacts, Android?)
Known Contacts

Local Contacts

Display the full address book ordered alphabetically, mixing Matrix enabled and non-Matrix enabled users
Add filter tabs at the top (@ara4n to provide a sketch) allowing to filter "Matrix contacts" | "All contacts" in the local address book

Known Contacts

blank in the Default screen, only populates when a search is engaged

Checklist updated

Suggested filter tabs at the top:

sketch for the filter tab provided by @ara4n:

Note: When a contact is removed from the "Known Contacts" section because a local contact matches with his Matrix identifier. The local contact should display the matrix avatar of this user.

The provided tab ("Matrix contacts" | "All contacts") is not adapted to the global search screen. W

e suggest adding a specific tick in the Local contacts header, see the following screenshot:

Checklist updated
@jeangb could you please test the lookup on msisdn.

Validated


prompt to start generation with (default: empty)
  -e                    process prompt escapes sequences (\n, \r, \t, \', \", \\)
  --prompt-cache FNAME  file to cache prompt state for faster startup (default: none)
  --prompt-cache-all    if specified, saves user input and generations to cache as well.
                        not supported with --interactive or other interactive options
  --prompt-cache-ro     if specified, uses the prompt cache but does not update it.
  --random-prompt       start with a randomized prompt.
  --in-prefix STRING    string to prefix user inputs with (default: empty)
  --in-suffix STRING    string to suffix after user inputs with (default: empty)
  -f FNAME, --file FNAME
                        prompt file to start generation.
  -n N, --n-predict N   number of tokens to predict (default: -1, -1 = infinity)
  --top-k N             top-k sampling (default: 40, 0 = disabled)
  --top-p N             top-p sampling (default: 0.9, 1.0 = disabled)
  --tfs N               tail free sampling, parameter z (default: 1.0, 1.0 = disabled)
  --typical N           locally typical sampling, parameter p (default: 1.0, 1.0 = disabled)
  --repeat-last-n N     last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size)
  --repeat-penalty N    penalize repeat sequence of tokens (default: 1.1, 1.0 = disabled)
  --presence-penalty N  repeat alpha presence penalty (default: 0.0, 0.0 = disabled)
  --frequency-penalty N repeat alpha frequency penalty (default: 0.0, 0.0 = disabled)
  --mirostat N          use Mirostat sampling.
                        Top K, Nucleus, Tail Free and Locally Typical samplers are ignored if used.
                        (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
  --mirostat-lr N       Mirostat learning rate, parameter eta (default: 0.1)
  --mirostat-ent N      Mirostat target entropy, parameter tau (default: 5.0)
  -l TOKEN_ID(+/-)BIAS, --logit-bias TOKEN_ID(+/-)BIAS
                        modifies the likelihood of token appearing in the completion,
                        i.e. `--logit-bias 15043+1` to increase likelihood of token ' Hello',
                        or `--logit-bias 15043-1` to decrease likelihood of token ' Hello'
  -c N, --ctx-size N    size of the prompt context (default: 512)
  --ignore-eos          ignore end of stream token and continue generating (implies --logit-bias 2-inf)
  --no-penalize-nl      do not penalize newline token
  --memory-f32          use f32 instead of f16 for memory key+value (default: disabled)
                        not recommended: doubles context memory required and no measurable increase in quality
  --temp N              temperature (default: 0.8)
  -b N, --batch-size N  batch size for prompt processing (default: 512)
  --perplexity          compute perplexity over the prompt
  --keep                number of tokens to keep from the initial prompt (default: 0, -1 = all)
  --mlock               force system to keep model in RAM rather than swapping or compressing
  --no-mmap             do not memory-map model (slower load but may reduce pageouts if not using mlock)
  -ngl
N, --n-gpu-layers N


name, license, datePublished and description are SHOULD and not MUST properties

According to https://www.researchobject.org/ro-crate/specification/1.1/root-data-entity.html#direct-properties-of-the-root-data-entity, name, license, datePublished and description are SHOULD and not MUST properties.
Currently the rocrate-validator reports these as MUST properties and validation fails, e.g. for an eln-file from RSPace with:
  The following requirements have not meet:

                                                                                [profile: RO-Crate Metadata Specification 1.1]
     [ ro-crate-1.1.8 ]: RO-Crate Root Data Entity REQUIRED properties

      The Root Data Entity MUST have a name, description, license and datePublished

          Failed checks

       [    MUST 8.3    ]  Root Data Entity: `licence` property:
                           Check if the Root Data Entity includes a license property (as specified by schema.org) to provide information about the
                           license of the dataset.
         Detected issues
         - [Violation on <./>]: The Root Data Entity MUST have a `license` property (as specified by schema.org).
                 SHOULD link to a Contextual Entity in the RO-Crate Metadata File with a name and description.
                 MAY have a URI (eg for Creative Commons or Open Source licenses).
                 MAY, if necessary be a textual description of how the RO-Crate may be used.

See also the discussion in this issue: https://github.com/TheELNConsortium/TheELNFileFormat/issues/88

Hi @salexan2001 !
We have already discussed this issue in #14, where @elichad pointed out that, according to the specification, the properties you mentioned are REQUIRED and not merely RECOMMENDED. What is recommended (SHOULD) is solely the value of these properties, while their presence is mandatory.
You can also find an example of a minimal RO-Crate in the specifications, which illustrates that these properties must be present.

Thanks for the info!
This issue is currently discussed here: https://github.com/TheELNConsortium/TheELNFileFormat/issues/88


Thanks for configuration information

I am using Asus vivobook 15  R542UQ-DM153 with bios version X542UQ .305. Currently  enabled  secure boot without CSM, If  i enable  CSM means, Fast boot will be automatically  disabled in bios.
Thanks to the authors, Great work

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3104beb2-e766-f5e7-49f7-a6488592fb2e
Version Independent ID: a016fbce-2bc7-c1bb-3035-053981676cac
Content: Switch from Legacy MBR disk to GPT disk with Windows 10 - Windows drivers
Content Source: windows-driver-docs-pr/bringup/switch-from-legacy-mbr-disk-to-gpt-disk-with-windows-10.md
Product: windows-hardware
GitHub Login: @barrygolden
Microsoft Alias: bagold

@RAJU2529, thanks for letting us know that you found the configuration and upgrade path information to be correct and helpful.


Profile screen All task details screen

Issue Description
Profile Section Update - clicking on All tasks section , which will open the new screen name named all task detail screen. Where we can give progress updates as well.
Expected Behavior
it should show the screen of status m -site screens
Current Behavior
no screen yet
Screenshots

Environment

Operating System: [e.g., Windows, macOS, Linux]
Browser (if applicable): [e.g., Chrome, Firefox, Safari]
Version of Software/Project: [e.g., v1.0.0]

Additional Information
[Include any additional context, logs, or details that might be helpful in addressing the issue.]
Checklist

[ ] I have read and followed the project's code of conduct.
[ ] I have searched for similar issues before creating this one.
[ ] I have provided all the necessary information to understand and reproduce the issue.
[ ] I am willing to contribute to the resolution of this issue.

Thank you for contributing to our project! We appreciate your help in improving it.
📚 See contributing instructions.

@shreya-mishra please assign me this task.

Task Breakdown:-
3 days for Ui
2 days for api integration
3 days for testing
3 days for approval and buffer.

@shreya-mishra ETA: 25/11/23


Xclim example not running

Hi, the notebook at https://planetarycomputer.microsoft.com/dataset/cil-gdpcir-cc-by#Climate-indicators is not working.
I am aware that Xclim has a new release and it apparently is not adjusted in your sever yet.

Thanks for the report. Follow along at https://github.com/microsoft/PlanetaryComputer/issues/212 for updates.
In the meantime, you might be able to pip install -U xclim after you're notebook server starts.


MAC address with uppercase letters can't be mapped

Hello, folks! Hope you are doing well :-)
I believe I found a bug today, hopefully with an easy fix.
In client.go 125-131:
	for _, nic := range nics {
		card := nic.(types.BaseVirtualEthernetCard).GetVirtualEthernetCard()

		mapping, ok := networkMappings.Mappings[card.MacAddress]
		if !ok {
			return nil, errors.New("no network mapping found for MAC address")
		}

card.MacAddress seems to be converted to lowercase characters, even if the user-provided mac field in network-mapping comes with uppercase letters, making networkMappings.Mappings[card.MacAddress] return nothing when the interface's MAC address in VMware has uppercase letters, causing a mapping mismatch and returning Error: no network mapping found for MAC address.
Example:

In Vcenter, the VM has the following MAC address: 00:50:56:A4:E2:53, as seen in the web GUI.
Trying to run a cutover (did not attempt running a migrate before, but I'd say the same thing would happen) with --network-mapping mac=00:50:56:A4:E2:53,network-id=<openstack-net-id>,subnet-id=<openstack-subnet-id> fails with Error: no network mapping found for MAC address. A mapping with lowercase letters also fails.
Logging some values here, I get:
card.MacAddress = 00:50:56:A4:E2:53
networkMappings.Mappings = map[00:50:56:a4:e2:53:{00:50:56:a4:e2:53   }]
mapping = { 00000000-0000-0000-0000-000000000000 00000000-0000-0000-0000-000000000000 }

The empty value of mapping makes me believe the uppercase letters are the problem here.
A workaround I tested was changing the interface's MAC address in VMware to have only lowercase letters.
Hope this makes sense.
Thank you!

What if you tried to simply using lower caase in the mapping?

I did, got the same error.


Update cosmos version to 0.4.1

High Level Description
Add metrics and documentation.
Related Issues

DCOS-17822 Release cosmos
DCOS-17823 Release cosmos enterprise

Checklist for all PR's

[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[x] Change log from the last version integrated (this should be a link to commits for easy verification and review): Change Log
[x] Test Results: link to CI job test results for component
[x] Code Coverage (if available): link to code coverage report

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and Review Process
What is the review process and when will my changes land?
All PRs require 2 approvals using github's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

@jsancio @cruhland

@mesosphere-mergebot label Ready For Review

@mesosphere-mergebot bump-ee

@takirala @jsancio @cruhland ping, please review.  Needed for final train today.


Exit code 20 on a seemingly successful analysis

When running kind on the command line successful checks seem to return an exit code of 20 despite nothing going wrong. Given the following file as input:
node Id(x: bool) returns (y: bool) ;
(*@contract import
  IdCon(x) returns (y) ;
*)
let
  --%MAIN;
  y = x;
tel

contract IdCon(x: bool) returns (y: bool);
let
  guarantee x = y ;
tel

Running kind2 produces the following output
$ kind2 trivial.lus
 kind2 v1.6.0-86-g4417dc5

=========================================================================================================================================
Analyzing Id
  with First top: 'Id' (no subsystems)

<Success> Property IdCon[l2c13].guarantee[l12c3] is valid by one state invariant generator (bool) after 0.042s.

-----------------------------------------------------------------------------------------------------------------------------------------
Summary of properties:
-----------------------------------------------------------------------------------------------------------------------------------------
IdCon[l2c13].guarantee[l12c3]: valid (at 1)
=========================================================================================================================================

$ echo "$?"
20

Is there something silently failing which I am missing which is triggering a non-zero exit code here?
System details:

kind2: v1.6.0-86-g4417dc5
Z3: version 4.11.2 - 64 bit
OS: Linux 6.0.2-arch1-1 x86_64

No, nothing is silently failing.
For historical reasons, Kind 2 does not follow the UNIX convention. It uses the following convention for exit codes:
0 = Unknown
2 = Error
10 = Unsafe
20 = Safe
Clearly we should document this atypical behavior.

We decided to change the behavior of Kind 2, and make Kind 2 follow the POSIX convention of returning exit code 0 for successful checks. PR #971 implements this change.


AllowMove and AllowResize requires AspectRatio

If you try to set the AllowMove or AllowResize parameters without having set the aspect ratio then you get a rather difficult to interpret JS exception when you run the code.
Either this should be made to work or when you set the AllowMove and AllowResize parameters it should throw an exception if the AspectRatio parameters has not been set. This would then give a more meaningful exception that implementers can use to correct their code. Alas I don't think optional [EditorRequired]  is a thing yet.
I am happy to put up a PR with this change but I thought maybe some discussion over what is the best way to handle this might be required.

Hi @owaits,
Sorry for the long delay. I agree that an exception is never a great dev experience for something that should in an ideal world be enforced at compile time. AFAIK, conditional required params aren't a thing, so that's out. The behavior is documented on AllowMove/AllowResize. Where would you throw the exception?
Thanks,
Stefan


Cache between pipeline builds

Now when a pipeline executes and exits, intermediate contents and the leftover pod will be cleaned up. The caching utility would be helpful for accelerating pipeline executions and reducing network traffic.

https://github.com/rancher/rancher/issues/16223

#19768 could resolve this


In what order are custom resources (in templates directory) deployed by Helm

As we know that the install order of Kubernetes types is given by the enumeration InstallOrder in kind_sorter.go Helm v2 source file
Where do the custom resources defined by an operator fit in here and how the dependency among these resources is taken care of?

I am not entirely sure what you are asking about. If I get something wrong then more details might help.
CRDs in Helm v2 are installed using the crd-install hook. You can read more about hooks in the v2 docs. These are installed first. That is, before the manifests in the file you linked.
Does that help?

@mattfarina  Consider this yaml - https://github.com/banzaicloud/logging-operator/blob/master/charts/logging-demo/templates/output_elasticsearch.yaml
This deploys a resource of kind "Output" which is a custom resource. When does this get deployed?  If you see other files here - https://github.com/banzaicloud/logging-operator/blob/master/charts/logging-demo/templates, in what order would each of these resources be created?

Ah, I understand the question now. CRDs are installed prior to other resource. But, in what order are custom resources (CRs) installed compared to each other and everything else. This is a good question.
Kinds that are not in the InstallOrder come after those listed in the InstallOrder. You can see this in the tests for the kind sorter and in the comments in the sorter itself.

I believe the question was answered, so am marking this as closed.


Add Reddit search

Is your feature request related to a problem? Please describe if so, otherwise leave this blank..
Describe the feature
<-- A clear and concise description of the feature. What will it do, what will it affect, etc. -->

Adds search query for Reddit

Provide any alternatives to the feature you've considered
Any other alternatives to the feature request you are asking for if we decide not to go with your feature request?
Additional information
<-- If you have anything additional to say, you can say it here. Otherwise you can leave this blank. -->

can i work on this


Add support for using ZIP files as the repo

Proposed change
Add support for downloading ZIP files instead of a repo:
repo2docker https://example.com/an-archive.zip
This comes straight from the repo2docker roadmap, admittedly in the "later" section, but IMO this could be a good first issue for new contributors.
AFAICS this should be pretty straightforward to implement, if we can agree on the following criteria for the content provider:

the URL contains .zip (or even ends with that)
a HEAD request returns content that could be a ZIP file (might not work on some/many servers), i.e. Content-Type: application/zip, Content-Encoding: gzip

Here's a little experiment for the headers:
>>> import requests
>>> requests.head('https://github.com/jupyter/repo2docker/archive/0.10.0.zip', allow_redirects=True).headers
{'Access-Control-Allow-Origin': 'https://render.githubusercontent.com', 'Content-Security-Policy': "default-src 'none'; style-src 'unsafe-inline'; sandbox", 'Strict-Transport-Security': 'max-age=31536000', 'Vary': 'Authorization,Accept-Encoding', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'deny', 'X-XSS-Protection': '1; mode=block', 'ETag': 'W/"0cd777a9ac8e56848f6f6b0df7b3acebc7c940035abaeea62516c93ec61d49b0"', 'Content-Type': 'application/zip', 'Content-Disposition': 'attachment; filename=repo2docker-0.10.0.zip', 'X-Geo-Block-List': '', 'Date': 'Thu, 14 Nov 2019 10:07:25 GMT', 'X-GitHub-Request-Id': '8B46:3ED5:7A93:101EE:5DCD275D'}
>>>
>>> requests.head('https://nordholmen.net/test.zip', allow_redirects=True).headers
{'Content-Type': 'application/zip', 'Content-Length': '201', 'Connection': 'keep-alive', 'Keep-Alive': 'timeout=15', 'Date': 'Thu, 14 Nov 2019 10:14:47 GMT', 'Server': 'Apache', 'Last-Modified': 'Thu, 14 Nov 2019 10:14:30 GMT', 'ETag': '"c9-5974bbf467936"', 'Accept-Ranges': 'bytes'}
>>>
>>> requests.head('https://www.dropbox.com/s/m6c2lmq6v1nn8pp/test.zip?dl=0', allow_redirects=True).headers
{'Server': 'nginx', 'Date': 'Thu, 14 Nov 2019 10:15:20 GMT', 'Content-Type': 'text/html; charset=utf-8', 'Connection': 'keep-alive', 'Vary': 'Accept-Encoding', 'Cache-Control': 'no-cache', 'Content-Security-Policy': "base-uri 'self' ; child-src https://www.dropbox.com/static/serviceworker/ blob: ; connect-src https://* ws://<IP_ADDRESS>:*/ws ; [...] x.com/static/serviceworker/ blob:", 'Content-Security-Policy-Report-Only': "frame-ancestors 'self' 'self' ; report-uri https://www.dropbox.com/csp_log?policy_name=metaserver-experimental ; script-src 'unsafe-eval' 'unsafe-inline' https://*", 'Dropbox-Streaming': 'V=1', 'Pragma': 'no-cache', 'Referrer-Policy': 'origin-when-cross-origin',  [...], 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains', 'Content-Encoding': 'gzip'}

Related issue: #795
Alternative options
Don't support it.
Who would use this feature?
People who want to test run (maybe privately) a repository, and users of not supported websites and repositories.
How much effort will adding it take?
Half a day.
Who can do this work?
Developer with basic Python skills, if helped with the repo2docker code base.

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:
https://discourse.jupyter.org/t/repo2dockerspawner-alternative-version/3584/7


Hard to identify the reason of an error

Is your feature request related to a problem? Please describe.
Working on a project, I spent a good couple of hours strugling with a mistake I made.
In a scenario where a property of an entity is an array of another entity, like the image below:

And that entity is the response of my usecase:

If I push an object that isn't an entity (does not implements BaseEntity), it throws an error at runtime:

Describe the solution you'd like
It would be nice to have a better error message, telling the developer he is pushing the wrong object to that array.
The example I created is very simple, so it's easier to identify the problem. But when I faced this problem, I was on a larger code base, so it took me a lot of time to understand what was going on.
Describe alternatives you've considered
Herbs is written in JS, so it's hard to handle this kind of errors because it only happens at runtime.
Thinking about the problem, may it would be possible to identify this by using Proxies on entity properties, so we can intercept the changes in the object and print a more specific message on the console.
Additional context
I created a minimal herbs project showing how to emulate the same problem.

Why not just fix the error?  🤷🏻♂

@eduardo-vortx this issue stil alive?

@italojs this issue is not about fixing the error. It is about showing us that it is hard to find the error.

@eduardo-vortx did you tried to use herbs audit inside the env?


cpu-x: 4.4.0 -> 4.5.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/X0rg/CPU-X/releases.
meta.description for cpu-x is: "Free software that gathers information on CPU, motherboard and more"
meta.homepage for cpu-x is: "https://github.com/X0rg/CPU-X"
meta.changelog for cpu-x is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 4.5.0 with grep in /nix/store/wb4s0qvnbgxrzx3pzkcr4lz0igpr2mcy-cpu-x-4.5.0
found 4.5.0 in filename of file in /nix/store/wb4s0qvnbgxrzx3pzkcr4lz0igpr2mcy-cpu-x-4.5.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
cpu-x

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/wb4s0qvnbgxrzx3pzkcr4lz0igpr2mcy-cpu-x-4.5.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A cpu-x https://github.com/r-ryantm/nixpkgs/archive/adf181269bc53be31ca4a19fdc479c2f50b9a3b0.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/wb4s0qvnbgxrzx3pzkcr4lz0igpr2mcy-cpu-x-4.5.0
ls -la /nix/store/wb4s0qvnbgxrzx3pzkcr4lz0igpr2mcy-cpu-x-4.5.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

cpu-x

Maintainer pings
cc @viraptor for testing.

Requires a libcpuid update first


TypeError: upfirdn2d() takes from 2 to 8 positional arguments but 10 were given

Dear Author,
I encountered this in running
python3 train.py --batch 3 --ckpt pretrained/tdmm_lizhen_full.pt --mode 3 path_to_dataset
I have installed stylegan_ops with logs looking fine.
I shall be most grateful if you may look into this. Thanks!
Sincerely,
Picard
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
logs in detail:
Traceback (most recent call last):
File "/mnt/swh/git/StyleAvatar/styleunet/train.py", line 340, in
train(args, loader, generator, discriminator, g_ema, g_optim, d_optim, device)
File "/mnt/swh/git/StyleAvatar/styleunet/train.py", line 147, in train
fake_img = generator(cond_img, latent)
File "/home/rtcai3/anaconda3/envs/geneface/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/swh/git/StyleAvatar/styleunet/networks/generator.py", line 91, in forward
cond_img = self.dwt(condition_img)
File "/home/rtcai3/anaconda3/envs/geneface/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/swh/git/StyleAvatar/styleunet/networks/modules.py", line 397, in forward
ll = upfirdn2d(input, self.ll, down=2)
File "/mnt/swh/git/StyleAvatar/styleunet/networks/stylegan2_ops/upfirdn2d.py", line 165, in upfirdn2d
out = UpFirDn2d.apply(input, kernel, up, down, pad)
File "/mnt/swh/git/StyleAvatar/styleunet/networks/stylegan2_ops/upfirdn2d.py", line 121, in forward
out = upfirdn2d_op.upfirdn2d(
TypeError: upfirdn2d() takes from 2 to 8 positional arguments but 10 were given

Please check the shape of cond_img, it should be (batch, 3, 256, 256)

Any more information or someone else can reproduce this problem? I can not reproduce this bug, Orz. It seems you are using the geneface's env. Maybe there are some conflicts?

Thanks very much for your informations.
I just used your
from torch.utils.cpp_extension import load
module_path = os.path.dirname(file)
upfirdn2d_op = load(
"upfirdn2d",
sources=[
os.path.join(module_path, "upfirdn2d.cpp"),
os.path.join(module_path, "upfirdn2d_kernel.cu"),
],
)
instead of
import upfirdn2d as upfirdn2d_op
and it worked.
I use another env since my server does not support CUDA 11.7. But I guess env may be not the point. Could you please show me your ninja version? Mine is ninja 1.11.1. If compiled upfirdn2d does not have enough advantage on saving training time. I choose source code instead.
Sincerely,
Picard


Smart plug v2 with latest esphome config spams logbook with Uptime reports

Every time the uptime sensor updates (60 seconds), a line is logged to the logbook. If you have a lot of smartplugs, its a lot of entries!

I am beginning to think that Athom Tech just do some products that look cool on paper but when it comes to software updates they are 0 ...

I noticed the same thing, from this commit merged by @tarontop:
https://github.com/athom-tech/athom-configs/commit/192ea652d37afdbeea634751795009b3af2d6183

I am beginning to think that Athom Tech just do some products that look cool on paper but when it comes to software updates they are 0

@hellcry37 In this case it looks like it was a community contribution (good!), doesn't seem like the author @Roving-Ronin has any formal association with Athom.

To confirm I do NOT have any formal links to Athom. Hell, be good if I was on their books, having just bricked another of my own plugs, testing updates for others to benefit from. As with most things ESPHome or Home Assistant related, the enhancement and/or bugfixing is something reliant upon the users to assist and contribute.
The preferable option as Lloyd kindly pointed out, is to assist / contribute... I'm no programmer myself but read the documentation and learn how to do it and lend up hand as and where you can, its more rewarding also than tossing 'grading remarks'.
As per above 'mention' Pull 53 created to provide you the easy option to change the update frequency in the substitutions section.
See this line:  occassional_update_sensor_interval: 3min

@Adrian57
To cut down on these entries being logged, try adding to configuration yaml:
recorder: !include recorder.yaml

then making a recorder.yaml file in the /homeassistant directory, with:
#purge_keep_days: 10
exclude:
  domains:
    - automation
    - update
    - button
    - light
    - script
  entity_globs:
    - sensor.weather_*
    - sensor.daylight_*
    - sensor.days_*
    - sensor.*_device_uptime
    - sensor.*_project_name
    - sensor.*_project_version
    - sensor.*_wifi_ip_address
    - sensor.*_wifi_ip_address
    - sensor.*_wifi_ssid
    - sensor.*_wifi_strength
    - sensor.*_wifi_signal
    - sensor.*_mac_address
  entities:
    - sensor.date_time
    - sensor.internet_time
    - sensor.time

then reload your configuration (or restart/reboot etc).
Note '#purge_keep_days: 10' is hashed out, as 10 days is the default timeframe for the recorder to keep data.


Stub DNS resolution of myhost.com in specs

The specs are currently failing as the placeholder domain myhost.com no longer resolves.

:+1:


After login, libraries view fails to load with "Request failed with status code 406".

Describe the bug
After logging with auth code, the sas content view populates correctly with folder and files, but the library view is empty and the error Request failed with status code 406 appears.
Steps to reproduce
Steps to reproduce the behavior:

Login
See the error

Expected behavior
To see the libraries and datasets.
Screenshots
N/A
Environment (please complete the following information):
Client OS: macos
Extension version: 0.1.15
SAS version: Viya 3.5
Additional context
N/A

Hi, I didn't reproduce it. A couple of questions for us to better understand the problem.
Are you able to run SAS code successfully?
Are you still getting the error if pressing the "refresh" icon on the library view?
Would you click Help > Toggle Developer Tools from the top menu and see if any detailed error in it?
What's your VS Code version?
Are you using a Language Pack in your VS Code? If so, what language is it?
Thanks.


TypeError: Expected int for argument 'num' not <tf.Tensor 'Squeeze:0' shape=() dtype=int32>.

I run the following code, but it has one error at    "tf.unpack(inputs,self.bathsize)",the error is as follows:
TypeError: Expected int for argument 'num' not <tf.Tensor 'Squeeze:0' shape=() dtype=int32>.
 def build(n_dict, word_dim):
       self.word_ids = tf.placeholder(tf.int32, shape=(None, None, 7), name='word_ids')
        self.bathsize = tf.shape(self.word_ids)[0]
        self.word_len=tf.shape(self.word_ids)[1]
        with tf.device("/cpu:0"):
            self.embedding = tf.Variable(tf.random_uniform([n_dict, word_dim], -1.0, 1.0), name='embedding', dtype='float32')
            self.embedded_input = tf.nn.embedding_lookup(self.embedding, self.word_ids)
        inputs = tf.reshape(self.embedded_input, [self.bathsize, self.word_len, -1])
        # inputs=tf.transpose(inputs,[1,0,2])
        inputs = tf.nn.dropout(inputs, self.dropout)
        self.inputs = inputs

        self.input_length = tf.fill([self.word_len], self.bathsize)
        self.input_length = tf.cast(self.input_length, dtype=tf.int64)
        with tf.variable_scope('forward'):
            self.lstm_fw_cell = rnn_cell.BasicLSTMCell(n_hidden)
        with tf.variable_scope('backward'):
            self.lstm_bw_cell = rnn_cell.BasicLSTMCell(n_hidden)
        lstm_state_fw = self.lstm_fw_cell.zero_state(self.word_len, tf.float32)
        lstm_state_bw = self.lstm_bw_cell.zero_state(self.word_len, tf.float32)
        # sequence_length=self.input_length

        outputs, output_state_fw, output_state_bw = rnn.bidirectional_rnn(
            self.lstm_fw_cell, self.lstm_bw_cell,
            tf.unpack(inputs,self.bathsize),
            initial_state_fw=lstm_state_fw,
            initial_state_bw=lstm_state_bw,
            sequence_length=self.input_length
        )

As I know, tf.unpack have one parameter num, when the code run to "tf.unpack(inputs,self.bathsize)", num=None.  When the bachsize if fixed, it succeed to run. But  I want the model inputs have variable bacthsize, do you have any idea to solve this?@Geoffrey Irving

Here is the documentation for tf.unpack:
https://www.tensorflow.org/versions/r0.11/api_docs/python/array_ops.html#unpack
Note this line in particular:
If num is not specified (the default), it is inferred from value's shape
Try changing your code above to just the following:
  tf.unpack(inputs)

Automatically closing due to lack of recent response. Please reopen when additional information becomes available. Thanks!

@tatatodd removing the parameter num will cause a ValueError: Cannot infer num from shape and cannot solve anything.
I'm now having the same problem when trying to use tf.unpack. Could anyone give an effective solution to  this?


Problem: hidden files interfere with bag validation

Is your feature request related to a problem? Please describe.
When submitting a transfer in the bag format, the bag validation check in Archivematica/a3m will fail if there are hidden files in the bag because the hidden files are not included in the bag manifest. This is a particular issue for Mac users, since Macs often add dotfiles (e.g. .DS_Store). This can be remedied by the user by manually removing hidden files from the bags before they are transferred; however, this is both cumbersome and limiting, since the dotfiles can be created every time the user interacts with a file.
A bag validation failure in Archivematica stops the transfer process altogether, so the user has to identify the bag that errored out, remove the hidden files, and restart the ingest.
Describe the solution you'd like
I'd like to prevent any hidden files from being transferred with the bag. The solution should check for and remove hidden files before the bag transfer is ingested into Archivematica/a3m.
In Legacy Enduro, this is done at the point when the bag is copied from the transfer source location to the processing location. Any file beginning with a . is not copied.
This feature should be configurable, so that users can keep hidden files if they choose. It would also be preferrable to allow users to edit the list of files that should be removed/ignored, as is done in Archivematica/a3m's Remove hidden files and directories and Remove unneeded files jobs.
Describe alternatives you've considered
The manual method mentioned above does work but it is susceptible to human error, and might need to be repeated should the user have a need to look at the files in the bag.
Additional context
Note that this is only a requirement for bagged transfers. For standard and other non-bagged transfer types, Archivematica/a3m remove hidden files as a matter of course during the early stages of processing.
The client for whom this has been an issue uses unzipped bags, which both lends itself to the problem manifesting AND provides the easy solution of simply not copying dotfiles, as is done in Legacy Enduro. I'm not sure how the issue would be dealt with in a zipped bag, where the whole bag is copied as a single entity. Perhaps focusing on the unzipped bag example is the easiest starting point.

@sallain should there be a premis event recorded for the file removal?

Note that due to issue #845 Enduro can not currently process unzipped Bags that are uploaded via MinIO.  Enduro should be able to process unzipped Bags using the filesystem watcher, but I've never tested this option to confirm it works.

@sallain should there be a premis event recorded for the file removal?

I think this raises an interesting point.  If a hidden file (e.g. .DS_Store) is present when the Bag is created, then I think BagIt will add the file to the Bag manifest and checksum files, and in this case we should not remove the hidden file because it will cause validation to fail.  If a hidden file is added after the Bag is created, then it will have no record in the Bag manifest or checksum files, so it must be remove for the Bag to validate.
In the second case, I don't think there is any need to add a PREMIS event about the removal of the hidden file - the file clearly was not meant to be part of the transfer payload.

@djjuhasz @jhsimpson The use case as described certainly falls into the latter category, which you could expand on to say that the hidden files are both unexpected and unwanted. However, in my opinion it's still a material change to the bag as deposited, regardless of whether or not
In order to add a PREMIS

I've done a bunch of work on this issue on branch dev/issue-850-remove-hidden-files, but it's turned into a giant PR.  I'm going to start again from main, and break up the changes into a number of smaller issues and PRs:

[ ] Fix hardcoded a3m shared dir (issue #864)
[ ] Create a common "Copy()" interface and for minio and filesystem watchers
[ ] Remove Bundle activity dependency on package service
[ ] Move unarchive functionality from Bundle activity to its own activity
[ ] Make the temporary transfer directory path created by the bundle activity testable
[ ] Add a "remove hidden files"  activity (after unarchive, before bundle)

@Diogenesoftoronto developed https://github.com/artefactual-sdps/remove-files-activity to remove hidden files from a transfer.  Because we are planning to use that script in preprocessing as a child workflow there's no need to re-implement it here.

Does this issue need to be closed? Or are there things that still need to be done? @sallain

I've created https://github.com/artefactual-sdps/temporal-activities/issues/2 for the implementation of the remove hidden files activity.


Add dry-run subcommand to xtask to run cargo publish --dry-run with required arguments

Seems to work as far as I can tell 😅

Considered adding a publish command instead which optionally allows for dry run, but I'm afraid I'll forget the argument. Can invert the logic I guess, but something like cargo publish --publish seems weird 😅  Open to suggestions here

Okay I ended up changing it to publish, which performs a dry-run by default but can optionally be supplied --no-dry-run. This command is a bit spicy but will be handy I guess 😅


Missing commas

Made bazel build //tensorflow/tools/pip_package:build_pip_package fail

Thanks! There are a couple of other issues in the compilation which require attention. Please report other issues you are experiencing in another issue!


bug on  Migrator.connect() default configuration invalid import "import(migrationFilePath)"

Describe the bug
When trying to run Await Migrator.connect()
with default values I get this error response
[!] (plugin commonjs--resolver) Error: invalid import "import(migrationFilePath)". It cannot be statically analyzed. Variable dynamic imports must start with ./ and be limited to a specific directory. For example: import(`./foo/${bar}.js`).

To Reproduce
Steps to reproduce the behavior:
set up Migrator with defualt configuration
      const migrator = await Migrator.connect({
        uri: `YOUR_MONGO_URI`,
      });
      ```

**Expected behavior**
Default values should create a connection for the migrator.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**

- OS: MAC OS 14.4.1

Add any other context about the problem here.

Hey @JasonThomasII can you fist check these samples
https://github.com/ilovepixelart/ts-express-swc using swc
https://github.com/ilovepixelart/ts-express-esbuild using esbuild
I think I need more context about your setup
If you using any other framework / bild system / bundler  you can create a small sample repo with your config so I can clone it and run locally to reproduce.

Hey @ilovepixelart,  sorry about letting this go stale. I got busy with other things wasn't wasn't able to reproduce the issue.


weaver: moving InvokeRpcPrefix to WeaverConstants

Moving stuff out of Weaver class

closing for now because seems more complicated than before


Fix self assign

The borrow order in assign matters when lhs and rhs are the same.

Codecov Report

Merging #21 (c9df6c5) into main (dbf631f) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #21   +/-   ##
=======================================
  Coverage   88.48%   88.49%
=======================================
  Files          15       15
  Lines        1859     1860    +1
=======================================
+ Hits         1645     1646    +1
  Misses        214      214

Impacted Files
Coverage Δ

optpy-std/src/lib.rs
24.43% <100.00%> (+0.43%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Bug: unable to clone public deck

On the live site, it's impossible to clone this deck because the Deck ID is claimed to be invalid:
https://ashes.live/decks/22307/

This was fixed on the front-end; the backend was actually working perfectly, but the front-end was sending the wrong deck ID in some instances.


add clusterName to otel initialization

Great.
One small change - as I did with the other components, let's remove the otel middleware/logging from health check methods such as /ready etc. to reduce noise...


Update updating documentation for the new upcoming version

I see that this is a bit outdated. https://github.com/hydephp/develop/blob/master/docs/digging-deeper/updating-hyde.md
For example the update:assets command is removed

Fixed in https://github.com/hydephp/develop/pull/1242


Switch Typescript types to DefinitelyTyped.

Switch Typescript types to DefinitelyTyped (npm @types/ramda)
fixes #2918

:herb:
LGTM.
Anyone have comments?  @ramda/core?

:bowtie:

Thank you very much, @estaub !


Why does the task Webpack DLL complain when trying to import another module.

Following #326, I did like advised and restarted a whole project using the configuration of the react boilerplate and the links provided.
Building the Webpack DLL...
/workspace/git.domain.com/domain/backoffice/node_modules/babel-core/lib/transformation/file/options/option-manager.js:334
        throw e;
        ^

Error: Options {"es2015":{"modules":false}} passed to /workspace/git.domain.com/domain/backoffice/node_modules/babel-preset-latest/lib/index.js which does not accept options. (While processing preset: "/workspace/git.domain.com/domain/backoffice/node_modules/babel-preset-latest/lib/index.js") (While processing preset: "/workspace/git.domain.com/domain/backoffice/node_modules/babel-preset-latest/lib/index.js")
    at /workspace/git.domain.com/domain/backoffice/node_modules/babel-core/lib/transformation/file/options/option-manager.js:320:17
    at Array.map (native)
    at OptionManager.resolvePresets (/workspace/git.domain.com/domain/backoffice/node_modules/babel-core/lib/transformation/file/options/option-manager.js:270:20)
    at OptionManager.mergePresets (/workspace/git.domain.com/domain/backoffice/node_modules/babel-core/lib/transformation/file/options/option-manager.js:259:10)
    at OptionManager.mergeOptions (/workspace/git.domain.com/domain/backoffice/node_modules/babel-core/lib/transformation/file/options/option-manager.js:244:14)
    at OptionManager.init (/workspace/git.domain.com/domain/backoffice/node_modules/babel-core/lib/transformation/file/options/option-manager.js:374:12)
    at compile (/workspace/git.domain.com/domain/backoffice/node_modules/babel-register/lib/node.js:103:45)
    at loader (/workspace/git.domain.com/domain/backoffice/node_modules/babel-register/lib/node.js:144:14)
    at Object.require.extensions.(anonymous function) [as .js] (/workspace/git.domain.com/domain/backoffice/node_modules/babel-register/lib/node.js:154:7)
    at Module.load (module.js:488:32)
-<EMAIL_ADDRESS>node_modules/bootstrap-styled/node_modules/react-boilerplate-dlls
<EMAIL_ADDRESS>/workspace/git.domain.com/domain/backoffice
├──<EMAIL_ADDRESS><EMAIL_ADDRESS>├── UNMET PEER DEPENDENCY<EMAIL_ADDRESS>└── UNMET PEER DEPENDENCY<EMAIL_ADDRESS>
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| ^2.0.0 but none was installed.
dka@dev-01:[/workspace/git.domain.com/domain/backoffice]:

This error happen when I try to do a npm install git+https://git@github.com:kopaxgroup/bootstrap-styled.git --save
I have defined my module as follow :
  output: {
    // filename: '[name].[chunkhash].js',
    filename: 'bootstrap-styled.js',
    library: "bootstrap-styled", // string,
    libraryTarget: "commonjs-module", // enum
  },

What is the purpose of this dll and why does he refuse my module ?

Because you're building a library and not an application you most likely don't need the full blown react-boilerblate webpack config. Try to look at some webpack configs of libraries. For example redux or react-redux. You could also look at these projects: https://github.com/styled-components/styled-components#built-with-styled-components. In addition I would advise you to read the webpack@2 docs or at least the concepts so you can understand what webpack does. In your current situation I think it would be enough if you transpile the files with babel (babel input-dir -o output-dir) unless you want a separate UMD bundle.


Media/YouTubeSubscribedChannels: (extended description)

| Level 2 Jeff | Computers |

#3864 closed #3864


anyone implemented a demo to  predict video?

Did you get anything running for this one?

See here: https://github.com/harvitronix/five-video-classification-methods/blob/master/demo.py


[Bug] structuredClone is not defined

Summary

I am trying to edit an asset using nautilus but it fails with the error
structuredClone is not defined

Current Behavior

The AssetBuilder does not work fine I guess. The problem occurs at this step.
My code:
import { providers, Wallet } from 'ethers'
import { LogLevel, Nautilus, AssetBuilder } from '@deltadao/nautilus'
import { Request, Response} from 'express'
const provider = new providers.JsonRpcProvider('https://genx.sagresearch.de')
const wallet = new Wallet(process.env.PRIVATE_KEY, provider)
Nautilus.setLogLevel(LogLevel.Verbose)

const customConfig = {
  metadataCacheUri: 'https://aquarius510.v4.delta-dao.com'
}
export async function updateAsset(req: Request, res: Response) {
  const nautilus = await Nautilus.create(wallet, customConfig)

  const result=await nautilus.getAquariusAsset('did:op:ac7543c622f1550fe81fdf8b25b91149c9759e4e4ee5edad78a20631baec6afa')
  console.log(result)
  const assetBuilder = new AssetBuilder(result);
  console.log("assetBuilder", assetBuilder)
 // update asset and return statuscode
}

Which node.js is being used? Please try version 18 or higher.


feat(model): add feature-gated rkyv implementations

This implements rkyv's Archive, Deserialize, and Serialize traits for most types if the rkyv feature is enabled.
Tasks:

[x] Implement traits for types
[x] Await https://github.com/rkyv/rkyv/pull/358 to remove the dependency patch
[ ] Publicly expose archived types and resolvers
[ ] Provide validation support for safe checked deserialization

If this goes through, considering rkyv will be additional overhead for future type changes.
Although, fortunately, the only manual implementation was written for Timestamp; for all other types the derive macros were sufficient.
Another downside is that the trait implementations bring one - most of the time even two - new types for every type they're implemented on (Archived{TypeName} and {TypeName}Resolver). This could lead to larger binaries and longer compile times. Since this is behind a feature flag and only enabled if people actually need rkyv though, I don't think this should be an issue.
The advantage would be that users have access to highly performant (de)serialization which can be especially useful for caching purposes.
It might even be an option to store data in the in-memory cache in serialized form to reduce its size and instead of dealing with references into the dashmaps, one could provide methods that take an FnOnce and execute it on archived types such as
impl InMemoryCache {
    pub fn user<F, O>(&self, user_id: Id<UserMarker>, f: F) -> Option<O>
    where
        F: FnOnce(&Archived<User>) -> O,
    {
        self.users
            .get(&user_id)
            .map(|bytes| unsafe { rkyv::archived_root::<User>(bytes) })
            .map(f)
    }
}

This just serves as an example, I can't say how much (if any) performance / memory improvement this would bring for now, maybe its more of a project for a third-party crate.
One could argue "Why stop at rkyv, traits for prost, abomonation, flatbuffers, and all other ones should be supported too!"
which is a valid point and remains for you to decide. I just think that rkyv comes with a great amount of utility and its performance scores amazingly in benchmarks.
If you're fine adding this, I'll look into completing the remaining task.

Since rkyv is by no means the only serialization library, it's only fair for me to compare against some alternatives to give a better picture.
I've added serialization implementations for some other crates to get a feeling for their ergonomics and benchmarked each one.
Performance
Benchmark results

rkyv's unvalidated deserialization is unbeatable which makes sense given that it doesn't really need to do much when deserializing. When insisting on validated safe deserialization, however, speedy would be the choice.
If size is more of a concern than speed, parity-scale-codec performs consistently better than the others. Especially rkyv performed surprisingly bad, presumably because its serialization relies on alignment to make the quick deserialization work.
Serialization performance is a tighter race between rkyv and speedy but speedy seems to be a little more consistent in the end.

Ergonomics

borsh: The derive macros don't offer much room for configuration which forces a manual implementation on recursive types.
parity-scale-codec: Very easy; simple derives is essentially all that's needed. And they even offer an option to optimize even more for size at the cost of speed.
speedy: Also very simple through derive macros. The only optional overhead to improve handling on enums is an additional macro attribute.
rkyv: A lot of work is done by the derive macros again. Whenever a manual implementation is required though, rkyv requires some unsafe code. Fields of type Option<Id<T>> require additional overhead for optimized usage. To provide validation support, another derive has to be added. That one doesn't play well with the bitflags! macro, requiring a manual implementation of unsafe code again. Also, validation only works on repr(Int) enums so I had to add a #[repr(u8)] to most enums.

Alternatives
Apart from the tested crates, there were a bunch more alternatives I could have tested but each one was carrying too much of a drawback for me to even consider them:

serde_bare, bincode, bson, nachricht, postcard, rmp, and ron all come with serde itself which would've been great but none of them allow deserialize_any which I don't think is feasible to avoid when dealing with discord's initial JSON.
capnp, prost and flatbuffers all performed decently well in the general benchmark and are supported across multiple languages since they operate on schema files for types but I figured the amount of overhead to keep those schemas in sync with twilight itself is just too much of a burden.
alkahest and dlhn are still in development and far from being sufficiently mature. I almost counted speedy in here too but its performance seemed too good to discount.
abomonation is just generally not advisable for any kind of production. It reeks of unsafe and potential UB.

Conclusion
speedy is the only crate that could keep up with rkyv's performance and even beat it at times. However, speedy is still fairly new and not as popular (yet). When it comes to size, rkyv doesn't even come close to parity-scale-codec's results.
Adding both rkyv and parity-scale-codec is a possibility but supporting both crates would add even more friction to future type changes. Also, rkyv was the least convenient to implement and likely requires a little more overhead for future support than the other candidates.
All in all, rkyv is not the clear-cut winner that I naively had in mind. It remains an awkward choice whether twilight should add support for a performant serialization crate.

On further thought, I think even more important than finding the best crate to serialize things efficiently, is to omit uninteresting fields in the first place from serialization. This cannot be done dynamically by any crate I looked into.
In the end, if users really want to squeeze the most performance out of serialization, it seems impossible to avoid cloning twilight and fine-tuning things manually.


javascript main error

i dont know what to do all i get it is javascript main error and discord hangs please help me with it

try this installer https://github.com/rauenzi/BetterDiscordApp/releases/ and choose BandagedBD and then while running the installer checkmark Zere's fork

thank you but this theme does not work https://github.com/gco360/better-discord-theme-gco

thank you but this theme does not work https://github.com/gco360/better-discord-theme-gco
im using BandagedBD

gco it must be
//META{"name":"A_GCO_theme","description":"made by reuben owner of gco","author":"reuben","version":"1.0"}*//{}

that's why it's not working


[Bug]: hotham-simulator on Linux crashes on exit

What happened?
It looks like destruction of XrContext leads to hotham-simulator crashing on linux.
test_xr_context_smoke_test which is literally just the following:
    #[test]
    pub fn test_xr_context_smoke_test() {
        XrContext::new().unwrap();
    }

crashes on linux (AMD) in State::destroy when calling destroy_swapchain. Commenting out destroy_swapchain (and destroy_surface to avoid a vulkan validation error) makes things no longer crash.
Version
latest
In which Hotham component are you seeing the problem on?
Simulator
What VR System are you seeing the problem on?
Other
What OS are you seeing the problem on?
Linux
Relevant log output
Finished test [unoptimized + debuginfo] target(s) in 0.05s
     Running unittests (/home/jmgao/Projects/v3/hotham/target/debug/deps/hotham-009eadd7dd1fbec7)

running 1 test
WARNING: radv is not a conformant Vulkan implementation, testing use only.
[HOTHAM_SIMULATOR] Created physical device: 0x7f960001a2b0
[HOTHAM_SIMULATOR] Done! Device created: 0x7f9600712070
[HOTHAM_SIMULATOR] enumerate_view_configuration_views called with: 2
[HOTHAM_SIMULATOR] Creating XR Swapchain..
[HOTHAM_SIMULATOR] ..done.
[HOTHAM_SIMULATOR] Building windows swapchain..
[HOTHAM_SIMULATOR] Creating window with visible true..
WINDOW SCALE FACTOR, 1.0
[HOTHAM_SIMULATOR] ..done.
[HOTHAM_SIMULATOR] Creating surface..
[HOTHAM_SIMULATOR] ..done
[HOTHAM_SIMULATOR] About to create swapchain..
[HOTHAM_SIMULATOR] Created swapchain: 0x120000000012. Sending..
[HOTHAM_SIMULATOR] Received swapchain: 0x120000000012
[HOTHAM_SIMULATOR] Creating descriptor sets..
[HOTHAM_SIMULATOR] Created descriptor pool 0x190000000019
[HOTHAM_SIMULATOR] Allocating descriptor sets with layouts: [0x1a000000001a, 0x1a000000001a, 0x1a000000001a]
[HOTHAM_SIMULATOR] Creating render pass..
[HOTHAM_SIMULATOR] ..done!
[HOTHAM_SIMULATOR] Creating pipelines..
[HOTHAM_SIMULATOR] ..done!
[HOTHAM_SIMULATOR] ..done
[HOTHAM_SIMULATOR] Returning with Swapchain(19791209299986)
[HOTHAM_SIMULATOR] Create action set called with "input"
[HOTHAM_SIMULATOR] Created path 0x7f9600896680 for /user/hand/left
[HOTHAM_SIMULATOR] Created path 0x7f9600896680 for /user/hand/right
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/left/input/grip/pose
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/left/input/aim/pose
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/right/input/grip/pose
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/right/input/aim/pose
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/left/input/squeeze/value
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/left/input/trigger/value
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/left/output/haptic
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/right/input/squeeze/value
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/right/input/trigger/value
[HOTHAM_SIMULATOR] Created path 0x7f96008ab710 for /user/hand/right/output/haptic
[HOTHAM_SIMULATOR] Created path 0x7f96008ab0c0 for /interaction_profiles/oculus/touch_controller
[HOTHAM_SIMULATOR] Created left hand space: SpaceState { name: "Left Hand", position: "x: -0.2, y: 1.4, z: -0.5", orientation: "x: 0.707, y: 0, z: 0, w: 0.707" }, Space(11407146275288697293)
[HOTHAM_SIMULATOR] Created left hand space: SpaceState { name: "Left Hand", position: "x: -0.2, y: 1.4, z: -0.5", orientation: "x: 0.707, y: 0, z: 0, w: 0.707" }, Space(2508067510750580967)
[HOTHAM_SIMULATOR] Created right hand space: SpaceState { name: "Right Hand", position: "x: 0.2, y: 1.4, z: -0.5", orientation: "x: 0.707, y: 0, z: 0, w: 0.707" }, Space(9823019420125182543)
[HOTHAM_SIMULATOR] Created left hand space: SpaceState { name: "Left Hand", position: "x: -0.2, y: 1.4, z: -0.5", orientation: "x: 0.707, y: 0, z: 0, w: 0.707" }, Space(12203213533413683905)
[HOTHAM_SIMULATOR] Attach action sets called
[HOTHAM_SIMULATOR] Destroy called..
[HOTHAM_SIMULATOR] Closed called!
[HOTHAM_SIMULATOR] Closed called!
[HOTHAM_SIMULATOR] Closed called!
[HOTHAM_SIMULATOR] Closed called!
[HOTHAM_SIMULATOR] Closed called!
error: test failed, to rerun pass '-p hotham --lib'

Caused by:
  process didn't exit successfully: `/home/jmgao/Projects/v3/hotham/target/debug/deps/hotham-009eadd7dd1fbec7 'resources::xr_context::tests::test_xr_context_smoke_test'` (signal: 11, SIGSEGV: invalid memory reference)

Thanks so much for this @jmgao ❤


Support for globs

Would it be possible for the watcher API to support Globs as arguments to the Add and Ignore methods? This would make watching specific directories for only certain file types much easier:
w.Add("./**/*.go")

Or is there a way to achieve this with the current interface?

Add a filter function may be a better way.
w.Add(path,filter)
func filter(fileName string) bool

Is there a good way to go about this with the CLI?

Hey guys!
For now I would think, the easiest way to do this would be to use the filepath.Glob method that the standard library provides for adding and ignoring files... :)
(If it seems reasonable to shortcut this and allow globs directly in the add and ignore methods I can consider that too)
For example:
package main

import (
	"fmt"
	"log"
	"path/filepath"

	"github.com/radovskyb/watcher"
)

func main() {
	w := watcher.New()

	paths, err := filepath.Glob("**/*")
	if err != nil {
		log.Fatalln(err)
	}

	for _, path := range paths {
		err := w.Add(path)
		if err != nil {
			log.Fatalln(err)
		}
	}

	for path := range w.WatchedFiles() {
		fmt.Println(path)
	}

	// ...
}

However, one thing to note is that it would decrease performance if the plan was to have the globs for ignoring files update in between polling to see if any other files added match the glob (if the polling duration is a short one), since it would have to re-glob to see if there were any other files that need to be ignored. If the plan is simply to just ignore what's already there though, then filepath.Glob should be sufficient like within the above example.
I hope that makes sense!

Did using filepath.Glob work out for you @kbirk? :)
@skinnyjames Was the CLI question something that you might find useful? Feel free to submit any pull requests if you want by the way, but otherwise I might be able to chuck it in the CLI sometime soonish.

noticed it picks up changes to .git as well by default

Yes it did work, thank you :). Eventually I needed the globs to be run between polling to check for file additions so I ended up writing my own solution down the line. You can close this issue if you like.

Please do add this to the CLI. I'd love to not use fswatch.


Schema provides all attributes, even if not all are being encoded.

Maybe something for v2 as this is breaking. It stems from this issue:
https://github.com/cloudcreativity/laravel-json-api/issues/136
At the moment the SchemaProviderInterface::getAttributes() will return all resource attributes, even if not all of them are being encoded (i.e. if the fields parameter has specified that only some should be included).
If you have an attribute that is costly to calculate, you have no way of knowing you don't need to do the calculation because the method isn't provided with the fields parameter.
It would be useful if the getAttributes method is given the information to know whether it needs to calculate an attribute, along the lines of $includeRelationships in the getRelationships method.
Probably something for v2 as changing the interface would be breaking?

Reopened this because the problem has been re-introduced in v3 by removing the $fieldKeysFilter from the getAttributes() method.

The lib has no implementation of SchemaInterface. It's on developer's side and they are free to implenent it the way they wish.
@lindyhopchris Can you please tone down a bit? I'm closing the issue because currently, it duplicates #236  :fire:


Make sybil test init files

❰marenz❙~/frequenz/frequenz-quantities-python(gitinitsybil)❱✔≻ pytest
===================================================================================== test session starts =====================================================================================
platform linux -- Python 3.11.9, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/marenz/frequenz/frequenz-quantities-python
configfile: pyproject.toml
testpaths: tests, src
plugins: asyncio-0.23.7, mock-3.14.0, hypothesis-6.104.2
asyncio: mode=Mode.AUTO
collected 206 items

tests/test_quantities.py .............................................................................................................................................................. [ 76%]
..............................................                                                                                [ 99%]
src/frequenz/quantities/__init__.py .                                                                                [ 99%]
src/frequenz/quantities/_quantity.py .                                                                                [100%]

==================================================================================== 206 passed in 12.73s =====================================================================================
❰

I am still wondering if a certain complexity of imports could cause problems...

Maybe try it in the SDK and channels repos too? Those are probably the repos with more files and more examples.

I think this was confirmed to work correctly now. Should we just go and fix repo-config instead and make a patch release with the fix so it is propagated to all repos automatically instead of adding this hack?

Haha, I wrote almost the same comment 3 weeks ago. Let's go for it in repo-config. If it doesn't work for some complex repo, we can always rollback, but keep adding this hack to more and more repos will make the rolling back much more complicated than handling it indirectly via repo-config.


[go_router] Redirection with Riverpod example

This is a request to add an example to the go_router package that shows how to set up redirection with Riverpod.
Use case
go_router is currently not very well documented and so it took me (as an individual who is familiar with how the different parts of the framework work) quite a while to figure out a solid way to use go_router with flutter_riverpod properly to handle a sign-in flow.
Proposal
Because of that, I feel like the existing example catalog should be updated with another sample to complement "Redirection" and "Asynchronous Redirection".
This is mostly because it works fairly differently under the hood.

Is this something you would be willing to integrate into the package? I would also be willing to contribute the example.
I think this would be very beneficial to also ensure that updates to the package do not break this setup.
Edit: sample here dartpad.dev & explanation here issuecomment-1269053217.
Setup
This method makes use of the fact that reconstructing GoRouter will call redirect again.
That means that by providing a redirect function that depends on providers, we can both handle calls to redirect caused by internal calls (Navigator.pop, context.go, etc.) as well as the providers updating.
It looks something like this:
// This is super important - otherwise, we would throw away the whole widget tree when the provider is updated.
final navigatorKey = GlobalKey<NavigatorState>();
// We need to have access to the previous location of the router. Otherwise, we would start from '/' on rebuild.
GoRouter? _previousRouter;

final routerProvider = Provider((ref) {
  final bool loggedIn = ref.watch(loggedInProvider);

  return GoRouter(
    initialLocation: _previousRouter?.location,
    navigatorKey: navigatorKey,
    routes: [...],
    redirect: (context, state) {
      // if the user is not logged in, they need to login
      final bool loggingIn = state.subloc == '/login';
      if (!loggedIn) {
        return loggingIn ? null : '/login';
      }

      // if the user is logged in but still on the login page, send them to
      // the home page
      if (loggingIn) {
        return '/';
      }

      // no need to redirect at all
      return null;
    },
  );
});

This setup is so much cleaner and easier to grasp than the existing Redirection example using provider.
Also note that I was not able to set this up using refreshListenable, which is not documented, confusing, and also does not work correctly from what I can tell (it can cause invalid states). Furthermore, the refreshListenable setup seems way more hacky than the riverpod setup above to begin with.
In conclusion, I think that this method is pretty powerful and I would like to see it included in the package publicly somehow in order to ensure that others can use it and it can be maintained (although it should always work if the package is implemented correctly and checks for equality properly without throwing away the widget tree).

@creativecreatorormaybenot
Do you plan to add this example yourself? If not, I can take this.
(Since Riverpod is supposed to be a better version of Provider, I think adding Riverpod examples for both "Redirection" and "Asynchronous Redirection" make sense to me)

Hi @creativecreatorormaybenot
On Riverpod's documentation pages, under the 3rd party examples folder, there's a repo I've made a while ago you can explore to check out possible solutions to integrate GoRouter and Riverpod together.
It's going to be updated with GoRouter v5 + Riverpod v2 as soon as this issue will be fixed.

Do you plan to add this example yourself? If not, I can take this.

@TahaTesser That would be super awesome! I certainly think that having Riverpod examples makes sense 👍
I would be glad to review 👌

Hi @lucavenir - I have explored that solution previously as well. But to me that was even more of a hack and did not work perfectly.

In your example snippet above, routerProvider will rebuild everytime there's an authentication change; this will most likely cause a whole app rebuild (onto the root of our apps), since I'd expect the main App to watch our routerProvider. This is most likely undesired.

This is not right. If you look at the snippet, you will see that I consciously pass a navigatorKey to GoRouter. This means that the tree will be kept upon "rebuilding" the routerProvider. This makes use of Flutter's ability to keep the tree by comparing its elements. Because the setup is the exact same before and after, the same widgets will be inserted and if you write your widgets correctly, all state will be kept and nothing will be discarded.

The above (and more caveats) are what motivated this repo to not only write down a simple example, but to properly analyze different possible approaches.

I very highly appreciate this effort 🙏 Thanks @lucavenir. However, I am sorry to tell you that I do not like the solutions in this repo. Especially piping all provider changes into listenables seems awful to maintain.
Feel free to try out my proposed solution and report if you can find any problems with it. This is also why I would really love to have an example/ for this setup in go_router because we could properly test if there are any problems with state being discarded or undesired rebuilds.
About rebuilds
Note that you are of course right that the App in this setup:
class App extends ConsumerWidget {
  const App({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    final router = ref.watch(routerProvider);
    return MaterialApp.router(
      routerConfig: router,
    );
  }
}

Will always rebuild when one of the dependencies of the routerProvider changes. However, since we pass the navigatorKey, we really do not care about this. This rebuild basically has minimal cost (and the framework could also trigger it at any time) and all of our actual tree, i.e. our pages returned from the router, that we may not want to rebuild unnecessarily will not rebuild. That is because the framework checks if the pages have updated and they will not update unless there is an actual change in navigation.
I do, however, agree that GoRouter could be further optimized for being reconstructed. One crazy idea I have is removing refreshListenable altogether and recommending to rebuild GoRouter using a ValueListenableBuilder instead (or use flutter_riverpod).

One crazy idea I have is removing refreshListenable altogether and recommending to rebuild GoRouter using say a ValueListenableBuilder instead (or using flutter_riverpod as outlined).
If we further optimize GoRouter for being reconstructed, this would be the perfect solution for redirection.

One crazy idea I have is removing refreshListenable altogether and recommending to rebuild GoRouter using say a ValueListenableBuilder instead (or using flutter_riverpod as outlined).
If we further optimize GoRouter for being reconstructed, this would be the perfect solution for redirection 🚀

One crazy idea I have is removing refreshListenable altogether and recommending to rebuild GoRouter instead. Say using ValueListenableBuilder or using flutter_riverpod as outlined.  To me this seems so much cleaner and would also reduce complexity.
If we further optimize GoRouter for being reconstructed, this would be the perfect solution for redirection 🚀

Hi @lucavenir, this is not right. If you look at the snippet, you will see that I consciously pass a navigatorKey to GoRouter. This means that the tree will be kept upon "rebuilding" the routerProvider. This makes use of Flutter's ability to keep the tree by comparing its elements. Because the setup is the exact same before and after, the same widgets will be inserted and if you write your widgets correctly, all state will be kept and nothing will be discarded.
Feel free to try out my proposed solution and report if you can find any problems with it.

I've been sloppy in reading your code, sorry if I made you lose time! Thanks for the detailed explanation.
This makes me rethink a lot of the integrations with Riverpod: we could just use Providers like we always do: we watch elements in there declaratively, without implementing external Listenables. I have to test this out and evaluate its scalability and if it's feasible (the Flutter tree won't rebuild - but our Provider containing the Router will).

Especially piping all provider changes into listenables seems awful to maintain.

I'm obsessed with best practices and I can't agree more! I'm planning to erase out all the "not-so-good" practices in some examples. You see, those examples were community - driven, they came out from a discussion in Riverpod's repo. Most of the examples feel like "legay code" ad this point. I've just released an update about this. I'm trying to maintain that repo as a "community wiki", so there's a good chance I'll touch the examples a lot in the next days.

This is also why I would really love to have an example/ for this setup in go_router

I absolutely agree with this. See the above sentences.

@creativecreatorormaybenot Do you mind take a look at this branch? I'm trying your suggestion but I still seeing the "root widget" rebuilding, there.

@lucavenir I am not sure what the "root widget" in your setup means, but here you go: https://dartpad.dev/0a77fd6105aeee44858b794e17e83ffc
This is a full setup using my riverpod + go_router approach.
Here is an in-depth showcase 💯  Here we have..
https://user-images.githubusercontent.com/19204050/194175676-3c4b94b6-fad8-4999-a94a-7a54701211fb.mp4
Note that the MaterialApp itself also does not rebuild internally because it does not change. So it is mainly the App.build call, which is very cheap.

Thank you @creativecreatorormaybenot for the in-depth explanation!! This is way clearer. I guess I'm still confused by Flutter's keys.

Hi @creativecreatorormaybenot !
I've updated the examples in this repo, thanks to you! I've cited you as a contributor, but I'm unsure on how to give you more credit than that 😛
Your feedback would be highly appreciated - and I'm guessing that the complete_example folder in there could be used to solve this issue 👀

I really appreciate the sample @lucavenir. The crediting is fine. I think a link to the issue would help for context 🙂
Note that I discovered a problem. In the case that you do not cover all of your routes in your redirect handler (e.g. if you have nested routes and only redirect to and from the top route), rebuilding the router will mess up your current state as the initial location is always /.
The workaround for this is adding:
...
// We need to have access to the previous location of the router. Otherwise, we would start from '/' on rebuild.
GoRouter? _previousRouter;

final routerProvider = Provider((ref) {
  return GoRouter(
    initialLocation: _previousRouter?.location,
    ...,
  );
});

I am not the biggest fan of this since it is even less clean (with the router being outside of any providers). However, it does work and is solid / non-fragile in theory.
Another implication is that modals will never work with this approach, i.e. routes that are pushed via GoRouter.of(context).push since they are not encapsulated in the current location. What does work is modals inside of a page, e.g. modal_bottom_sheet. So this is not an issue.
For routes that are pushed on top of the whole stack / not part of the routes defined in routes, I have come up with my own solution that integrates way better with riverpod anyway.
That is to say, I still greatly prefer this method over the previous methods. Although, we should be aware of the limitations.

I really like your samples - good job! They are not complex enough to have encountered the edge cases I mentioned, so it might be worth playing with it a bit more to discover these as well 👍

I am not sure what the "root widget" in your setup means @lucavenir, but here you go: https://dartpad.dev/0a77fd6105aeee44858b794e17e83ffc This is a full setup using my riverpod + go_router approach.
Here is an in-depth showcase 💯 Here we have (sound on 🔈)..
Screen.Recording.2022-10-05.at.22.19.27.mp4
Note that the MaterialApp itself also does not rebuild internally because it does not change. So it is mainly the App.build call, which is very cheap.

This solution is not working as shown in the video on the latest stable channel 3.7.3. @creativecreatorormaybenot

It may be that you skipped one of the steps I described @Neelansh-ns. I just tested the Gist at https://dartpad.dev/0a77fd6105aeee44858b794e17e83ffc and it works for me.

Hi there,
I want to share some thoughts.

I feel this example is very much needed. I'm still pretty much proud (😂) of the examples from my repo, but maybe having an "official" go-to solution is best;
After some trials, and after riverpod v2, together with riverpod_generator, I like this solution better. In a nutshell: there's no passing of a ref object, router isn't rebuilding on authentication changes and our notifier actually implements Listenable, which is what's required by current's go_router's API.
All of this conversation might be overridden by this comment from go_router maintainers. I have a gut feeling that there's a good chance this issue is being "delayed" for a good reason: go_router is still on its way of a "best" design;
Therefore, taking all of this into account (i.e. go_router APIs have changed in a way that is making the maintainers think 'refreshListenablemight be deprecated) and adding [this other comment](https://github.com/flutter/flutter/issues/116651#issuecomment-1361716448) to the equation, maybe [this solution](https://github.com/flutter/flutter/issues/116651#issuecomment-1428028054) might be a good reference. Note: I don't like this very much, I don't find it declarative and I don't like the fact the responsibility of triggering redirects is moved away fromrouter`.

@creativecreatorormaybenot Would like to ask on this. I actually liked your provided solution  as everything make sense in first look. However,

I have come up with my own solution that integrates way better with riverpod anyway.

What was your solution on handling modals on top of the route stack?
And as for this. How do you update the _previousRouter then?
...
// We need to have access to the previous location of the router. Otherwise, we would start from '/' on rebuild.
GoRouter? _previousRouter;

final routerProvider = Provider((ref) {
  return GoRouter(
    initialLocation: _previousRouter?.location,
    ...,
  );
});

creativecreatorormaybenot
@creativecreatorormaybenot hello , sorry but your example only works for the web platform, testing on Android the widgets are rebuilt despite having the navigatorkey.
I think that even though it is rebuilt, the changes in authentication occur at startup and when closing, so in the worst case it would only be rebuilt twice while using the application.
Anyway I would like a cleaner solution if anyone has an idea how to use go_router with firebase

A note on _previousRouter?.location; since GoRouter 9.0.0, it should be _previousRouter?.routerDelegate.currentConfiguration.fullPath.

Hi there,
I solved this problem by creating a router provider that who takes care about update the router every time the auth state changes.
I did some tests but i'm not sure if is conceptually right or not, so I would appreciate it if you could give me some feedback.
RouterNotifier:
@riverpod
class RouterNotifier extends _$RouterNotifier {
  @override
  FutureOr<GoRouter> build(WidgetRef reff) {
    ref.watch(authNotifierProvider);
    return routerGenerator(
      ref: reff,
      rootNavigatorKey: rootNavigatorKey,
      shellNavigatorKey: shellNavigatorKey,
    );
  }
}

In the MaterialApp i have just added the watcher to the gorouter notifier value
MaterialApp.router(
        title: 'App',
        debugShowCheckedModeBanner: false,
        routerConfig: ref.watch(routerNotifierProvider(ref)).value,
      )

Than this is the function that create the GoRouter object
GoRouter routerGenerator({
  required WidgetRef ref,
  GlobalKey<NavigatorState>? rootNavigatorKey,
  GlobalKey<NavigatorState>? shellNavigatorKey,
}) {
  return GoRouter(
    navigatorKey: rootNavigatorKey,
    initialLocation: PostsPage.route,
    redirect: (context, state) {
      if (!ref.read(authNotifierProvider.notifier).isSignedIn()) {
        return LoginPage.route;
      }
      return null;
    },
    routes: ...

Thank you for your feedback!

I was getting weird bugs when my GoRouter object reference changed due to using ref.watch inside the router provider. I think it's better to use ref.read inside the redirect method, so your GoRouter object reference is always the same, and listen to the auth status provider in the root widget and manually redirect based on the new auth status.