override default validation behavior in custom keyword

I might be trying to bend the rules a bit much here, but is it possible to override the validation behavior of other keywords in a custom keyword - perhaps like so:
{ type: 'string', allowAFunctionAsWell: true }

Understandably this could be implemented with anyOf/oneOf, but in my particular case this would make implementation much cleaner if it were possible.

It's not possible at the moment. I have some doubts that it is a good idea to be honest, particularly for type keyword.
You may want to use custom keywords instead. I've recently started adding some useful keywords to ajv-keywords, both to use them and to serve as implementation examples...

@epoberezkin so I'm using ajv to replace an existing home brewed schema system in an ODM (we've briefly discussed this via email), and it supports ReQL terms which are functions later evaluated by the database driver. For example, let's say I have the following schema:
let Model = test.thinkagain.createModel(test.table(0), {
  type: 'object',
  properties: {
    id: { type: 'string' }
  }
});

It's reasonable for the user to want to do something like let m = new Model({ id: r.uuid() }), which will result in a string once processed by the server and returned, however on initial validation it will fail because r.uuid evaluates to a function. I'm trying to find a way to easily support that, instead of monkey-patching every schema provided to createModel to change a simple definition to e.g. oneOf: [ <schema they provided>, { myKeyword: 'that supports ReQL Terms' } ] which may make validation results a bit confusing. One solution that seemed reasonable to me was to patch the schemas to just add a keyword that might instead override the type in this special case.
I totally understand this is a very special contortion that sort of exists outside the intentions of json-schema, but I figured it was worth the discussion!  I think this would improve the user experience, as adding the anyOf/oneOf is more of a non-trivial modification of the initial schema.
Thoughts?

I think using a custom keyword { "typeof": ["string", "function"] } is better than redefining standard type keyword for several reasons.

https://github.com/epoberezkin/ajv-keywords#typeof

@epoberezkin one of the main goals of using json-schema in my case is to actually be able to reuse the schema for the front and backend of my system. I'm trying to make the schema I use for this particular model as "pure" as possible, using such a keyword sort of completely defeats the purpose. Further, the goal here is to internally amend the given schema only in the case where you are going to actually use the rethinkdb driver (the backend), so you could still use the same schema for form generation on the front end (no need to accept functions here).
I guess the real issue I have presently is that there is no way to bypass the default validation behavior at all. I could have, for instance, manually checked the schema and somehow omitted the property using a function from validaton as an alternative to the proposed keyword solution of bypassing the type check - if there had only been a way to do so. So I guess more to the point: is there some method you might find more palatable here for bypassing validation rules? (I'd also like to point out that I wasn't initially proposing we be allowed to override the type keyword itself, although that would work too, but instead allow another keyword to indicate that validation had succeeded and no other validation functions should be run - a potential optimization in some cases as well perhaps)

You can define custom keyword that would analyse errors collected so far and remove them based on some criteria. So your allowAFunctionAsWell could be a custom keyword that would check that if type validation failed but the data is a function it would remove the validation error.

@epoberezkin ah very clever 😄 I'll give that a shot, as ever I appreciate the quick feedback

@epoberezkin if I'm not misunderstanding this would have to be done as an inline validator, because we don't have access to the aggregated errors in a plain compile right?

got it, at least an incredibly basic form of ignoring all errors;
keywords.ignoreErrors = {
  errors: 'full',
  statements: true,
  inline: (it, keyword, schema) => {
    return 'return true;';
  }
};

I think I can get this working from here, cheers!

Yes, that's correct, it will have to be inline. You've probably found that already: https://github.com/epoberezkin/ajv/blob/master/CUSTOM.md#reporting-errors-in-custom-keywords :)
By the way, return true; may not complete validation if it is in some subschema that was compiled to a separate function rather than inlined. But in general, that's the idea.

Okay, I'll start with the simple cases and move from there. If it's part of a subschema there's no real way to fully bypass validation though right?

Subschemas that don't have refs inside them are inlined (can be turned off). Also, before you return you need to clear errors. But I don't think you want to return, do you? I thought you would update vErrors and errors.

See this too: https://github.com/epoberezkin/ajv/blob/master/CUSTOM.md#short-circuit-validation

@epoberezkin aye, I've seen it thanks! I've got an incredibly early preview of the work I've been doing over here: https://github.com/mbroadst/thinkagain, cheers!


Add a copy of the MIT license to the repository

The asset submission says the asset's license is MIT. Please add a copy of the MIT license to the repository to confirm this :slightly_smiling_face:

I uploaded the MIT LICENSE file, thanks.

Thanks! The asset is now approved: https://godotengine.org/asset-library/asset/570


Cannot save into fields

Why Iframe module cannot save into fields?
Content
BottomContent

Closing this for now. If you are still having this issue then we're going to need some more info. :)

I've tested this feature and I can confirm that you can indeed save into fields. Possibly there is some configuration in your project.
In any case, I apologise for not answering this for 2 years.


Crash reporting heaven

Summary

I have used RN for a long time, and for all this time, crash reporting has been less great than native development crash reporting. At some point, companies like sentry, bugsnag and a bunch of others started supporting sourcemaps for js crashes in RN, which helped a lot.
But native crashes were (and still are) much harder to diagnose.
..Until now :D
I have make a repo of a sample RN app, included this PR in it, and some code and screenshots to help.
The repo is here.
Explanation
I was trying to get good crash reports from native crashes in iOS for a looong time. I spoke with people in sentry, in bugsnag and more, and I could not get this solved. There was no clear way to get the native crashed to display correctly.
I made two repos here, one for sentry and one for bugsnag, demonstrating the correct js handling and the bad native handling.
After all this, and talks with their support, twitter etc, I investigated further, on why this was happening. I thought there must be some reason that native crashes look bad in all the tools, and in the same way. Maybe it's not their fault, or up to them to fix it, or maybe they didn't have the experience to fix it.
In a test project I created, I checked what's up with the RCTFatalException, and I found out that the React Native code is catching the NSExceptions that come from any native modules of a RN app and converting it into an string and sending it to RCTFatal that created an NSError out of that string. Then it checks if the app has set a fatal error handler and if not, goes ahead and throws that NSError.
The problem here is that NSException has a bunch more info that the resulting NSError is missing or is altering. Turning the callstack into a string renders crash reporting tools useless as they are missing the original place the exception was thrown, symbols, return addresses etc. In both repos above it can be seen that both tools were thinking that the error happened somewhere in the RCTFatal function, and it did, since we create it there, losing all the previous useful info of the original exception. That leaves us with just a very long name including a callstack, but very hard to actually map this to the code and dsym.
I added a fatal exception handler, that mirrors the fatal error handler, as the error handler is used around React Native internal code.
Then I stopped making a string out of the original NSException and calling RCTFatal, and I simply throw the exception. This way no info is lost!
Finally, I added some code examples of native and js crashes and added a part in the RNTester app, so people can see how a js and a native error look like while debugging, as well as try to compile the app in release mode and see how the crash report would look like if they connect it to bugsnag or sentry or their tool of choice.
I have attached some images at the bottom of this PR, and you can find some in the 3 repos I linked above.
Changelog

[iOS] [Fixed] - Changed the way iOS native module exceptions get handled. Instead of making them into an NSError and lose the context and callstack, we keep them as NSExceptions and propagate them.
[General] [Added] - Example code for native crashes in iOS and Android, with buttons on RNTester, so developers can see how these look when debugging, as well as the crash reports in release mode.
Test Plan

You can take the app here and add your sentry dsn etc, or comment out the sentry line in App.js and add bugsnag or instabug or fabric or any other tool you like, and test the result of a native crash. Maybe also compare it with the existing native reports produced by the current RN version.
(I have cancelled any sensitive info that is contained in these repos like api keys etc there, so no worries for that.)
Appendix
Good js crashes:

Bad native crashes (pre-PR):

Good native crash (post-PR):

Maybe @HazAT and @fractalwrench are interested in this too.

Hm I will take a look for adding tests. I might need some help/guidance for this, but I will look.

Would I test the native module in a try-catch and check the exception there, or would I test in js by calling the native module and do the try-catch dance there? 🤔

I am adding an extra commit, that keeps the old RCTFatal behavior on the case of RCT_DEBUG being true. That is because when developing and debugging, it's more useful to see the string-constructed callstack on the app.

If we debug with Xcode running, the exception breakpoint will stop where it should. (in RNTester, that is CrashyCrash.m:19).
If we debug without Xcode (expo, or just running the app in device or simulator but without Xcode), we should get some info on the redbox. (this is the current master behavior).

If we are on release build type, then we want the original exception, without wrappers or string callstacks. We need the original actual exception.

Before the commit I'll do now, in this PR, the redbox while debugging was
 because or removing the RCTFatal call. I added it back, but restrict it only in the debug mode. This gives us as much info as possible in debug, and the correct info in release.

@pvinis Awesome work!

I just tried the merged stack on sentry and it works soo well. <3


Readme should have usage snippet

I would strongly suggest, the readme should have a usage vignette, e.g.,
from rolch.model import Foo
from rolch.data import load_bar
from rolch.prettyplot import veryprettyplot

X, y, X1, y1, X2, y2 = load_bar()
m = Foo()
m.fit(X, y)
y_pred = m.predict(X1)
m.update(X1, y1)
y_pred2 = m.predict(X2)

veryprettyplot(everything)

Good point!
Note to self: keep in mind fornhttps://github.com/simon-hirsch/rolch/pull/21

This was implemented in https://github.com/simon-hirsch/rolch/pull/21


Unable to compile TF 1.3 from source using full MKL

Steps to reproduce:
git clone https://github.com/tensorflow/tensorflow.git test
cd test
git checkout r1.3
yes "" | TF_NEED_CUDA=0 TF_NEED_MKL=1 TF_DOWNLOAD_MKL=0 MKL_INSTALL_PATH=<path>/l_mkl_2017.3.196/inst/mkl ./configure
bazel build --config=mkl -c opt --verbose_failures //tensorflow/tools/pip_package:build_pip_package

OS version: Ubuntu Linux 14.04
Bazel version: 0.5.3
Error message:
ERROR: missing input file '//third_party/mkl:libmklml_intel.so'
ERROR: <path>/tensorflow/test/third_party/mkl/BUILD:16:1: //third_party/mkl:intel_binary_blob: missing input file '//third_party/mkl:libmklml_intel.so'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
ERROR: <path>/tensorflow/test/third_party/mkl/BUILD:16:1 1 input file(s) do not exist

The configure script is creating symlinks in third_party/mkl/ for libmkl_rt.so (see here), which is fine, but not for libmklml_intel.so (see here), which doesn't exist in the full MKL distribution. However third_party/mkl/BUILD references libmklml_intel.so. Is this a bug or is use of the full MKL library not supported in TensorFlow 1.3?

@gunan do you happen to know on the top of your head?

AFAICT, my changes are not in 1.3 branch.
At master, you should only need to configure, then run bazel build with --config=mkl, and that should be it.
On 1.3, we still use the way contributed by intel. I believe this was not a very well tested build path, so it is likely this was broken all along.
I would recommend setting TF_DOWNLOAD_MKL to 1 and see if it works when configure downloads MKL itself.

Yes, it works with TF_DOWNLOAD_MKL=1 but this downloads MKL-DNN, not the full MKL. If one wants to build MKL support into Eigen then, I believe, one needs to use the full MKL but maybe MKL support in TensorFlow is limited to MKL-DNN? Comments in the r1.3 configure script seem to imply that it should support the full MKL which is why I tried it.
By the way, the instructions on the Intel site for building TensorFlow with MKL (which have also been communicated on various TensorFlow support channels) are, I believe, incorrect. It says one should build with the command bazel build --config=mkl --copt="-DEIGEN_USE_VML" -c opt //tensorflow/tools/pip_package:build_pip_package but this has two potential errors:

One should include --config=opt in addition to --config=mkl to enable all the regular (non-MKL) CPU optimizations in addition to the MKL features.
The flag to compile MKL vector operations into Eigen is EIGEN_USE_MKL_VML, not EIGEN_USE_VML (as explained here).

Correcting the second of these two potential errors yields a new issue in TensorFlow: it fails to compile with a bazel error (something about MKL header files getting included without them being declared in a BUILD file). I don't have the details of that error to hand but it should probably be logged as separate issue anyway and if the MKL build options are changing in the next release maybe it's best to just wait.
I hope the next version of TensorFlow will make clear which of the nine potential configurations of Eigen and TF each built with one of (no MKL, MKL-DNN, full MKL) is supported. It's also not clear whether the optional use of CUDA is an orthogonal consideration or could interact with the use (or not) of MKL/MKL-DNN.

@tobyboyd for more information on this.
MKL support is at the moment provided by Intel, not officially by us yet.
That is why installation guides are not on our webpage, but on intel's page.
I recommend reaching out to them for support with the above questions.

Thanks. I've started corresponding with Intel here: https://software.intel.com/en-us/comment/1911911

A workaround has been provided for using the full MKL with TF r1.3; details at the Intel forums link above.
Additional information about building TF with the full MKL, including Eigen-MKL integration, can be found in the comments attached to this document: https://software.intel.com/en-us/articles/tensorflow-optimizations-on-modern-intel-architecture


Suspicious B62qrEpK1mp7FUpy1mXLzWZm3QHBBczjqwn3DQFJJKrLy8Tyaci2CGD report

Address
B62qrEpK1mp7FUpy1mXLzWZm3QHBBczjqwn3DQFJJKrLy8Tyaci2CGD
Evidence
https://minaexplorer.com/transaction/Ckpa18UL3cV7t7QGHQuanpHvSTQRYHYjRK1Jorwchux7bz6QJ1y55
Link
https://minaexplorer.com/wallet/B62qrEpK1mp7FUpy1mXLzWZm3QHBBczjqwn3DQFJJKrLy8Tyaci2CGD

All of them are same one but it's correct

All of them are the same one from different addresses probably coordinated stuff to avoid getting flagged for all addresses in a short time.

All added to the blacklist :)


Server crash with Electro Seperator

This is from the Horizons Daybreaker pack v1.0.2 but I thought I would post it here also...
Version:
8.1.3
I believe the crash is due to having an Electro Separator with a speed upgrade in it? No idea...
Console Log: http://paste.feed-the-beast.com/view/74494211
Crash Report: http://paste.feed-the-beast.com/view/97a2ad9b
Can it be repeated:
Place speed upgrades in Electro Separator.
Known Fix:
Remove the speed upgrades in an Electro Separator.

Fixed in development.


tests: Expand coverage

[x] This contribution adheres to CONTRIBUTING.md.
[x] New tests have been created for any new features or regression tests for bugfixes.
[x] tox successfully runs, including unit tests and style checks (see CONTRIBUTING.md).

Coverage increased (+0.06%) to 64.23% when pulling 7eee7d1c1de06874b2de4b5fffc522bbd2091941 on epage:more into 861169a39ee682cbda39ad774a8bc16afd6b29db on ni:master.


Add view for 'no projects yet' -page

There's no users without roadmaps currently so to test this, you need to remove someone from their roadmaps and then log in. Create a new project button doesn't do anything yet, it needs to be updated later when the modal is ready.

Fixed the typo and moved css-classes that were used in multiple places to the shared.scss folder 👍 I left the footer in the createProject-page module untouched since it has its own translations and positions

Added all your suggestions!


Fix color of "Search for Class by Pathway!" Button on Admin Portal

Navigate to https://hasspathways.com/admin-portal
See how the "Search for Class by Pathway!" button is a different color from the rest? Fix that!

Some later commit has fixed this issue -

(at least, they don't look different to me), so I'm closing this.


i cannot find Future by lighting?

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
i cannot find future by lighting.

oh

but sir it doesnt look like it changes my game


Move logic to ViewModel so it can be unit-tested

From a test-driven perspective it would be beneficial to get most of the logic covered in small unit tests since this is the fastest and most reliable way of testing an Android application. I believe a ViewModel can be unit-tested whereas a Fragment cannot, without heavy mocking.

Already done in bea8e859


Missing versions of .NET

Determining .NET 1.0 framework via the registry doesn't work with these instructions. You have to iterate through the key "SOFTWARE\Microsoft\NET Framework Setup\Full" looking for entries starting with "v". Then you find the language code and a product key. Once you've got the product key, you can then look in "SOFTWARE\Microsoft\NET Framework Setup\Product{0}" and then get details if it's installed and the version key.
Secondly, installing .NET 3.0 isn't detected using this code in the registry. I assume that is because it only adds a few Windows XXX Foundation elements. Only from .NET 3.0 SP1 is visible in the registry.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2115da38-e95e-46fc-1e2d-da88ef92c6cf
Version Independent ID: a3790e8a-0654-851b-c991-ab7382025406
Content: How to: Determine which .NET Framework versions are installed
Content Source: docs/framework/migration-guide/how-to-determine-which-versions-are-installed.md
Service: unspecified
GitHub Login: @rpetrusha
Microsoft Alias: ronpet

Thanks @jcurl. @rpetrusha can you take a look at this feedback?

May i suggest this for powershell?
(Get-ChildItem "HKLM:SOFTWARE\Microsoft\NET Framework Setup\NDP" -Recurse | ? {$.Property -eq 'Version'}) | Get-ItemPropertyValue -Name Version | Group | % {$.Name}

Would it help if I put my repository on GitHub which I try to implement the rules?

There's a tool called asoft .NET version detector: http://asoft.be/prod_netver.html
I like this one because I don't need to do all the logic and keep it updated without having to touch code. They keep it updated with every new build/release.
It can export to xml/text and just show via the UI.
My 2cts

Hi! Any way to reinstall all .net framework packages in W2K8_R2 with registry values, etc, etc, etc?
I'm starting to lose hope.. It started when I tried to reinstall WSUS server.. it wouldn't install as .net framework was >3.5. So, read all about net framework and tried to use a tool (https://blogs.msdn.microsoft.com/astebner/2012/08/04/net-framework-cleanup-tool-now-supports-net-framework-4-5-and-windows-8) to uninstall all framework "4" versions. It worked ok, and i managed to reinstall WSUS... but sadly it wouldn't work ok, so I uninstalled the WSUS role once again, and used the tool once more to cleanup all net framework remains.
Now, for the 2nd time, i tried to install the WSUS role and it wouldn't install , as it stated in the logs, detected only the framework 2.0 running, so it wouldn't install. Since then, i'm trying to reinstall all net framework packages, but unsuccessfully... i deleted parts of the registry, i removed entries manually... well.. now i'm a little lost!
This is a production server and it has a lot of software, and some don't work now, because they need framework 4 to be installed (including sql server 2008 express - had it, but i'm using 2005 version, wich works actually)... i can't install framework 4 whatsoever, by windows update, or any other means...
So, is there a way to put it all working again? I was avoiding to restore windows to a previous date, so i had next to put all databases backups, files and folders, permissions, etc... to restore them to the actual state.
Help?!?
Thank you in advance!

I'm trying to keep this PowerShell function that handles the logic updated: https://github.com/EliteLoser/DotNetVersionLister - and it's … on GitHub, so feel free to let me know when I'm out-of-date. ;) I wove to defend the .NET version listing to my grave! (hm)


Automated cherry pick of #7950

Cherry pick of #7950 on cloud.
/cc  @pablovelezvidal

Will try to auto merge this PR once all tests and checks are passing. This might take up to an hour.

Trying to auto merge this PR.

Pull Request successfully merged
SHA: 94428f45ce8bd3013027e668c8ad3f76cbfa2127


(PUP-7098) New acceptance test: utf-8 characters in the catalog

This PR adds new acceptance test to verify correct behavior when
catalog containing utf-8 character5s is compiled and then applied
from cache.

CLA signed by all contributors.


[DUOS-622][risk=low] Add sorting to table columns

Addresses
https://broadworkbench.atlassian.net/browse/DUOS-622
Changes

Add utility function for creating sort onClick function
Modify ResearchConsole to use sort function
Modify the DarTable component to allow for sorting

Have you read Terra's Contributing Guide lately? If not, do that first.
I, the developer opening this PR, do solemnly pinky swear that:

[ ] PR is labeled with a Jira ticket number and includes a link to the ticket
[ ] PR is labeled with a security risk modifier [no, low, medium, high]
[ ] PR describes scope of changes

In all cases:

[ ] Get a minimum of one thumbs worth of review, preferably 2 if enough team members are available
[ ] Get PO sign-off for all non-trivial UI or workflow changes
[ ] Verify all tests go green
[ ] Squash and merge; you can delete your branch after this
[ ] Test this change deployed correctly and works on dev environment after deployment

this looks great and works great! my only question is if we want to do sorting on election status? It's not included in this PR and I don't think it would very be helpful to have an alphabetical sort on election status, but I thought I'd ask

Yeah, like you said alphabetic sort on election status doesn't really offer much compared to something like dates or titles. On top of that the search bar does a better job filtering out statuses anyway, so I don't mind it being excluded here.


Chore/move apikey location

rationale is that devs are more likely to look for the apikey config inside ~/ than Application Support dir for dev tools
I'm not as concerned with the deno binary but probably makes sense to have all the config together so devs can check its version if they want to, so I moved it over as well (this one is also easier to move again later if we want since the state of it is managed fully automatically by the CLI)

Hmm I don't think I'm fully sold on this. The idea of XDG basedirs is that it's a way to store user data in a cross-platform way - I don't have to think about whether the CLI is running on macOS, Linux, Windows, or even FreeBSD, OpenBSD, the spec handles that gracefully. I don't know what the implication of storing in ~ is on Windows - is it going to store on C:\?
Another point is that this is really an implementation detail. We don't want the user to actually see this directory, everything inside it is managed by the CLI. Not putting it in $HOME gets it out of the way. Note that this is a different usecase from a global config file, which the user edits manually - in that case it might be sensible to put it in $HOME/supabase.toml to aid discovery.

main motivation is to stay consistent with most dev command line tools, including tools like pulumi, aws, ssh etc. who all store their credentials there and in this format:
% cat ~/.pulumi/
credentials.json  plugins/          workspaces/

compare this to my Application Support dir which is almost exclusively "Big Letter" desktop applications

It personally tripped me up when I went looking to ensure that the correct key was being persisted after running supabase link, which is why I think it should only be hidden in a place where most devs know to look if they need to

also looks like it's also common practice to store it there on Windows:

By default, the credentials file is stored here: On Windows: C:\Users\ username . aws\credentials.


Try normalizing w to the expression level of each gene

Within each column of w, the magnitude of the resulting effect is proportional to the expression level of the gene it gets multiplied with. To try and normalize out this effect, I'm interested in how the results change if we divide each column of w by the expression level of its corresponding gene within the control condition.

what if some expression values equals zero?

Hmm... so if the expression level is zero, the perturbations of that column have no effect. This helped me see that there's a typo in my note above—I think we should multiply, not divide, by the expression level.

The resulting graph after normalizing w:

I remove genes whose control expression level equals zeros from the w matrix because I think the all-zeros columns do not fulfill the requirement of the adjacency matrix while calculating PageRank.

Also have obtained several positive feedback loops:

Seems that these components are all top upstream regulators based on PageRank, and I created a subgraph containing all interactions among these components:

This looks fantastic! That's extremely interesting that all of the positive feedbacks are in the upstream regulators. I think we have a great hypothesis here—it looks like increasing YBX1 would suppress MITF and promote SOX10, so it could drive an intermediate steady state between MITF+ sensitive cells and SOX10+ resistance. We'll have to see what these look like in the steady states.


On macOS, os.tmpdir() isn't the real path

❯ node --version
v7.4.0
❯ node -pe 'os.tmpdir()'
/var/folders/_6/p8qxp_3n62zg9081tvb0lcc80000gn/T
❯ node -pe 'fs.realpathSync(os.tmpdir())'
/private/var/folders/_6/p8qxp_3n62zg9081tvb0lcc80000gn/T

When writing a Node.js module using temp-write, __filename inside that module is different than the file path that is returned by temp-write.
Using macOS Sierra 10.12.3.

Shouldn't os.tmpdir() return the actual path? Weird that it doesn't follow the symlink. That's very surprising. I can do that here, but would you mind opening an issue on Node.js if you agree?

I can do that here, but would you mind opening an issue on Node.js if you agree?

Done: https://github.com/nodejs/node/issues/11422


RocksDB crash: ~/.mina-config/trust/LOCK: Resource temporarily unavailable

Hi team!
I just tap enter during the daemon was running...
See attached the report
coda_crash_report_2021-04-20_04-00-59.390114.tar.gz

Btw, I also installed Promotheus + Grafana during.. maybe it impacted something

{
  "timestamp": "2021-04-20 04:00:59.385023Z",
  "level": "Fatal",
  "source": {
    "module": "Init__Coda_run",
    "location": "File \"src/app/cli/src/init/coda_run.ml\", line 603, characters 2-26"
  },
  "message": "Unhandled top-level exception: $exn\nGenerating crash report",
  "metadata": {
    "exn": {
      "commit_id": "a42bdeef6b0c15ee34616e4df76c882b0c5c7c2a",
      "sexp": [
        "monitor.ml.Error",
        [
          "Failure",
          "IO error: While lock file: /home/boquet_mathieu/.mina-config/trust/LOCK: Resource temporarily unavailable"
        ],
        [
          "Raised at file \"stdlib.ml\", line 33, characters 17-33",
          "Called from file \"src/lib/rocksdb/database.ml\", line 16, characters 34-63",
          "Called from file \"src/lib/trust_system/peer_trust.ml\", line 106, characters 15-33",
          "Called from file \"src/app/cli/src/mina.ml\", line 922, characters 25-54",
          "Called from file \"src/deferred0.ml\", line 56, characters 64-69",
          "Called from file \"src/job_queue.ml\" (inlined), line 131, characters 2-5",
          "Called from file \"src/job_queue.ml\", line 171, characters 6-47",
          "Caught by monitor coda"
        ]
      ],
      "backtrace": [
        "Raised at file \"stdlib.ml\", line 33, characters 17-33",
        "Called from file \"src/lib/rocksdb/database.ml\", line 16, characters 34-63",
        "Called from file \"src/lib/trust_system/peer_trust.ml\", line 106, characters 15-33",
        "Called from file \"src/app/cli/src/mina.ml\", line 922, characters 25-54",
        "Called from file \"src/deferred0.ml\", line 56, characters 64-69",
        "Called from file \"src/job_queue.ml\" (inlined), line 131, characters 2-5",
        "Called from file \"src/job_queue.ml\", line 171, characters 6-47"
      ]
    },
    "pid": 10809
  }
}

This seems to be a problem accessing the trust system database. Can you try deleting the lockfile at ~/.mina-config/trust/LOCK?
If it works, we should consider doing this automatically.

had same issue. Solved by deleting the mina-config folder
rm -R path/.mina-config

This is ok just by removing the LOCK file

Closing this one because of no activity for a long time and I hope that the most recent versions of Mina Daemon behave more stable for you.
Please feel free to report new issues should you face any.
Thank you.


Added a disclaimer that we are experiencing issues with location API

We are currently experiencing issues with the Location API on desktop. I've added a disclaimer to the device permissions documentation for developers. We are working on a fix.

Docs Build status updates of commit d6a6782:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/tabs/how-to/native-device-permissions.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 539ba38:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/tabs/how-to/native-device-permissions.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Closes #1473.


Travis CI build is failing

See https://travis-ci.org/aheui/naheui/builds/72676825.
Actually this problem occurred by https://github.com/aheui/snippets/commit/1de473c1475f4d4abe073783e0d4d00585c16b27, as "../cli.js --interactive-message=false" has being treated as a name of single file.

fixed test script. but bieup test is failing.
maybe because of https://twitter.com/x_nuk/status/625326778005860353

fixed by https://github.com/aheui/naheui/pull/7


Round out data/list

The git history is sort of messy for this… could you try and clean it up a little? You almost certainly want to be git rebaseing to pull in new changes, not git mergeing or git pulling, since the latter two commands produce the spurious merge commits.
To clean things up, you might be able to just squash the commits together, but that might cause some weird diffs. It might be easiest to just git cherry-pick the non-merge commits onto a fresh branch. I’m not the most skilled with git in this particular scenario.
(If you can’t figure it out, don’t worry about it—I can do the history rewriting myself—but give it a try.)

Sorry about that. Thanks for the advice!


Share Link incorrectly gives path routed instead of subdomain routed URL.

OS: Windows
Version of IPFS Desktop: 0.32.0

Describe the bug
When you right click on a file in Files and choose "Share Link" you are given a URL like https://<host>/ipfs/<cid>.  This should be of the form https://<cid>.ipfs.<host> for security reasons.
To Reproduce
Steps to reproduce the behavior:

Go to Files tab.
Right click on any file.
Choose "Share Link"
Notice the link provided uses path routing.

Expected behavior
Subdomain routing is always used.
Additional context
Path routing is known to be insecure for websites that use cookies, local storage, etc.  This is well documented in the IPFS documentation and the documentation and security experts all recommend using subdomain routing whenever possible (which is almost always possible).  These share links are encouraging people to share URLs that are insecure by default, and we should instead be using subdomain by default.

@lidel Thank you for your detailed response.
I agree with your suggestion that "ipfs-webui should have something similar, with an extra check to decide which gateway to use based on the shared CID." I'll proceed in this direction.
Thanks for the guidance!

@lidel @SgtPooki

Are the labels ok?

I think the labels are decent, but they feel a little too similar. Maybe we need a disclaimer saying that certain content that doesn't need origin-isolation for the subdomain gateway input.

The strategy here is to always use the public subdomain gateway (dweb.link) and use the public path gateway (ipfs.io) as a fallback in case the CID v1 length is > 63 chars
Is this ok?

@lidel @SgtPooki
https://github.com/acul71/ipfs-webui-fork/pull/1

@acul71 are you using AI or some GUI tools? That PR makes no sense (it merges from main in this repo  to a branch in your repo, and changes described in PR description are not in git).
What you want to do is to open PR from your form into this repository.

@lidel
Sorry did submit too late in the night....
Thanks is this ok:
https://github.com/ipfs/ipfs-webui/pull/2255


peer code review

Hi Luis. This is an awesome job!.
Here are some suggestions...
Try using simpler mocking functions to test the functionality of said functions.
Consider exporting an array in your test file to mock the local storage and user inputs.

Hi Luis. Great job in implementing unit tests on your project. After reviewing the code, I found that it is always better to separate the functions and do mock tests on it, rather than using the functions from your original file itself. This would be a simpler way of implementing mock functions and test them.
Good Luck!!! Keep Going.

Thanks for the feedback guys! With your advice, we were able to make the testing work.


Json Deserialize struct error when step over deserialize statement

Issue Description
Json Deserialize error when step over deserialize statement
Steps to Reproduce
using System.Text.Json;
var _options = new JsonSerializerOptions()
{
    WriteIndented = true,
    IncludeFields = true
};
var teststruct = new TestStruct()
{
    str1 = "hello",
    str2 = "world",
    num = 1,
    data = new SubTestStruct()
    {
        number = 2,
        str3 = "bug",
    }
};
var testStructStr = JsonSerializer.Serialize(teststruct, _options);
Console.WriteLine(testStructStr);
var deTestStructStr = JsonSerializer.Deserialize<TestStruct>(testStructStr, _options);
Console.WriteLine(deTestStructStr.num);

struct TestStruct
{
    public string? str1;
    public string? str2;
    public int num;
    public SubTestStruct data;

}
struct SubTestStruct
{
    public int number;
    public string str3;
}

breakpoint at var deTestStructStr = JsonSerializer.Deserialize<TestStruct>(testStructStr, _options);
Step Over

when no breakpoint

Expected Behavior
Deserialize Consistent behavior
Actual Behavior
Deserialize error
Environment information
VSCode version: 1.80.1
C# Extension: 1.26.0
Mono Information
        OmniSharp using mono: 6.12.0
Dotnet Information
.NET SDK:
 Version:   7.0.203
 Commit:    5b005c19f5
运行时环境:
OS Name:     Mac OS X
OS Version:  13.4
OS Platform: Darwin
RID:         osx.13-x64
Base Path:   /usr/local/share/dotnet/sdk/7.0.203/
Host:
Version:      7.0.5
Architecture: x64
Commit:       8042d61b17
.NET SDKs installed:
7.0.203 [/usr/local/share/dotnet/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 7.0.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 7.0.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Other architectures found:
None
Environment variables:
Not set
global.json file:
Not found
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download

Visual Studio Code Extensions

Extension
Author
Version

csharp
ms-dotnettools
1.26.0

theme-dracula
dracula-theme
2.24.3

vscode-language-pack-zh-hans
MS-CEINTL
1.80.2023071209

Thanks, next question: can you try disabling AVX code generation in your app's process? Assuming you are configuring the debugger to launch using launch.json, you can do so with:
            "env": {
                "DOTNET_EnableAVX": "0"
            }

Thanks! In this case you are running into https://github.com/dotnet/runtime/issues/78991. Hopefully the .NET runtime will address it soon. In the meantime, I hope that work around will keep you unblocked.


feat: use nybbles crate

https://github.com/danipopes/nybbles

any way we can avoid the .0 in the crate?

Need a wrapper for Codec impl


Merge pull request #1 from GarageGames/development

-merge from origin -Fixed thread statics.

oOoops sry


memory leak

I met the isuue of memory leak,  when It run on second epoch, the  memory run out with memory size of 64G. anyone have ever been in the same situation ?

We did not meet the problem. Could you share more detail?

Thanks for you kindly reply. It shows that memory usage increased continuously, it will run out 64G when trained 2 epoches on coco Dataset, by the way, I trained the project with distribute lauch.

I Noticed that [COCO Mask API is leaking memory when used with Multithreading
(https://github.com/cocodataset/cocoapi/issues/202#top)


Set GitHub pages

From the main branch, update the repository description with the proper website link. Follow the link to check if everything is fine.

Set the GitHub page and checked that is working fine.


 The error I get is:
L:   1 | P:   1 |  TMP | Unrecoverable failure in Jinja templating:
                       | <sqlfluff.core.templaters.jinja.JinjaTemplater.process.<locals>.Undefined
                       | object at 0x1051cbd60> is not safely callable. Have you
                       | configured your variables?
                       | https://docs.sqlfluff.com/en/latest/configuration.html

Below one of the scripts where this is happening:
{%- set stuck_rows_in_a_row = 3 -%}
{%- set fcs_criteria_in_seconds = 30 -%}
{%- set low_fps = 20 -%}

WITH recording_started AS(SELECT * FROM {{ ref('fct_recording_started')}} WHERE NOT COALESCE(is_test_user, FALSE))

, recordings_stopped AS (SELECT * FROM {{ ref('fct_recording_stopped') }})

, archives AS(
    SELECT
        rso.archive_id,
        rsa.session_id,
        rso.stream_id,
        rso.recording_id,
        rso.studio_id,
        rso.recording_stopped_reason,
        rso.user_id IS NOT NULL AS is_user,
        rso.expected_duration_in_seconds,
        rsa.server_timestamp,
        rsa.recorder_version,
        rsa.audio_input_media_device_label,
        rsa.video_input_media_device_label,
        rsa.user_id,
        rsa.is_test_user,
        rsa.anonymous_id,
        rsa.studio_role,
        rsa.studio_owner_id,
        rsa.studio_owner_email,
        COALESCE(rsa.recording_type, rso.recording_type) AS recording_type,
        COALESCE(rsa.live_call_id, rso.live_call_id) AS live_call_id,
        COALESCE(rsa.client_id, rso.client_id) AS client_id,
        rsa.timestamp AS recording_started_timestamp,
        rso.timestamp AS recording_stopped_timestamp,
        rso.uploaded_at AS batch
    FROM recording_started rsa
    JOIN recordings_stopped rso
        ON rsa.archive_id = rso.archive_id
    WHERE rsa.source = 'new'
    )

, waiting_room AS(
    SELECT
        arc.archive_id,
        count(jwr.client_id) AS number_participants_in_waiting_room_during_recording
    FROM {{ ref('fct_joined_waiting_room') }} jwr
    JOIN archives arc
        ON arc.studio_id = jwr.studio_id
            AND jwr.timestamp BETWEEN arc.recording_started_timestamp AND arc.recording_stopped_timestamp
    WHERE jwr.studio_recording_status = 'recording'
    GROUP BY 1
)

, max_upload_progress AS(
    SELECT
        archive_id,
        MAX(upload_progress_uploaded_bytes) AS max_upload_progress_uploaded_bytes
    FROM {{ ref('fct_recording_progression_stream') }}
    GROUP BY 1
)

, sync_issues AS(
    SELECT
        s.archive_id,
        upload_progress_uploaded_bytes,
        CASE
        WHEN upload_progress_uploaded_bytes = 0 THEN 'beginning'
        WHEN upload_progress_uploaded_bytes = max_upload_progress_uploaded_bytes  THEN 'end'
        ELSE 'middle' END AS moment_flag,
        COUNT(1) AS n_rows,
        DATEDIFF('minute', MIN(timestamp), MAX(timestamp)) AS range_in_minutes
     FROM {{ ref('fct_recording_progression_stream') }} s
     LEFT JOIN max_upload_progress s2 ON s.archive_id = s2.archive_id
     GROUP BY 1, 2, 3
     HAVING n_rows >= {{ stuck_rows_in_a_row }}
        AND range_in_minutes >= {{ stuck_rows_in_a_row }}
)

, sync_issues_pivoted AS(
    SELECT
        archive_id,
        TRUE AS stuck, -- single value, any agg is ok
        BOOLOR_AGG(moment_flag='beginning') AS stuck_at_0, -- single value, any agg is ok
        BOOLOR_AGG(moment_flag='middle')  AS stuck_at_middle, -- having at least 1
        BOOLOR_AGG(moment_flag='end') AS stuck_at_end -- single value, any agg is ok
    FROM sync_issues
    GROUP BY 1
)

, streams_cadence_test AS(
    {{  test_event_cadence(
        model= ref('fct_recording_progression_stream'),
        grouping_column='archive_id', time_column='timestamp',
        date_part='minute', threshold=1) }}
)

, rec_prog_stream AS(
    SELECT
        rps.archive_id,
        COUNT(DISTINCT rps.archive_id || rps.timestamp) AS stream_count,
        BOOLOR_AGG(NULLIF(rps.frames_per_second, 0) < {{ low_fps }} ) AS any_stream_is_low_fps,
        BOOLAND_AGG(NULLIF(rps.frames_per_second, 0) < {{ low_fps }} ) AS all_stream_is_low_fps,
        IFF(any_stream_is_low_fps IS NOT NULL,
             DIV0(
                COUNT_IF(rps.frames_per_second < {{ low_fps }}),
                COUNT(1)
             ), NULL) AS proportion_stream_low_fps,
        IFF(any_stream_is_low_fps IS NOT NULL,
             DIV0(
                COUNT_IF(rps.frames_per_second = 0),
                COUNT(1)
             ), NULL) AS proportion_stream_fps_equal_zero,
        ANY_VALUE(recording_resolution) AS recording_resolution,
        BOOLOR_AGG(e.archive_id IS NOT NULL) AS archive_has_stream_w_cadence_problem,
        VAR_POP(upload_speed) AS var_uploading_speed,
        MIN(upload_speed) AS min_uploading_speed,
        MAX(upload_speed) AS max_uploading_speed,
        VAR_POP(download_speed) AS var_downloading_speed,
        MIN(download_speed) AS min_downloading_speed,
        MAX(download_speed) AS max_downloading_speed,
        VAR_POP(frames_per_second) AS var_frames_per_second,
        STDDEV_POP(frames_per_second) AS stddev_frames_per_second,
        MIN(frames_per_second) AS min_frames_per_second,
        AVG(frames_per_second) AS avg_frames_per_second,
        MAX(upload_progress_uploaded_bytes) AS max_upload_progress_uploaded_bytes,
        MAX(upload_progress_upload_percentage) AS max_upload_progress_upload_percentage,
        ANY_VALUE(IFNULL(stuck, FALSE)) AS got_stuck,
        ANY_VALUE(IFNULL(stuck_at_0, FALSE)) AS got_stuck_at_zero,
        ANY_VALUE(IFNULL(stuck_at_middle, FALSE)) AS got_stuck_at_middle,
        ANY_VALUE(IFNULL(stuck_at_end, FALSE)) AS got_stuck_at_end
    FROM {{ ref('fct_recording_progression_stream') }} rps
    LEFT JOIN sync_issues_pivoted sip
        ON sip.archive_id = rps.archive_id
    LEFT JOIN streams_cadence_test e
        ON e.archive_id = rps.archive_id AND e.timestamp = rps.timestamp
    GROUP BY rps.archive_id
)

, errors_joined AS(
    SELECT
        arc.archive_id,
        {{ pivot(
            'error_type',
            dbt_utils.get_column_values(table= ref('fct_error_notifications'), column= 'error_type'
                , where= "error_type IN ('video_muted_error', 'storage_full')"),
            agg= 'sum',
            quote_identifiers= False,
            bool= True,
            prefix='has_'
        ) }}
    FROM {{ ref('fct_error_notifications') }} e
    JOIN archives arc
        ON e.anonymous_id = arc.anonymous_id
            AND e.timestamp BETWEEN arc.recording_started_timestamp AND arc.recording_stopped_timestamp
    GROUP BY arc.archive_id
)

, live_calls_features AS( SELECT * FROM {{ ref('live_calls_features') }})

, record_media_error AS( SELECT * FROM {{ ref('fct_record_media_error_pivoted') }})

, recordings AS (
    SELECT
        r.*,
        a.is_enterprise -- asessing whether the studio where the recording was created belongs to a currently enterprise account
    FROM {{ ref('dim_recordings') }} r
    LEFT JOIN {{ ref('dim_studios') }} s
        ON r.studio_id = s.studio_id
    LEFT JOIN {{ ref('dim_productions') }} p
        ON s.production_id = p.production_id
    LEFT JOIN {{ ref('dim_accounts') }} a
        ON p.account_id = a.account_id
    WHERE file_type = 'local'
)

, screen_share AS (SELECT DISTINCT session_id FROM {{ ref('dim_recordings') }} WHERE source = 'mediaboard-screenshare')

, identity AS (SELECT * FROM {{ ref('segment__identity') }} )

, studios AS (SELECT * FROM {{ ref('dim_studios') }} )

, users AS (SELECT * FROM {{ ref('dim_users') }} )

, final AS(
    SELECT
        s.archive_id,
        s.session_id,
        s.stream_id,
        s.live_call_id,
        s.recording_id,
        s.studio_id,
        s.client_id,
        s.user_id,
        s.user_id IS NOT NULL AS is_user,
        r.is_enterprise,
        s.recording_started_timestamp,
        s.recording_stopped_timestamp,
        s.recording_type,
        s.recording_stopped_reason,
        s.expected_duration_in_seconds,
        s.studio_role,
        r.duration_in_seconds,
        r.duration_in_seconds = 0 AS duration_is_zero,
        TRUNCATE(s.expected_duration_in_seconds - r.duration_in_seconds, 0) AS duration_difference,
        r.client_status,
        s.server_timestamp,
        s.recorder_version,
        SPLIT_PART(s.recorder_version, '.', 1) AS recorder_major_version,
        SPLIT_PART(s.recorder_version, '.', 2) AS recorder_minor_version,
        r.filename,
        COUNT(s.archive_id) OVER(PARTITION BY s.session_id, s.client_id) = 1 AS is_only_archive_for_client_id_in_session,
        COALESCE(lcf.mic_connected_during_recording, FALSE) AS mic_connected_during_recording,
        COALESCE(lcf.camera_connected_during_recording, FALSE) AS camera_connected_during_recording,
        COALESCE(lcf.mic_disconnected_during_recording, FALSE) AS mic_disconnected_during_recording,
        COALESCE(lcf.camera_disconnected_during_recording, FALSE) AS camera_disconnected_during_recording,
        -- ads.mic_connected_actually_set_during_recording,
        -- ads.camera_connected_actually_set_during_recording,
        lcf.number_of_initial_mics_connected,
        lcf.number_of_initial_cameras_connected,
        wr.number_participants_in_waiting_room_during_recording,
        lcf.avg_mic_latency,
        lcf.var_mic_latency,
        lcf.mic_permission_changed,
        lcf.camera_permission_changed,
        lcf.initial_echo_cancellations_settings,
        lcf.initial_operating_system AS operating_system,
        lcf.initial_browser_type_version,
        lcf.initial_graphic_card AS graphic_card,
        lcf.initial_gpu_vendor AS gpu_vendor,
        lcf.initial_device_permission_camera AS device_permission_camera,
        lcf.initial_device_permission_mic AS device_permission_mic,
        lcf.initial_hardware_concurrency AS hardware_concurrency,
        lcf.initial_device_permission_notifactions AS device_permission_notifactions,
        lcf.min_charging_level_during_recorded,
        lcf.max_charging_level_during_recorded,
        lcf.has_client_charged_during_recording,
        lcf.var_perc_storage_space_available,
        lcf.min_perc_storage_space_available,
        lcf.max_storage_used,
        lcf.min_storage_used,
        lcf.var_used_storage_space,
        lcf.var_available_storage_space,
        lcf.storage_range,
        lcf.max_storage_bins,
        lcf.min_storage_bins,
        lcf.range_storage_bins,
        lcf.initial_storage_estimate_js_storage_usage,
        lcf.initial_perc_storage_space_available,
        lcf.initial_storage_bins,
        rps.stream_count,
        rps.proportion_stream_low_fps,
        rps.proportion_stream_fps_equal_zero,
        rps.any_stream_is_low_fps,
        rps.all_stream_is_low_fps,
        rps.recording_resolution,
        rps.var_uploading_speed,
        rps.min_uploading_speed,
        rps.max_uploading_speed,
        rps.var_downloading_speed,
        rps.min_downloading_speed,
        rps.max_downloading_speed,
        rps.var_frames_per_second,
        rps.stddev_frames_per_second,
        rps.min_frames_per_second,
        rps.avg_frames_per_second,
        rps.max_upload_progress_uploaded_bytes,
        rps.max_upload_progress_upload_percentage,
        rps.got_stuck,
        rps.got_stuck_at_zero,
        rps.got_stuck_at_middle,
        rps.got_stuck_at_end,
        INITCAP(TRIM(REGEXP_REPLACE(s.audio_input_media_device_label, '\\(....:....\\)'))) AS audio_input_media_device_label,
        INITCAP(TRIM(REGEXP_REPLACE(s.video_input_media_device_label, '\\(....:....\\)'))) AS video_input_media_device_label,
        {{ star(ref('fct_record_media_error_pivoted'), relation_alias='rme', except=['archive_id'], prefix= 'media_error_') }},
        ej.has_video_muted_error,
        ej.has_storage_full AS has_storage_full_error,
        r.is_first_recording,
        r.auto_gain_control_enabled,
        r.echo_cancelation_enabled,
        r.noise_suppression_enabled,
        r.persistence,
        r.user_feature_flags_auto_adapt_encoder,
        r.user_feature_flags_auto_adapt_resolution,
        r.user_feature_flags_safe_media_stream,
        r.user_feature_flags_snapshots,
        r.user_feature_flags_web_recorder_raw_parts,
        r.user_feature_flags_single_stream,
        ss.session_id IS NOT NULL AS session_has_screen_shared,
        s.studio_owner_id,
        s.studio_owner_email,
        rps.archive_id IS NULL AS has_no_associated_recording_progression_stream_event,
        rps.archive_has_stream_w_cadence_problem AS has_stream_w_cadence_problem,
        r.archive_id IS NULL AS has_no_associated_mongo_recording,
        lcf.live_call_id IS NULL AS has_no_associated_live_call_data,
        r.duration_in_seconds AS recording_duration_in_seconds,
        r.shift_estimation,
        r.git_version_tag,
        HOUR(s.recording_started_timestamp) AS recording_started_hour_of_the_day,
        ABS(TRUNCATE(s.expected_duration_in_seconds - r.duration_in_seconds, 0)) AS abs_duration_diff,
        CASE
            WHEN abs_duration_diff < 2 THEN 'A. <2 Seconds'
            WHEN abs_duration_diff <= 5 THEN 'B. 2-5 Seconds'
            WHEN abs_duration_diff <= 10 THEN 'C. 6-10 Seconds'
            WHEN abs_duration_diff <= 30 THEN 'D. 11-30 Seconds'
            WHEN abs_duration_diff <= 60 THEN 'E. 31-60 Seconds'
            WHEN abs_duration_diff < 60*3 THEN 'F. 1-2 Minutes'
            WHEN abs_duration_diff < 60*6 THEN 'G. 3-5 Minutes'
            WHEN abs_duration_diff < 60*11 THEN 'H. 6-10 Minutes'
            WHEN abs_duration_diff < 60*31 THEN 'I. 11-30 Minutes'
            WHEN abs_duration_diff < 60*61 THEN 'J. 31-60 Minutes'
            ELSE 'K. >60 Minutes'
        END AS time_difference_bucket,
        (abs_duration_diff > {{ fcs_criteria_in_seconds }}) AND NOT duration_is_zero AS file_cut_short,
        BOOLOR_AGG((client_status = 'uploaded' OR client_status IS NULL) AND file_cut_short) OVER(PARTITION BY s.session_id) AS session_has_file_cut_short,
        CASE
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 0 AND HOUR(s.recording_started_timestamp) BETWEEN 0 AND 6 THEN 'Sun 1'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 0 AND HOUR(s.recording_started_timestamp) BETWEEN 6 AND 12 THEN 'Sun 2'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 0 AND HOUR(s.recording_started_timestamp) BETWEEN 12 AND 18 THEN 'Sun 3'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 0 AND HOUR(s.recording_started_timestamp) BETWEEN 18 AND 24 THEN 'Sun 4'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 1 AND HOUR(s.recording_started_timestamp) BETWEEN 0 AND 6 THEN 'Mon 1'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 1 AND HOUR(s.recording_started_timestamp) BETWEEN 6 AND 12  THEN 'Mon 2'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 1 AND HOUR(s.recording_started_timestamp) BETWEEN 12 AND 18 THEN 'Mon 3'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 1 AND HOUR(s.recording_started_timestamp) BETWEEN 18 AND 24 THEN 'Mon 4'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 2 AND HOUR(s.recording_started_timestamp) BETWEEN 0 AND 6 THEN 'Tue 1'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 2 AND HOUR(s.recording_started_timestamp) BETWEEN 6 AND 12 THEN 'Tue 2'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 2 AND HOUR(s.recording_started_timestamp) BETWEEN 12 AND 18 THEN 'Tue 3'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 2 AND HOUR(s.recording_started_timestamp) BETWEEN 18 AND 24 THEN 'Tue 4'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 3 AND HOUR(s.recording_started_timestamp) BETWEEN 0 AND 6 THEN 'Wed 1'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 3 AND HOUR(s.recording_started_timestamp) BETWEEN 6 AND 12 THEN 'Wed 2'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 3 AND HOUR(s.recording_started_timestamp) BETWEEN 12 AND 18 THEN 'Wed 3'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 3 AND HOUR(s.recording_started_timestamp) BETWEEN 18 AND 24 THEN 'Wed 4'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 4 AND HOUR(s.recording_started_timestamp) BETWEEN 0 AND 6 THEN 'Thu 1'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 4 AND HOUR(s.recording_started_timestamp) BETWEEN 6 AND 12 THEN 'Thu 2'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 4 AND HOUR(s.recording_started_timestamp) BETWEEN 12 AND 18 THEN 'Thu 3'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 4 AND HOUR(s.recording_started_timestamp) BETWEEN 18 AND 24 THEN 'Thu 4'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 5 AND HOUR(s.recording_started_timestamp) BETWEEN 0 AND 6 THEN 'Fri 1'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 5 AND HOUR(s.recording_started_timestamp) BETWEEN 6 AND 12 THEN 'Fri 2'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 5 AND HOUR(s.recording_started_timestamp) BETWEEN 12 AND 18 THEN 'Fri 3'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 5 AND HOUR(s.recording_started_timestamp) BETWEEN 18 AND 24 THEN 'Fri 4'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 6 AND HOUR(s.recording_started_timestamp) BETWEEN 0 AND 6 THEN 'Sat 1'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 6 AND HOUR(s.recording_started_timestamp) BETWEEN 6 AND 12 THEN 'Sat 2'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 6 AND HOUR(s.recording_started_timestamp) BETWEEN 12 AND 18 THEN 'Sat 3'
            WHEN DAYOFWEEK(s.recording_started_timestamp) = 6 AND HOUR(s.recording_started_timestamp) BETWEEN 18 AND 24 THEN 'Sat 4'
            ELSE 'NA'
        END AS day_and_part_of_the_day,
        s.batch
    FROM archives s
    LEFT JOIN live_calls_features lcf
        ON s.live_call_id = lcf.live_call_id
            AND s.client_id = lcf.client_id
    LEFT JOIN waiting_room wr
        ON wr.archive_id = s.archive_id
    LEFT JOIN rec_prog_stream rps
        ON rps.archive_id = s.archive_id
    LEFT JOIN record_media_error rme
        ON rme.archive_id = s.archive_id
    LEFT JOIN errors_joined ej
        ON ej.archive_id = s.archive_id
    LEFT JOIN recordings r
        ON r.archive_id = s.archive_id
    LEFT JOIN screen_share ss
        ON ss.session_id = s.session_id
        -- WHERE (r.client_status = 'uploaded'
        --     OR r.client_status IS NULL)
)

SELECT *
FROM final
{{  deduplicate (key='archive_id',
    time_column='recording_started_timestamp', criteria='first') }}

@cherns31 Do you also get an error when you run the sqlfluff lint command in the terminal? You can test by looking at the Output tab and checking the SQLFluff channel. That should show you the exact sqlfluff lint command being run. Copy that and paste it in the terminal. If it ends with a - you may have to replace that with the filepath.

Basically just trying to figure out if it is a problem with my extension or SQLFluff in general.

This is a bug in SQLFluff.
@cherns31 , can you please move this issue to the main SQLFluff repo?


Store the path for referrers with home directories

Until now, the webring always normalized referrers to the hostname, with no path (like tilde.club). Here's a case where that breaks down.
A classic method of user management for shared servers like http://tilde.club/ is to store each user's content under a top-level path like /~gabe.
Gabe would be at tilde.club/~gabe, Edward would be at tilde.club/~edward, and so on. One person curates /~gabe, another person curates /~edward, and a third person curates the tilde.club homepage itself.
In the case of shared servers, when someone adds webring links to a page on (e.g.) tilde.club/~gabe/webring.html, the webring stores tilde.club. We should instead respect each user's individuality and store that site's URL as tilde.club/~gabe.
Fixes #44.

Been thinking about this, sorry I haven't responded. What if we just use the full URL instead of hostname or hostname/~username? The redirection is created when the user clicks on the link. So for example, in your test, if the referrer is http://tilde.club/~ford/something.html, then that means the link is on that page. If redirect to http://tilde.club/~ford/, we're in the same boat as we are now.
This also makes it possible to have anyone use any URL. Is there a reason we used only hostnames in the past? Normalization?

So for example, in your test, if the referrer is http://tilde.club/~ford/something.html, then that means the link is on that page. If redirect to http://tilde.club/~ford/, we're in the same boat as we are now.

Well, sort of. I think that, based on the naming convention, /~ford is likely to be a hub of ford's activity and something.html should be accessible from there.

What if we just use the full URL?  Is there a reason we used only hostnames in the past? Normalization?

Yes, I think we were worried about double entries for a site, both malicious and unintentional. For example, I have webring links on every blog post on gabebw.com, but I don't want 1 webring entry per blog post. Normalizing to the host prevents that.

I should say that I'm definitely not against just using the raw referer. But I can't think of anything offhand that solves my blog-post-webring-link problem.

Ah yes, thank you for reminding me. Another solution could be redirects to any arbitrary domain but limited one per hostname. Also not a silver bullet.
What if we just put this one in manually? It's not as if we're getting 100 requests a day or something. We could just deal with it on a case by case basis.

What if we just put this one in manually?

I did actually change it manually: https://hotlinewebring.club/

We could just deal with it on a case by case basis.

I think that's the best solution here.


Unable to provision private AKS Cluster

Rancher Server Setup

Rancher version: 2.6.3
Installation option (Docker install/Helm Chart): Docker install with --debug

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc):

Proxy/Cert Details:

Information about the Cluster

Kubernetes version: 1.21.9
Cluster Type (Local/Downstream):

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider): 3 node downstream private AKS Cluster, provisioned through the Rancher UI

User Information

What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom) admin

If custom, define the set of permissions:

Describe the bug

I am unable to successfully provision a private AKS cluster using the Rancher UI on Rancher 2.6.3. I tried three times on a fresh docker install.
Try 1

Try 2 - appears cluster was in a failed state in Azure

Try 3 - appears now that Azure never created the nodes at all. There was no evidence of this cluster in Azure portal

To Reproduce

Run Rancher 2.6.3 in --debug mode on a docker install
Login as admin
Add Azure cloud credentials
Create a 3 node, private downstream cluster using the Rancher UI. Save the import commands. (normally here, I would run the import command on a bastion VM in the same vnet as the downstream cluster to import the private AKS Cluster into Rancher. But, I was not able to reach that point)

Result
Unable to provision private AKS Cluster.
Expected Result

Private cluster provisions successfully in AKS and is ready for import.

Seeing the same thing as Try 3 with Waiting for API to be available message on rancher 2.6-head with kubernetes 1.22.6. Same repro steps.
Docker logs
2022/04/19 15:37:38 [DEBUG] could not get kubeconfig for cluster c-nbl6w
2022/04/19 15:37:38 [INFO] waiting for cluster AKS [c-nbl6w] to update
2022/04/19 15:37:38 [DEBUG] DesiredSet - No change(2) provisioning.cattle.io/v1, Kind=Cluster fleet-default/c-nbl6w for provisioning-cluster-create c-nbl6w
2022/04/19 15:37:43 [DEBUG] AKS cluster [c-nbl6w] will be managed by aks-operator-controller, skipping update
2022/04/19 15:37:43 [DEBUG] could not get kubeconfig for cluster c-nbl6w
2022/04/19 15:37:43 [INFO] waiting for cluster AKS [c-nbl6w] to update
2022/04/19 15:37:43 [DEBUG] DesiredSet - No change(2) provisioning.cattle.io/v1, Kind=Cluster fleet-default/c-nbl6w for provisioning-cluster-create c-nbl6w
2022/04/19 15:37:48 [DEBUG] AKS cluster [c-nbl6w] will be managed by aks-operator-controller, skipping update
2022/04/19 15:37:48 [DEBUG] could not get kubeconfig for cluster c-nbl6w
2022/04/19 15:37:48 [INFO] waiting for cluster AKS [c-nbl6w] to update
2022/04/19 15:37:48 [DEBUG] DesiredSet - No change(2) provisioning.cattle.io/v1, Kind=Cluster fleet-default/c-nbl6w for provisioning-cluster-create c-nbl6w
2022/04/19 15:37:53 [DEBUG] AKS cluster [c-nbl6w] will be managed by aks-operator-controller, skipping update
2022/04/19 15:37:53 [DEBUG] could not get kubeconfig for cluster c-nbl6w
2022/04/19 15:37:53 [INFO] waiting for cluster AKS [c-nbl6w] to update
2022/04/19 15:37:53 [DEBUG] DesiredSet - No change(2) provisioning.cattle.io/v1, Kind=Cluster fleet-default/c-nbl6w for provisioning-cluster-create c-nbl6w
2022/04/19 15:37:58 [DEBUG] AKS cluster [c-nbl6w] will be managed by aks-operator-controller, skipping update
2022/04/19 15:37:58 [DEBUG] could not get kubeconfig for cluster c-nbl6w
2022/04/19 15:37:58 [INFO] waiting for cluster AKS [c-nbl6w] to update
2022/04/19 15:37:58 [DEBUG] DesiredSet - No change(2) provisioning.cattle.io/v1, Kind=Cluster fleet-default/c-nbl6w for provisioning-cluster-create c-nbl6w

In order to consolidate the issues we are working on, the issues referenced here will be fixed as a part of https://github.com/rancher/rancher/issues/37164. Closing as duplicate.


Scanning network for connection

Hello!
I created a websocket server using Python.
I want to make my application to scan the network and find the server IP. Without manually hardcoding it.
Do you have any solution to this? Thanks.

IP is not enough, you also need to find the port. It seems quite odd and is completely irrelevant for this library.
I suggest closing this issue.

@AngCosmin like @thbaja mentioned this is out of the scope for this library. If you make it make it open source as it sounds useful for other folks.


Sphinx integration

Related ticket: MVP-4568

Codecov Report
All modified lines are covered by tests :white_check_mark:

Comparison is base (045ca10) 99.72% compared to head (2602b90) 99.72%.

:exclamation: Current head 2602b90 differs from pull request most recent head 8b65354. Consider uploading reports for the commit 8b65354 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #41   +/-   ##
=======================================
  Coverage   99.72%   99.72%
=======================================
  Files          11       11
  Lines         359      359
=======================================
  Hits          358      358
  Misses          1        1

Files
Coverage Δ

pandasaurus/utils/pandasaurus_exceptions.py
100.00% <ø> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fix SBC implementation and improve overflow detection

Correct decimal mode subtraction in SBC operation
Implement accurate overflow detection for non-decimal mode
Fix carry flag calculation as inverse of borrow
Add comments explaining complex bit manipulation for overflow detection
Update tests to verify correct behavior in both decimal and non-decimal modes

This commit addresses issues with the Subtract with Carry (SBC) operation,
ensuring correct behavior in both decimal and non-decimal modes, and
improves code readability with added comments.
Fixes #105

Do you know why the corresponding fix is not necessary in the implementation of ADC? The bug is not present there for some reason?
Does this PR supercede #96 ?
(I also happen to know that for reasons having to do with false positives, the ADC and SBC are underrepresented by solid65, maybe that's how the bug crept in to this crate)

I'm happy with this, good job

Do you know why the corresponding fix is not necessary in the implementation of ADC? The bug is not present there for some reason?
Does this PR supercede #96 ?
(I also happen to know that for reasons having to do with false positives, the ADC and SBC are underrepresented by solid65, maybe that's how the bug crept in to this crate)

Oh, I just made the minimal change for now to see if I correctly understood the issue. The ADC is probably broken, too. We can tackle that next.
I think we can rebase #96 on top of master to see where we're at once this is merged.

Merged. Thanks for the quick review.


Unknown process name ????

[Unknown process name] CGDataProviderCreateDirect: Invalid size = 0
[Unknown process name] CGImageCreate: invalid image provider: NULL.
[Unknown process name] CGDataProviderCreateDirect: Invalid size = 0
[Unknown process name] CGImageCreate: invalid image provider: NULL.

I am also getting same issue. Is there any update on this?


Get mesh to pubchem compound mappings

PubChem provides PubChem Compound identifiers to MeSH terms as a 2 column TSV at ftp://ftp.ncbi.nlm.nih.gov/pubchem/Compound/Monthly/2020-04-01/Extras/CID-MeSH (not a typo, they really don't give this file an extension).
Note that many (112K of 120K) of the mappings correspond to MeSH supplementary records (that start with a C)

I've exploited lexical ambiguity to find mappings between MeSH and ChEBI in Gilda, some of those mappings would trivially map further to PubChem from ChEBI. But I guess a set of MeSH<->PubChem mappings could also be obtained using the same approach in cases where a ChEBI intermediate isn't available.

There are 120K direct mappings available from pubchem. sorry this wasn't clear. I updated the description


Unique checks

for all associations

https://github.com/HPI-SWA-Lab/BP2016H1/commit/331e171af4f5135353c01a168a8a71550f647a1e


Parse percentage as float

Woops sorry - wasn't meant to land in here 😉
But would be nice maybe in the future if you could still parse the percentage as floats.


pass function into Header or Custom Navigators in react native web?

Hi,
In native react navigation, we can use navigation.setParams to pass our functions into Header (by using  navigationOptions and navigation.getParam as described in this)
But on the web, setParams does not work properly and serialize function into query-strings and urls get very messy.
Is there any better way to pass function into Header or Custom Navigators in react native web?

@steventilator A better way to handle this is to have the constructor or componentDidMount assign the function at a file level scoped variable that then would get used in your navigationOptions callback. Would have to have a dummy function assigned or check for undefined/null in case navigationOptions gets called before componentDidMount/constructor.


Unable to run

I'm having an issue that seems similar to #6. The prepare script no longer exists, though, so I couldn't use the same resolution.
Here's what I get when I try to run the run.sh script:
ava-sim % ./scripts/run.sh
go: downloading github.com/fatih/color v1.9.0
......
go: downloading google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71
go: downloading golang.org/x/text v0.3.6
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
tmp dir located at: /var/folders/5s/ftvk5y7s4wn4_nclslwn3h_40000gp/T/ava-sim1183226240
tmp dir located at: /var/folders/5s/ftvk5y7s4wn4_nclslwn3h_40000gp/T/ava-sim1183226240
panic: open build/system-plugins/evm: no such file or directory

goroutine 7 [running]:
github.com/ava-labs/ava-sim/manager.StartNetwork({0x2185718, 0xc00047ab80}, {0x0, 0x0}, 0xc000c9ed80)
	path/to/ava-sim/manager/manager.go:95 +0xb52
main.main.func2()
	path/to/ava-sim/main/main.go:63 +0x2d
golang.org/x/sync/errgroup.(*Group).Go.func1()
	path/to/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	path/to/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x8d
exit status 2

I'm running this on a pre-M1 Macbook Pro.

Whoops, I forgot to build. But now I'm having an issue with building:
ava-sim % echo $GOPATH
/path/to/go
ava-sim % ./scripts/build.sh
Building Coreth @ v0.8.1-rc.0 ...
./scripts/build.sh: line 14: GOPATH: unbound variable

Fixed - I defined GOPATH incorrectly, somehow


Add support for older Laravel versions with Laravel 7 support

Laravel 7.0 bug found by and fixed by @drjdr
Added support for older Laravel versions after symfony changes in UploadedFile

Tests passed on all version.


feat: add quaternary color for selected dropdowns

Part of the color contrast epic. During development this will live on the feature/color-contrast branch.
If you want to test this locally, there are also some changes in scratch-gui on the feature/color-contrast branch, commit 8533704. This would involve building scratch-blocks and using npm link scratch-blocks on scratch-gui.
Resolves

Resolves ENA-244

Proposed Changes

Add a quaternary color to blocks
Use the quaternary color for styling selected dropdown menus (core/field_dropdown.js)
Add a menuHover color property for hovered menu items

Reason for Changes
The design plans for the high contrast mode have a lighter color for selected dropdown menus. This is a fourth color that does not match the primary, secondary, or tertiary colors. Perhaps this fourth color will be used for other theming in the future.

Test Coverage
This is mostly a styling change which the existing automated tests are not testing. Used scratch-gui to see that the styles look correct for the high contrast theme.

Added test to verify jsonInit sets all 4 colors and makes the source block's colors available to the dropdown.

Something interesting to note about this design is that previously, a selected dropdown's fill matched the stroke color of the block border (i.e. the tertiary color), which is also used for the input border:

In this implementation, since the quaternary color is introduced, that fill is different from the block border - and thus distinct from the border of the input as well! (Can't run locally ATM but see main demo screenshot.)
Folk viewing the repo from outside (👋) don't have access to the design documents, but it wouldn't surprise me if this is an intentional change. It almost reflects focus rings in standard browser/UI design. (It's not quite the same, because the stroke itself doesn't actually become bolder or darker when selected, but it does come close by increasing the contrast between input fill and stroke.)
Just thought I'd point out the parallel with existing accessible UI design! The color contrast epic looks quite epic ^_^

Thanks for pointing out this color not matching the border color @towerofnix. This design is based off of the work in https://lab.scratch.mit.edu/contrast-blocks/. I believe it is intentional, but I will double-check with the designers next time I am talking to them.


fix(fetch): do not optimistically rely on "location" in fetch interceptor

Related: mswjs/msw#1555

@kettanaito works for me. I tend to reach for boring or more verbose solutions, so I probably wouldn't do it like that, but your code/project/rules. 👍👍👍

Thanks for fixing this, @till! Welcome to contributors. I will release this change from that intermediate branch today.

Thanks for your help!


Suggest to move cancellation steps to a separate substory.

This example harms readability of the main story.
We advise users to move each method with rollbacks into a separate substory.
class Action:

    @story
    def do(I):
        I.one
        I.two
        I.three

    @story
    def one(I):
        with I.first:
            ~I.cancel_first

    @story
    def two(I):
        with I.second:
            ~I.cancel_second_a
            ~I.cancel_second_b

Superseded by #627


cricfree.live

@adguard-bot commented on Wed Jul 31 2019
Issue URL (Ads)
http://cricfree.live/watchfeeds/demand/12
Comment
Username: @TesterMan247

--
The information below is automatically generated. Do not delete it.
Custom userscripts:
https://userscripts.adtidy.org/beta/assistant/4.3/assistant.user.js
https://userscripts.adtidy.org/beta/popup-blocker/2.5/popupblocker.user.js
https://userscripts.adtidy.org/beta/adguard-extra/1.0/adguard-extra.user.js

Screenshots

Screenshot 1

System configuration

Information
value

Platform:
Win

AdGuard version:
7.1.2872.7

Browser:
Chrome

AdGuard driver:
WFP

Filters:
AdGuard Annoyances,AdGuard Base,NoCoin Filter List,AdGuard Tracking Protection,AdGuard Experimental

It seems that AdGuard Browsing Security module causes that AdGuard`s JS is not injected to the page.
Steps to reproduce

Enable Browsing Security
Add this rule:
webtv.ws##body
Go here - http://webtv.ws/
Website is blocked by Browsing security, so click on "Proceed anyway" button

Screenshot

Expected behavior

Websie should be blank.
Actual behavior

AdGuard`s JS is not injected to the page.
Screenshot

Now disable Browsing Security, clear site data and refresh website

Screenshot:

Your environment

AdGuard for Windows Nightly 7.1.2872 (CL 1.4.120), happens also with stable build


type: name 'unicode' is not defined

May 2022 at 10:22:53
Occupatum Error                 Error in plugin execution GetUiDynamicList:

File "plugin.py", line 298, in sensorDevices
type: name 'unicode' is not defined

Fixed in 2022.0.2


How to retry when response code is 409

Could you please let me know how to retry when an HTTP response code is 409 Conflict with @azure/core-pipeline-rest package?
As far as reading the code, the package seems to use 408, 500, 502-504, 506 or more as response codes that are need to retry.
https://github.com/Azure/azure-sdk-for-js/blob/82996230773ab8295f06b17a8b6f449f9d9f2a8c/sdk/core/core-rest-pipeline/src/retryStrategies/exponentialRetryStrategy.ts#L81-L94
Or please add 409 to the retry target status code.
Background:
We are trying to develop a GitHub actions aca-review-apps that operates revision status of Azure Container Apps and uses @azure/core-pipeline-rest via @azure/arm-appcontainers.
If the operation by the action and the one from Azure portal are conflicted, then Azure platform returns HTTP 409 error as the action result. In this case we think it's needed to retry the operation.

I'm not very familiar with Container Apps, but I don't think 409 is a retryable response code from a HTTP perspective as it usually indicates there is a problem with the request itself (e.g. you are issuing a PUT to an old version.)
Are you saying that an exact replay of the same request to this service will succeed without any conflict resolution needed?
/cc @JeffreyRichter @johanste

Correct, normally a 409 is not retryable.

Thank you for your response, @xirzec @JeffreyRichter.
In this ACA case, the REST API for update(Container Apps - Update) returns 409 during update from other clients, but after finishing the update from others, the API with exact same request returns 202.
This is why I think 409 in this case is retryable.

If the PATCH operation is implemented as a long-running operation (which it sounds like it is), then retrying would potentially take a long time (by definition). For Azure data-plane operations, we forbid implementing PATCH as a long-running operation and instead require it be implemented as a POST-action LRO and if multiple clients issue this simultaneously, then the operations are queued and executed sequentially. So, no 409 would be returned.
I think the container apps should consider changing their pattern to this as the data-plane pattern is better. A 409 should not be considered retryable. Depending on the language, you might be able to get the SDK to retyr on a 409 but you wouldn't want this behavior for any other operation on the client object.

I understood that:

it's not appropreate for Azure operations to return 409 to PATCH request to update resources.
For long-running-operation, the requests should be POST requets, queued, and processed sequentially.

Is it right?

First, data-plane and control-plane do not always follow the same guidelines. Second, there are some HTTP standards that should be adhered to and some parts of the standard can be ambiguous.
A 409-Conflict indicates that the current state of the resource and the operation being attempted are in conflict with each other. There is no reason to think that the resource will dynamically change to resolve the conflict. Instead, it is more likely that the client request has to change to resolve the conflict and this is why retrying shouldn't make sense for a 409 and also why the service should not return a 409 in this case.
So this is what is wrong. Now, we can discuss potential ways to fix this. The simple "solution" is to allow retries for this specific operation should the service return a 409. This also assumes, that the 409 is always being returned for this specific reason and not ever returned for any other reason. I don't know if this assumption can be made for this specific service operation.
Our Azure data-plane guidelines, were designed taking this race condition into account and so if the service were to implement this feature using Azure data-plane guidance, this problem is averted. Perhaps this operation can change its implementation to follow the data-plane guidance but the control-plane team would have to get involved in this and changing the behavior is a breaking change and Azure's policy very strongly discourages breaking changes.
The data-plane guidance disallows LRO PATCH operations for many reasons and you're experiencing one of them now. The data-plane guidance to enable this feature is to use an LRO POST-action pattern. I do think the control-plane should adopt this guidance too.

Thank you @JeffreyRichter for your explanation.
Because both of retrying for 409 response under a specific situation only and changing control-plane interface might not be practical, it's better that an application (GitHub Action in my case) retries to request in current status.
Is it appropriate to add a custom retry policy/strategy for 409 response to pipeline if the retry logic should be implemeted on application side?

If you add retry on 409 to the pipeline (via client options) then this will apply to ALL operations on the client and this is not ideal.  It may OK for this service but we definitely wouldn't do this for all services. What a customer could do is use one client for all operations except this one and use another client for this 1 operation where the retry policy is configured to retry on 409.

Some language SDKs allow configuring the retry policy via client options. From the code you show above, it looks like JS does not support this. The JS SDK could add this feature and a customer could be instructed to add support for 409 to solve the problem in this issue. For example, our Go SDK lets the customer configure/override the HTTP status codes to retry on; see https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/azcore/policy/policy.go#L106
The JS SDK should add this feature too (if not already there).

If you add retry on 409 to the pipeline (via client options) then this will apply to ALL operations on the client and this is not ideal. It may OK for this service but we definitely wouldn't do this for all services.

Is it safe to add/remove the custom policy before/after a specific operation?

No, once a client is constructed, its options are immutable. This is by-design so that behavior of a client can't be changed by arbitrary code - the behavior is fixed over the client's lifetime. Also, in languages that support threads, this prevents race conditions which are near impossible to debug. You could create a new client before calling each operation but this will create a lot of garbage and cause more GCs.

Could you please let me confirm again?
Do you mean usage of the methods addPolicy and removePolicy in the following code is not valid?
    const client = new ContainerAppsAPIClient(credential, subscriptionId);
    const currentAppProperty = await client.containerApps.get(/* parameters */); // it doesn't need to retry for 409

    // :

    // add custom retry policy for 409 response on updating
    client.pipeline.addPolicy(customRetryPolicy({ maxRetries: 10 }), {phase: "Retry"});
    // update
    await client.containerApps.beginUpdateAndWait( /* parameters */   ); // it might return 409 and retry is needed
    // remove custom retry policy
    client.pipeline.removePolicy({name: customRetryPolicy.name, phase: "Retry"})

If this code works, I'd be surprised. In fact, I don't think that the pipeline field should be accessible to you. If this code "works" then I still wouldn't not recommend it. Let's say that the addPolicy function executes successfully, and then an exception occurs, now what state is the pipeline in? If this code executes again, then the policy is added multiple times to the pipeline. Basically, the pipeline gets corrupt with this kind of code. This is why the pipeline field should not be accessible to you. In other languages is it not accessible to you.

Looks like the above code works on my side, removePolicy also can remove the specified policy from client.pipeline, then client opration is executed under the default pipeline state after the code.
In this issue, there are the following possible ways, but all of them are not easy or recommended, right?

change JS SDK
change Azure REST API interface
add/remove policy to client object's pipeline
dedicated client object for retrying for 409 response

Changing the sdk to support retry options with status codes should be done.
Fixing the rest API in a non breaking way is likely possible.
A dedicated client that uses the changed sdk is a fine idea.
Changing the pipeline after client construction is strongly discouraged.
-- Jeffrey Richter

From: Hirofumi Horikawa @.>
Sent: Thursday, September 22, 2022 4:19:38 PM
To: Azure/azure-sdk-for-js @.>
Cc: Jeffrey Richter @.>; Mention @.>
Subject: Re: [Azure/azure-sdk-for-js] How to retry when response code is 409 (Issue #23298)
Looks like the above code works on my side, removePolicy also can remove the specified policy from client.pipeline, then client opration is executed under the default pipeline state after the code.
In this issue, there are the following possible ways, but all of them are not easy or recommended, right?

change JS SDK
change Azure REST API interface
add/remove policy to client object's pipeline
dedicated client object for retrying for 409 response

—
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-js%2Fissues%2F23298%23issuecomment-1255651337&data=05|01|jeffreyr%40microsoft.com|91b7038b35584cafab7c08da9cf0ebd4|72f988bf86f141af91ab2d7cd011db47|1|0|637994855807371791|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=6HKhw7f8FlnVbA09bTT6mK6Qa%2B6mh%2FwPuVoWluWSCpE%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAARLJP7FP42RLSPVPVTEATDV7TSQVANCNFSM6AAAAAAQSRN4RA&data=05|01|jeffreyr%40microsoft.com|91b7038b35584cafab7c08da9cf0ebd4|72f988bf86f141af91ab2d7cd011db47|1|0|637994855807371791|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=cgs4PpcvUVNPAawEwrRNnCBQkwxF0iKPUQyhMSBG1WY%3D&reserved=0.
You are receiving this because you were mentioned.Message ID: @.***>

Thank you @JeffreyRichter !
We will consider how to achieve this on our side based on your comment.

@xirzec, is the js libraries retrying if a Retry-After header is present? If not, why not? The general recommendation is that a service should only include a Retry-After header if it feels like the request can be retried without any modifications.

@johanste Thank you for your comment.
In this case, the response doesn't have Retry-After or similar header as following.
This is a log output by setting env AZURE_LOG_LEVEL=verbose.
azure:core-rest-pipeline:info Request: {
  "url": "https://management.azure.com/subscriptions/**REDACTED**/resourceGroups/**REDACTED**/providers/Microsoft.App/containerApps/**REDACTED**?api-version=2022-03-01",
  "headers": {
    "content-type": "application/json",
    "accept": "application/json",
    "accept-encoding": "gzip,deflate",
    "user-agent": "azsdk-js-arm-appcontainers/1.1.0 core-rest-pipeline/1.9.2 Node/v16.17.0 OS/(x64-Linux-<IP_ADDRESS>-microsoft-standard-WSL2)",
    "x-ms-client-request-id": "86d602a7-038d-42d7-8c99-801d5f3c6ebe",
    "authorization": "REDACTED",
    "content-length": "827"
  },
  "method": "PATCH",
  "timeout": 0,
  "disableKeepAlive": false,
  "streamResponseStatusCodes": {},
  "withCredentials": false,
  "requestId": "86d602a7-038d-42d7-8c99-801d5f3c6ebe",
  "allowInsecureConnection": false,
  "enableBrowserStreams": false
}
azure:core-rest-pipeline:info Response status code: 409
azure:core-rest-pipeline:info Headers: {
  "cache-control": "no-cache",
  "pragma": "no-cache",
  "content-length": "229",
  "content-type": "application/json; charset=utf-8",
  "expires": "-1",
  "x-ms-ratelimit-remaining-subscription-resource-requests": "496",
  "api-supported-versions": "2022-01-01-preview, 2022-03-01, 2022-06-01-preview",
  "server": "Microsoft-IIS/10.0",
  "x-powered-by": "ASP.NET",
  "x-ms-request-id": "2987476b-c044-4666-9b63-0a70095bbec4",
  "x-ms-correlation-request-id": "2987476b-c044-4666-9b63-0a70095bbec4",
  "x-ms-routing-request-id": "JAPANWEST:20220922T230849Z:2987476b-c044-4666-9b63-0a70095bbec4",
  "strict-transport-security": "max-age=31536000; includeSubDomains",
  "x-content-type-options": "nosniff",
  "date": "Thu, 22 Sep 2022 23:08:48 GMT"
}

@xirzec, is the js libraries retrying if a Retry-After header is present? If not, why not? The general recommendation is that a service should only include a Retry-After header if it feels like the request can be retried without any modifications.

Yes we support throttling using Retry-After with no additional effort from the consumer.
@horihiro Back to the issue at hand, is it true that your 409 will resolve itself in a relatively short timeframe? By default, we attempt to retry 3 times and give up after no more than around 10 seconds. While you can configure this to allow for greater delays, this configuration applies to all retry reasons, not just a single HTTP status code.
I think my suggestion here would be to not rely on pipeline magic for automatic retries and simply try/catch the client operation. This way you can detect the 409 and decide to execute the operation again as many times as you want with whatever pauses or delays you feel are necessary.

@xirzec Thank you for your suggestion. We will discuss how to retry on our side, including your suggestion also!

@horihiro Back to the issue at hand, is it true that your 409 will resolve itself in a relatively short timeframe? By default, we attempt to retry 3 times and give up after no more than around 10 seconds. While you can configure this to allow for greater delays, this configuration applies to all retry reasons, not just a single HTTP status code.

@xirzec According to the following logs shown by AZURE_LOG_LEVEL=verbose, the process never attempt to retry when returning 409.
Both of two retry strategies (throttlingRetryStrategy and exponentialRetryStrategy) included in defaultRetryPolicy were skipped in this situation.
azure:core-rest-pipeline:info Request: {
  "url": "https://management.azure.com/subscriptions/**REDACTED**/resourceGroups/**REDACTED**/providers/Microsoft.App/containerApps/**REDACTED**?api-version=2022-03-01",
  "headers": {
    "content-type": "application/json",
    "accept": "application/json",
    "accept-encoding": "gzip,deflate",
    "user-agent": "azsdk-js-arm-appcontainers/1.1.0 core-rest-pipeline/1.9.2 Node/v16.17.0 OS/(x64-Linux-<IP_ADDRESS>-microsoft-standard-WSL2)",
    "x-ms-client-request-id": "91fe19d8-634c-4909-936b-27181ea65edf",
    "authorization": "REDACTED"
  },
  "method": "PATCH",
  "timeout": 0,
  "disableKeepAlive": false,
  "streamResponseStatusCodes": {},
  "withCredentials": false,
  "requestId": "91fe19d8-634c-4909-936b-27181ea65edf",
  "allowInsecureConnection": false,
  "enableBrowserStreams": false
}
azure:core-rest-pipeline:info Response status code: 409
azure:core-rest-pipeline:info Headers: {
  "cache-control": "no-cache",
  "pragma": "no-cache",
  "content-length": "229",
  "content-type": "application/json; charset=utf-8",
  "expires": "-1",
  "x-ms-ratelimit-remaining-subscription-resource-requests": "499",
  "api-supported-versions": "2022-01-01-preview, 2022-03-01, 2022-06-01-preview",
  "server": "Microsoft-IIS/10.0",
  "x-powered-by": "ASP.NET",
  "x-ms-request-id": "983935a5-1750-409b-bfba-284bac01032f",
  "x-ms-correlation-request-id": "983935a5-1750-409b-bfba-284bac01032f",
  "x-ms-routing-request-id": "JAPANEAST:20220924T114715Z:983935a5-1750-409b-bfba-284bac01032f",
  "strict-transport-security": "max-age=31536000; includeSubDomains",
  "x-content-type-options": "nosniff",
  "date": "Sat, 24 Sep 2022 11:47:14 GMT"
}
azure:core-rest-pipeline retryPolicy:info Retry 0: Received a response from request 91fe19d8-634c-4909-936b-27181ea65edf
azure:core-rest-pipeline retryPolicy:info Retry 0: Processing 2 retry strategies.
azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy throttlingRetryStrategy.
azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped.
azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy exponentialRetryStrategy.
azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped.
azure:core-rest-pipeline retryPolicy:info None of the retry strategies could work with the received response. Returning it.
Deployment Failed with Error: RestError: Cannot modify a container app 'ctapp-aca-revision-deploy1' because there is an active provisioning operation in progress. OperationId: 'd49648d5-d883-4efd-a8c1-da959b015592'.

Sorry for not being clearer. My advice is to forget about the pipeline and handle retrying the operation on the client yourself, such as:
await retryOperation(() => {  return client.containerApps.beginUpdateAndWait(/* params*/) });

async function retryOperation(callback: () => Promise<void>, retryCount = 0): Promise<void> {
  try {
    await callback();
  }
  catch (e) {
     if (isRestError(e) && e.statusCode === 409) {
          // set this to whatever limit you want
          if (retryCount < 5) {
              // wait some number of seconds
              await delay(5000);
              return retryOperation(callback, retryCount + 1);
          }
     }
     throw e;
  }
}