Json builtin function doesn't accept empty strings

Using the composer, I found out that we don’t accept empty string value:
Example:
This works: = json('{"SR":{"Q1":"1","Q2":"2"}}')
This fails:   = json('{"SR":{"Q1":"","Q2":"2"}}')

Thanks @scheyal, we will look into this soon.

Initial investigation showed SDK support empty fine, but it fails in Composer main branch.

@scheyal @boydc2014 @Danieladu
The error is caused by the parser of string interpolation. ${} should yield an empty string but thorws an error.  While evaluating a json expression, it will resolve the data bindings recursively. So once it encountered a empty string "". It will be resolved to an empty string interpolation .
I created a PR to fix this issue. After it is merged, this issue will be fixed.


Progress Closure Parameter for Image Downloads

Closes #48

Thanks for putting this together @AnthonyMDev!
I rebased your changes and made a few small tweaks which were pushed into master in ed5ab8d0, acb73b35 and 4c45bc34 while maintaining your attribution. The only things I changed were the names in the ProgressHandler (should be read, not write) and flipped to dispatch_async instead of dispatch_sync in the ImageDownloader when calling the progress closure. I also made a few formatting changes and updated some of the docstrings.
Thanks again!

Great! Thanks for getting these pulled in!

Could you please post an example of how exactly this progress handler can be used?


Skip API operations for diff run

What this PR does / why we need it:
Diff run should be usable when the authorised RBAC role doesn't have
write permissions for resources in the cluster.
Diff run on app creation used the dry run feature of the Kubernetes
API, to avoid persisting kapp ConfigMap changes. However this still
checks RBAC permissions, and errored if the role did not have write
permissions for ConfigMaps.
The kapp ConfigMap was also being updated when doing a diff run for an
existing app.
This skips the API operations for updating the ConfigMap when doing a
diff run.
Which issue(s) this PR fixes:

Fixes #791
Does this PR introduce a user-facing change?

- Diff run will not check anymore for appropriate RBAC permissions.

Additional Notes for your reviewer:
Review Checklist:

[x] Follows the developer guidelines
[x] Relevant tests are added or updated
[ ] Relevant docs in this repo added or updated
[ ] Relevant carvel.dev docs added or updated in a separate PR and there's
a link to that PR
[ ] Code is at least as readable and maintainable as it was before this
change

Additional documentation e.g., Proposal, usage docs, etc.:

Hmm, not sure why the new test TestDiffRunReadOnlyContext is failing. On my machine with minikube v1.31.1 (kubernetes v1.27.3):
=== RUN   TestDiffRunReadOnlyContext
Running 'kapp delete -a test-e2e-rbac-app -n kapp-test --yes'...
Running 'kapp delete -a diff-run-read-only -n kapp-test --yes'...
Running 'kapp deploy -a test-e2e-rbac-app -f - -n kapp-test --yes'...
Running 'kubectl get secret scoped-sa -o jsonpath={.data.token} -n kapp-test'...
Running 'kubectl config view --minify -o jsonpath={.clusters[].name} -n kapp-test'...
Running 'kubectl -redacted-'...
Running 'kubectl config set-context scoped-context --user=scoped-user --cluster=minikube'...
==> diff-run using scoped context
Running 'kapp deploy -f - -a diff-run-read-only --diff-run --kubeconfig-context=scoped-context -n kapp-test --into-ns kapp-test --yes'...
Running 'kubectl get configmap diff-run-read-only -n kapp-test -o yaml -n kapp-test'...
Running 'kubectl config delete-context scoped-context -n kapp-test'...
Running 'kubectl config delete-user scoped-user -n kapp-test'...
Running 'kapp delete -a test-e2e-rbac-app -n kapp-test --yes'...
Running 'kapp delete -a diff-run-read-only -n kapp-test --yes'...
--- PASS: TestDiffRunReadOnlyContext (0.52s)

The other two tests (TestFormattedError, TestIgnoreFailingAPIServices) seem to be failing on the develop branch also, but I can investigate further.

Hey @vixus0! Thank you so much for the PR. Like you mentioned the other 2 tests are failing on develop as well, most probably because of some changes in the latest version of Kubernetes. I will fix them as soon as I get some time.
Regarding the new test, I will have to take a closer look to be sure what the issue is. Is it consistently working for you on your machine?

Yeah, I can consistently run the test on my machine. It seems in CI the initial kapp deploy step for actually setting up the scoped role is failing:
kapp: Error: waiting on reconcile secret/scoped-sa (v1) namespace: kapp-test:
  Errored:
    Getting resource secret/scoped-sa (v1) namespace: kapp-test:
      API server says: secrets "scoped-sa" not found (reason: NotFound)

The PR to fix the other 2 tests is merged, so you can rebase on those changes now for the tests to pass.


Odds are incorrect

README.md states:

He will wipe out about one third of your files if he snaps fingers without gauntlet. And If he has the gauntlet, he will crush about half of them.

However, the following line actually wipes out about one quarter of your files if he snaps fingers without gauntlet.
https://github.com/anandundavia/thanos-js/blob/c170118bad3f6062ad820dc2a67485d17d6f970c/lib/snap-fingers.js#L29

Ah yes, I will change the readme.md.
You are welcome to raise a PR if you want to :D


kwallet-pam: just provide plugin path for qtbase

kwallet sets a limit of 1000 for a single characters for environment
variables read from the socket[1]. wrapQtApps gives us a huge value
for QT_PLUGIN_PATH (up to 13000 bytes on my system!) Since this was
overflowing, the Qt plugin loading mechanism was hitting a segfault
when it was trying to parse the truncated QT_PLUGIN_PATH.
So for now, we can just unset QT_PLUGIN_PATH in the pam_kwallet_init
script. kwalletd5 has its own QT_PLUGIN_PATH which it can use.
This problem occured on 20.03, but not 19.09. It’s unclear what
changes were made in that time, but likely that previously we weren’t
getting a QT_PLUGIN_PATH set in the plasma5 startup at all. This means
that in 19.09 our QT_PLUGIN_PATH value must have been small enough to
fit into the 1000 char limit.
Fixes #77290
/cc @ttuegel

So for now, just get a QT_PLUGIN_PATH value for qtbase, providing the
xcb plugin which is the one we need. XDG_DATA_DIRS appears to be
unnecessary.

Let's also add qtwayland, otherwise I expect it will break for our Wayland users.

The build fails for me:
Builder called die: makeWrapper doesn't understand the arg QT_PLUGIN_PATH
Backtrace:
105 makeWrapper /nix/store/akb1l64z5f8w0k473zzwyvc5y6jlaqks-hook/nix-support/setup-hook
145 wrapProgram /nix/store/akb1l64z5f8w0k473zzwyvc5y6jlaqks-hook/nix-support/setup-hook
81 _callImplicitHook /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
98 _eval /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
41 runHook /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
1189 fixupPhase /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
1299 genericBuild /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
2 main /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh

But please add a comment why we're doing this. Someone could easily look at this and be like "well this needs to be updated for wrapQtApp` and break it again because we have no testing.

The build fails for me:
Builder called die: makeWrapper doesn't understand the arg QT_PLUGIN_PATH
Backtrace:
105 makeWrapper /nix/store/akb1l64z5f8w0k473zzwyvc5y6jlaqks-hook/nix-support/setup-hook
145 wrapProgram /nix/store/akb1l64z5f8w0k473zzwyvc5y6jlaqks-hook/nix-support/setup-hook
81 _callImplicitHook /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
98 _eval /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
41 runHook /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
1189 fixupPhase /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
1299 genericBuild /nix/store/izwfky32fcnz4gg9vj6yl35x0x4x2g4v-stdenv-linux/setup
2 main /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh

Updated with a fix for this - I was just passing in wrong args to wrapProgram.

This doesn't build for me.
The build error
-- Could NOT find KF5Wallet (missing: KF5Wallet_DIR)
-- Could NOT find KF5Wallet: found neither KF5WalletConfig.cmake nor kf5wallet-config.cmake
CMake Error at /nix/store/cmriqbxnpcgmbll06pnl1h3va08x2xvr-cmake-3.16.5/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find KF5 (missing: Wallet) (Required is at least version
  "5.62.0")
Call Stack (most recent call first):
  /nix/store/cmriqbxnpcgmbll06pnl1h3va08x2xvr-cmake-3.16.5/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  /nix/store/hxvvgjkvpp22g1rykzcb734649sgzjz0-extra-cmake-modules-5.66.0/share/ECM/find-modules/FindKF5.cmake:110 (find_package_handle_standard_args)
  CMakeLists.txt:38 (find_package)

-- Configuring incomplete, errors occurred!
See also "/build/kwallet-pam-5.17.5/build/CMakeFiles/CMakeOutput.log".
builder for '/nix/store/znxzdhm9hplgijnwgisj95khf5g88kpd-kwallet-pam-5.17.5.drv' failed with exit code 1
cannot build derivation '/nix/store/shahgnysbssvysmza9fvh5rrb75gg9ih-gdm.pam.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/rg1lr50hdf3i1pjih6phsiqazsyky3vb-kdm.pam.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/18swwxbj8hhdrpsydkk7gaplqnwd6vf4-lightdm.pam.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/0sbrpqyyw86zy3h4d38qy7v0c0axq6qx-sddm.pam.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vhin09vlcsyqd1cw2s3bm01zd0hjsjd1-system-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/mi6dhdjrgl4nzmqigag91p4lspck371p-dbus-1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/w6hk1qksyi43w55647m5wkppdkp8r13z-unit-polkit.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/3idhgmb8w406ksn93fg12m2v6kjjbfww-unit-systemd-fsck-.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/833injq3r8l3lgx33k9hn01aja76sm51-unit-dbus.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/bn5zxj0ndac4bpxaxln8pia969k98clc-unit-dbus.service.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/h9c7zhvw3r6z8y5g1s9bsc3052yli21y-system-units.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/y8dl7l74qc50jb7k828vgy6q8l5rfwyn-user-units.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/iggcv5dx7vs4085hf131lqh72vrbsnv3-etc.drv': 8 dependencies couldn't be built
cannot build derivation '/nix/store/rk4pl3c261hn73rivvcknbi5dl8ghc75-nixos-system-nixos-testing-20.09pre-git.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/ckjsj74a7mdp8jjqcibfn7xxs7gscnv6-closure-info.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/jxlfrs62yaaxv4x210wisiy5jrxnfphn-run-nixos-vm.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/avcbdpw6byq55jw7pkwxs4zx2390xzph-nixos-vm.drv': 2 dependencies couldn't be built
error: build of '/nix/store/avcbdpw6byq55jw7pkwxs4zx2390xzph-nixos-vm.drv' failed

However, putting back the kwallet dependency in buildInputs helps resolving this, and also fixes the coredump. I'm not sure why it was removed here, but it seems like putting it back and leaving the rest alone resolves the linked issue.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/declarative-wrappers-another-idea/6661/1


pull request for e2e test at 11/15/2016 11:26:08

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


chore: parse source from CLI

@aladdin-add Can you merge this and adjust the test runner to run this code?


exports: need for better support

screenshots:

I suppose it some sort of array joining.

fixed this particular case (but JavaScript exports generally are weird beast, there is ES6 export keyword which is pretty easy to analyze, and there are e.g. CommonJS module.exports which are a little bit harder)

exports support improved. (ES6 exports and CommonJS)


Histogram error handling

Following on #16, it handles all current panic scenarios for our bin strategies, returning an Option when a suitable bin collection cannot be generated.

I've proposed some changes in LukeMathWalker/ndarray-stats#4. In #24, we decided to use Result<Option<_>, _> in some places for consistency with methods that return None for empty input arrays. After seeing how that approach affected this PR, though, I think we should use Result<_, _> instead after all. Doing so simplifies the implementation and public API. There really isn't much disadvantage to returning Result even for things like .mean() (which now returns Result<A, EmptyInput> instead of Option<A>). There should be no performance cost, and it's easy to convert Result<T, E> to Option<T> if needed with the .ok() method on Result. I think I've also addressed all of @munckymagik's comments.

Looking at the code in https://github.com/LukeMathWalker/ndarray-stats/pull/4 after the shift to Result everywhere I do agree it looks much cleaner (and less nested thanks to ? and error conversions)!

@jturner314 @LukeMathWalker looking really good.
There was just one other comment I made about | vs ||.
Otherwise 🚢 it.

Taken care of that one @munckymagik :+1:


Investigate LAN Integration / PoC

Title of ticket:  Investigate LAN Integration / PoC
Description
Investigate LAN integration and create a barebones PoC, ensuring that we have access to the LAN from a pod in OpenShift.
There are 3 options we're investigating (in descending preference), though we are open to other options:

Volume mounting at OpenShift / OS level
In-memory library, like a .NET Core lib.
Spin up a Windows VM

Dependencies
Dependencies on ISTORE request to grant us access to the LAN. We need to find out what exactly these dependencies are (eg do we need to whitelist the OpenShift IPs? Credentials?)
DOD

[ ] Have a pod deployed in OpenShift that is able to READ/WRITE/DELETE files in the LAN.
[ ] Have integration info documented somewhere private.

Here are my notes so far:

OpenShift LAN Integration
Options
Summary: Option A ("Volume Mounting") is preferable, and looks likely at this time, but option B is still fully viable.
Option A ("Volume Mounting"): We mount, or attach, the LAN directly to the OpenShift containers using built-in OpenShift features.

PRO: Tightest integration. LAN appears as file-system to OpenShift.
PRO: RH and Kube examples  ^1 [^RH-Cifs] [^Kube-example]

CON: May not be possible, esp. since CITZ/LAN/OpenShift are "locked down"

Option B ("Code Integration"): We handle LAN integration in code.

PRO: Still have full LAN access, just it's fully our responsibility instead of integrated into OpenShift ^2

CON: More likely to have limitations than volume mounting (but none currently found)

Notes
The LAN is a windows network drive, which uses SMB[^6]. For Linux/OpenShift, we need SMB/Samba adaptors like cifs.^0
Matt has put in ISTORE request for TEST lan.
Footnotes
[^RH-Cifs]: RH 3.x cifs mounting, should be relevant for 4.x https://www.openshift.com/blog/mounting-cifs-shares-in-openshift
[^Kube-example]: Official Kubernetes NFS / NTFS example https://github.com/kubernetes/examples/tree/master/staging/volumes/nfs
[^6]: https://en.wikipedia.org/wiki/Samba_(software)#:~:text=Samba is a free software,or as a domain member.

We are pending an iStore consultation order for more details and LAN setup.

Marking this as CLOSED, as we are not proceeding with OpenShift<->LAN integration.


add make kernel to compile hyper kernel & modules

build temporary image contains kernel source, kernel config, hyperstart source.
and run this image to build hyper kernel&modules, the kernel and initrd.img
stays in hyperstart-source/build/result/ directroy.
Signed-off-by: Gao feng<EMAIL_ADDRESS>

please split make-kernel.sh into multiple files.


Package is invalid: 'CRX_HEADER_INVALID'.

Getting this when trying to drag and drop a theme from the /themes directory into the opera://mods tab.

My browser info:
Opera GX
LVL5 (core: 109.0.5097.108)
Checking for updates…
Update stream:Early Access
System:Windows 11 64-bit
Chromium version:123.0.6312.124

Do the files before https://github.com/catppuccin/opera-gx/commit/66cb4057837073e6d913053cf6d8fe0c4adde6d0 work?

yes the files before https://github.com/catppuccin/opera-gx/commit/66cb4057837073e6d913053cf6d8fe0c4adde6d0 work fine

Gotcha. Usually browser extensions work with just zip files but I guess Opera mandates the CRX header. @isabelroses do you know how you zipped the CRX files before my Whiskers PR?

Gotcha. Usually browser extensions work with just zip files but I guess Opera mandates the CRX header. @isabelroses do you know how you zipped the CRX files before my Whiskers PR?

For my rough memory, i signed them all myself by zipping each of them up in the browser.

Ah, shoot. I'm not sure if we can automate this. Feel free to revert my previous PR for now!

Reverted for now, https://github.com/catppuccin/opera-gx/pull/5. Creating a meta issue to track automation.


any way to store tree data but the unique field only ???

I need to insert bulk data in a table, I am getting this data from api backend..
const Movie = use("App/Models/Movie");

const movies = [{
 id: 1, movie_id: 1, name: "Despicable 2",
 id:2 movie_id: 2, name: "I, Robot"
}],

const movies_created = await Movie.createMany(movies) // works very well

but problem happens where the same data repeat again
const movies = [{
 id: 6, movie_id: 6, name: "Pirates of the Caribbean",
 id:2 movie_id: 2, name: "I, Robot" // this raw again insert in table
}]

const movies_created = await Movie.createMany(movies) // 1 duplicate data

how to solve this.. only insert if not exists ???
movies.forEach(movie => {
  await Movie.create(movie)
})

using forEach will take forever as i need to insert bulk data of may be thousand or more movies ??? Also, in Laravel We can use createOrUpdate(), can we have createManyOrUpdateMany ???
createManyOrUpdateMany({movie_id: 2}, {
 id: 6, movie_id: 6, name: "Pirates of the Caribbean",
 id:2 movie_id: 2, name: "I, Robot"
})

I hope you will be getting this data from the user input. So the best place is to validate the uniqueness of the field inside the database using http://adonisjs.com/docs/4.0/validator.
If you think that your implementation needs more checks for finding the uniqueness, then feel free to add your own validation rule defined here. http://adonisjs.com/docs/4.0/validator#_extending_validator

Closing since no answer from issue reporter.


Predict future popularity trends and revenues with machine learning

We have:

Google trend index data from 2006 to 2020 at a monthly resolution of Canon, Nikon, and Sony cameras, DSLR, and mirrorless
Annual revenues of camera divisions of Canon, Nikon, and Sony

We want to somehow predict future trends.

@tangaw What's your take on this?

I'd like to predict future market shares of these 3 brands

Will need quite a few more independent variables to produce an acceptable model.


Accumulate the gradient to a tuple for lazy add operation

The accumulated add is to fuse the add operations to improve the performance of backward. Take the below piece of graph for example. The x variable is input for 3 functions, so the gx should be accumulated on the 3 functions backward. Currently implementation add the current gx to the new gx on each functions backward. However on googlenet/googlenet v2, there are many forked functions for one input, so the fuse for accumulated add is meaningful.
X_
|               |                |
Func1      Func2     Func3
This patch is platform independent. It should both benefit for GPU and CPU devices. The basic idea for patch accumulate the gx to a tuple instead of doing add operation when the new gx need to be added to current gx. When the gx is inputted as gy for the next layer, the actual fused add should be done on the tuple of gx. We can call the approach as lazy add. Since there are 2 backward functions in the framework, the lazy add is applied both in _backward_main() and _back_prop(). We implemented a function node (AccumulateAdd) to calculate the sum of a tuple of array. Fuse add optimization can be done in this function node, and we will submit that patch in another PR.

@niboshi
I refactor the accumulate_add. Could you help to review?

jenkins, test this please

@niboshi
I modify the code as you suggest. Could you help to review?

Refactor test code.

@toslunar
Sorry for response later. Thanks for the comments. I clean the code when lazy_grad_sum is turned off. The test case is under developing.

Hi toslunar/niboshi
I notice there is some change in master. Very glad to see ideep has been integrated to master chainer. However there is a conflict in my PR and I don't know the process to solve it. The test case for lazy_grad_sum is pushed to github. If it is possible, could you integrated the patch of the PR to the master?

@toslunar
Test case is added in test_variable.py. Could you help to review if there is any other issues. Thanks.

jenkins, test this please

@niboshi
I have updated the patch according to your comments. :)

Thank you!
Finally, could you add chainer.functions.add on top of function list documentation? (docs/source/reference/functions.rst)
We would need a new section (like "Arithmetic functions"?).

Ok. By the way, do we need move chainer.functions.absolute from Matchematical functions to Arithmetic functions?

Thank you for pointing out.
We may consider reordering of the functions later, but for now could you leave them as they are?

Ok, I will only add chainer.functions.add to Arithmetic functions.

Thanks!
jenkins, test this please

Hi niboshi,
All checks have passed. Is there some process we should follow before merge the PR?

LGTM!
Thank you very much and I'm sorry for taking time to merge.

Thanks!


Version photos when updating EXIF

Since updating EXIF rewrites the file and can cause corruption we should consider copying the previous version of the file to some known location.
Taken from this comment on Medium.
<EMAIL_ADDRESS>

Just as an idea, maybe just dump out the EXIF each time it changes and version that... it wouldn't save from file corruption (that's for backups and bitrot detection), but it would save from the accidental trashing of EXIF data (input wrong date, etc.)

@bdillahu dumping the EXIF is an interesting idea. If we version the photos themselves it also backs up the EXIF (inside the backed up photo). It's a bit heavy handed but in the point taken from the comment on Medium it serves the primary purpose of protecting all the bits of a photo when it's modified and not just EXIF.
Dumping the EXIF is more closely related to supporting XMP sidecar files. I thought there was an issue for it but can't seem to find one.

https://github.com/jmathai/elodie/issues/341


Set tenant ID for predict request

Description
Tenant ID changes to support predict API. With this change was able to successfully test predict API with remote model and multi tenancy enabled. Following changes are included in this PR:

Set tenant ID during predict request
MLModelManager to support getModel using tenantID. Removed passing SDKClient as input and moved it to class object.
Pass SDKClient to MLModelManager. TODO: SDKModule is being initialized twice. Move to a uniform mechanism to initialize SDKClientModule.
Bug fix in DDB client to pass tenant ID.

Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

LGTM.
Except integ tests are failing. Not sure why, is it the two client thing?
After this is merged, I'll pick up the TODOs with the injection:
* I'll change SdkClient from an interface to a concrete class, with an internal implementation as a field that it delegates to.  Then we can inject it in CreateComponents

* I'll have the class take settings when instantiating to choose which client to implement

* I'll update the SdkClientModule to be a util class to help with all the above

* I'll add a new client for Aws Sigv4 support per https://github.com/opensearch-project/opensearch-java/blob/main/guides/auth.md

Integ tests were failing due to transient exception. Rerunning integ tests succeeded.
Thanks for quick turnaround with SDKClient module refactoring in PR: https://github.com/opensearch-project/ml-commons/pull/2638


Fix sourcemaps for HMR mode

Must be specified explicitly or it won't be properly generated

1. Why is this change necessary?
Because you don't have style sourcemaps in HMR mode.
2. What does this change do, exactly?
It activates the sourceMaps for the loaders. According to the webpack documentation they should be activated by the devTools option, but the css-loader and postcss-loader don't respect that. Therefore you don't have a sourcemap.
3. Describe each step to reproduce the issue or behaviour.
Open frontend in HMR mode and try to inspect a style.
4. Please link to the relevant issues (if any).
5. Checklist

[ ] I have written tests and verified that they fail without my change
[x] I have squashed any insignificant commits
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Thanks for your contribution!
We will merge your PR soon :)


Update Week 1 Quiz - Practical aspects of deep learning.md

According to the last quote dated 2017-11-08

This question is randomly selected from two candidate questions.


rbg-hex.py

1.run the program
2. test case doesnt work
FF7F00 shouldve printed
that was not printed

macOS
intel chip
change the 1000000 to 0
yes related issue with b
high priority issue
always reproduced
high impact

thank you very helpful


⚠ Bofrost.es has degraded performance

In d9be269, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1566 ms

Resolved: Bofrost.es performance has improved in 733e1f6 after 7 minutes.


accept \0 as entry separator  and multiline entries

Category

[x ] fzf binary

Allows searching blocks of text with embedded \n

having looked at the close issues, I notice that other have requested this and I saw why I would probabhaly not implemented.
A related feature could be to work on the input line up to a separator, defined as an option, this allows the user to squeeze in some user data
print "1 :magic: user data 1\n2 :magic: user data 2" | fzf -user_delimiter ':magic:' | further_processing

Have you tried these? https://github.com/junegunn/fzf/blob/master/man/man1/fzf.1#L287-L292
(related #660)

No I haven't, till now, because they weren't in the man page of the version I run. --read0 does work with my version and that makes things much easier.
Thank you.


Feature: Add spinning loader until full list of policies is successfully download on the client

Explanation of Change

Fixed Issues

$ https://github.com/Expensify/App/issues/52766
PROPOSAL: https://github.com/Expensify/App/issues/52766#issuecomment-2485802225
Tests

Test 1:
Precondition: Be on 3G / Slow internet

Go to Troubleshoot page in settings > Click "Clear cache and restart"
Go to Workspaces

Verify that: you will see a loading spinner before seeing the workspace list page / empty workspace screen
Test 2:
Precondition: Be on 3G / Slow internet

Go to Troubleshoot page in settings > Click "Clear cache and restart"
Go to Workspace switcher LHP

Verify that: you will see a loading spinner before seeing the workspace list page / create workspace screen

[x] Verify that no errors appear in the JS console

Offline tests

Test 1:

Go to Troubleshoot page in settings > Click "Clear cache and restart"
Go to Workspaces

Verify that: you will not see a loading spinner before seeing the workspace list page / empty workspace screen
Test 2:
Precondition: Be on 3G / Slow internet

Go to Troubleshoot page in settings > Click "Clear cache and restart"
Go to Workspace switcher LHP

Verify that: you will not see a loading spinner before seeing the workspace list page / create workspace screen
QA Steps

Test 1:
Precondition: Be on 3G / Slow internet
(Note that this will not be reproducible on fast internet)

Go to Troubleshoot page in settings > Click "Clear cache and restart"
Go to Workspaces

Verify that: you will see a loading spinner before seeing the workspace list page / empty workspace screen

Test 2:
Precondition: Be on 3G / Slow internet

Go to Troubleshoot page in settings > Click "Clear cache and restart"
Go to Workspace switcher LHP

Verify that: you will see a loading spinner before seeing the workspace list page / create workspace screen

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I used JaimeGPT to get English > Spanish translation. I then posted it in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.ts or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] I added unit tests for any new feature or bug fix in this PR to help automatically prevent regressions in this user flow.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native
https://github.com/user-attachments/assets/cb7a9912-1e0e-4f63-8e06-300d95757628

Android: mWeb Chrome
https://github.com/user-attachments/assets/19ef7c1f-8de1-46a2-9485-7c4c1fa01f25

iOS: Native
https://github.com/user-attachments/assets/5fc74d7c-7d69-4b2d-bcad-b80239018ae9

iOS: mWeb Safari
https://github.com/user-attachments/assets/fc1af147-4bc6-46de-8b3a-4e9b18516a31

MacOS: Chrome / Safari
https://github.com/user-attachments/assets/bd4dc376-b366-4c1e-8449-d083055997f7

MacOS: Desktop
https://github.com/user-attachments/assets/f3ff4937-c320-461e-a7fd-b65126e0a3b7

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.


update telemetry language features

@rtfpessoa there are issue in the CI on the activated test, no ?

@Kyle-Verhoog do you know what happened here ? https://github.com/DataDog/system-tests/actions/runs/3089788748/jobs/4997809005
@rtfpessoa you can ignore this failure

@Kyle-Verhoog do you know what happened here ? https://github.com/DataDog/system-tests/actions/runs/3089788748/jobs/4997809005

@ZStriker19 looks like test_single_rule_always_keep_span_sampling_sss011 is a little flaky, should we expand the range a bit?

Yeah I'll expand it

https://github.com/DataDog/system-tests/pull/496


HW04 ready for grading

@vincenzocoia @gvdr @ksedivyhaley @JoeyBernhardt @mynamedaike @pgonzaleze @derekcho

Hi Alexander,
Great job on your homework 4! I have a few more specific comments below:

Your README file was clear and included links to resources. Made it a cinch to navigate your repo. You also included a reflection.
You completed the 2 required activities.
I like that you supressed the packages warnings/messages: made it neater and easier for me as the reader.
Your code for displaying caption via kable on the “Mean life exp” table looks correct to me, I'm not sure either why the caption doesn't display. Have you tried running the caption with format = HTML or tried kable_styling( )?
Your comments and reflections on your processes and challenges within the assignment was a nice added personal touch. I like that you contrasted plotting using tables in the least efficient way (wide format) with efficient tables (long format) via dcast. You mentioned you searched online and chatted with your lab members re tidyr and reshape - I'm curious as to what the pros and cons are? And why you ultimately chose the reshape package?
Second activity on joins looks great to me! It goes through all the joins and describes the process as well!

Thanks for sharing your repo with me. Hope you have a great rest of STAT545.
Melanie

Hi Alexander,
Good job on your homework 4. You have a well-organized repo and readme file. Here are some comments.
-For the missing caption, I also have this issue when develop my own plot. But sometimes, it works with html format, maybe you can try that.
-You showed how to make plot when having long or wide format and discuss the benefit from each format.
-You fully understand join function and explored different join function.
-You finished the assignment excellently, but I would suggest to do one or two more tasks to have a better sense about reshaping and merge/match function.
Overall, you did a great job!
Maria

Reshaping activity: Yes (see comments)
Join activity: Yes (see comments)
Reflect on process: Yes
Bonus (merge/match): No
Comments:

We were expecting you to use gather() and/or spread() for reshaping, though dcast does have a similar function and you explain it well.
If you haven't seen an example of the more “complicated” reshaping during your peer review, feel free to message me and I can find one for you.
There actually are plots that are easier to do with wide data. If you want to plot, for example, the mean life expectancy in Africa vs the mean life expectancy in Asia, it is easier to have those in separate columns. Your discussion of why you usually want long data is good, though.
Your exploration of what happens when Ghana has a missing population value is interesting. However, it is easier to see how the joins work when there is a country that is missing entirely – that is to say, when not all countries in gap_a are in gap_b and vice versa.
Reading your assignments is both easy and entertaining. Thank you for that.

Your mark will be distributed later. If you would like more feedback, please feel free to message me on slack.


V1.6.0/test ci

Codecov Report

:exclamation: No coverage uploaded for pull request base (v1.6.0/base@58feea4). Click here to learn what that means.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             v1.6.0/base      #12   +/-   ##
==============================================
  Coverage               ?   15.77%
==============================================
  Files                  ?       57
  Lines                  ?     5520
  Branches               ?        0
==============================================
  Hits                   ?      871
  Misses                 ?     4575
  Partials               ?       74

Flag
Coverage Δ

unittests
15.77% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Dedicated types

Separate Optionalize into 2 different types to make it more maintainable
Add the AnythingBut type that Skillz requested
Add some helper types to dry code a bit (and export them too)
Optionalize works with tuples now (I personally needed it to work with tuples)

(No breaking changes)

run-tests

run-tests

run-fmt

run-fmt


Multiple plugins

Moving crawl code from features_crawler to plugins based code for these features: config, file, and package. On top of this, this change includes moving some of the shared code between plugins to common .py files (e.g. os_crawler.py for os code common to os_container_crawler and os_host_crawler). Besides doing this, had to change some of the tests as well.

Current coverage is 83.59% (diff: 34.21%)

Merging #173 into master will decrease coverage by 2.35%

@@             master       #173   diff @@
==========================================
  Files            30         37     +7
  Lines          2884       3030   +146
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           2479       2533    +54
- Misses          405        497    +92
  Partials          0          0

Powered by Codecov. Last update 71d83d9...d8a8b95

@sahilsuneja1 updated the change.

all well, merging.


Marker is there but not showing

Issue description
I have an issue with markers, if I hover the mouse on the map, I can see the event trigger in there, also if I have an info window and I click on the marker, it would show the info window.
But the marker is not visible, is like invisible,  I already set the visible and zIndex values to test and still not showing.
I also tried to remove the custom icon and still.
Steps to reproduce and a minimal demo of the problem
I am figuring it out how to reproduce this error.
Is there any thing that I can try ?

It was a chrome temporary bug


AWS region not being honored

Checklist

[x] Upgrade Jets: Are you using the latest version of Jets? This allows Jets to fix issues fast. There's a jets upgrade command that makes this a simple task. There's also an Upgrading Guide: http://rubyonjets.com/docs/upgrading/
[x ] Reproducibility: Are you reporting a bug others will be able to reproduce and not asking a question. If you're unsure or want to ask a question, do so on https://community.rubyonjets.com
[x] Code sample: Have you put together a code sample to reproduce the issue and make it available?  Code samples help speed up fixes dramatically. If it's an easily reproducible issue, then code samples are not needed. If you're unsure, please include a code sample.

My Environment

Software
Version

Operating System
OSX

Jets
2.3.4

Ruby
2.5.7

Expected Behaviour
The developer guide / deploy guides should show you how to honor the region in AWS
https://rubyonjets.com/quick-start/
https://rubyonjets.com/reference/jets-deploy/
Current Behavior
If you follow the developer guide, you will see that the jets defaults to us-east-1 instead of what is defined in the ~/.aws/config file like the jets-deploy says it should.
Step-by-step reproduction instructions
Pre-test
Make sure that the aws cli app is NOT installed or at least not in your PATH. You can verify this
git:(master) ✗ aws configure get region
zsh: command not found: aws

Test
Setup the ~/.aws/config as described in the docs
Run the deploy command. It will use us-east-1
Code Sample
You can see if the aws cli isn't installed it skips over the config:
https://github.com/tongueroo/jets/blob/112d9ff4f91c4dafc160d63371f15aaaafe0ec89/lib/jets/aws_info.rb#L17
Solution Suggestion
Either update the docs to say the CLI is required or don't call the aws cli tool (preferred, b/c you don't need to start another process just to read from a text file).

What should be the expected behaviour if there is no AWS CLI installed and no ~/.aws/config is found? Should the deployment fail or just go ahead and deploy on us-east-1?
Maybe we can deploy it on us-east-1 and then display a message to notify the user about this?

What should be the expected behaviour if there is no AWS CLI installed and no ~/.aws/config is found?

I don't have a strong opinion about this.
My problem is that the docs say to add ~/.aws/config but then don't honor what is in the file if the cli isn't installed. This doesn't make sense because the docs don't say the AWS client is required. I think you should either:

require the AWS client to be installed
Use ruby to directly read from that file instead of starting up another process (making your deploy slower).

I have just 1 problem with the second option, if there any are other parts of Jets, which also depend on AWS CLI being installed(maybe @tongueroo can chime in)? Because if so, then updating the docs to require AWS CLI makes more sense. Either way we can make a small PR on it.

I would update the docs and the code:
https://github.com/tongueroo/jets/blob/112d9ff4f91c4dafc160d63371f15aaaafe0ec89/lib/jets/aws_info.rb#L17
This should be throwing an error if the client isn't present. It seems like the intention was for it to be optional (based on the code and preceding comment).

RE: AWS region not being honored

Dug into this. Unable to reproduce. Was able to deploy to different regions with different aws profiles in ~/.aws/config:
$ AWS_PROFILE=tung-west aws configure get region
us-west-2
$ AWS_PROFILE=tung-east aws configure get region
us-east-1
$

$ AWS_PROFILE=tung-east jets deploy
...
$ AWS_PROFILE=tung-east jets url
API Gateway Endpoint: https://0m17ffdv9l.execute-api.us-east-1.amazonaws.com/dev
$ AWS_PROFILE=tung-west jets deploy
...
$ AWS_PROFILE=tung-west jets url
API Gateway Endpoint: https://ui6ce467hc.execute-api.us-west-2.amazonaws.com/dev
$

$ AWS_PROFILE=tung-east aws apigateway get-rest-apis
{
    "items": [
        {
            "id": "0m17ffdv9l",
            "name": "demo-dev",
            "createdDate":<PHONE_NUMBER>,
            "binaryMediaTypes": [
                "multipart/form-data"
            ],
            "apiKeySource": "HEADER",
            "endpointConfiguration": {
                "types": [
                    "EDGE"
                ]
            }
        }
    ]
}
$ AWS_PROFILE=tung-west aws apigateway get-rest-apis
{
    "items": [
        {
            "id": "ui6ce467hc",
            "name": "demo-dev",
            "createdDate":<PHONE_NUMBER>,
            "binaryMediaTypes": [
                "multipart/form-data"
            ],
            "apiKeySource": "HEADER",
            "endpointConfiguration": {
                "types": [
                    "EDGE"
                ]
            }
        }
    ]
}
$

Details of jets deploy: https://gist.github.com/tongueroo/7b244c11c27ec882c9bb17c9d73af00f
Wondering if AWS_REGION might be set?  When it is set, it takes higher precedence than what's in ~/.aws/config.  https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html#config-settings-and-precedence  So then it would always deploy to the same region, regardless of what's in ~/.aws/config.
It's actually a useful way to deploy to multiple regions without having to create another AWS_PROFILE. IE:
AWS_REGION=us-east-1 jets deploy
AWS_REGION=us-west-2 jets deploy

RE: require the AWS client to be installed

Yes. There are places where jets calls shells out to the aws cli. Would like to make it optional. Thought it was working. Unsure if it's worth the time.  Think will just update the docs to say install it to move on.

RE: Use ruby to directly read from that file instead of starting up another process (making your deploy slower).

Sure. Not shelling out to the aws cil is preferable. Will consider a PR for it.
Tip: Used a2ikm/aws_config in the tongueroo/aws-mfa-secure gem achieve this there.  So that's how it would be done. But in the aws-mfa-secure gem itself, it shells out as a trick to grab temporary credentials because the gem itself overrides aws-sdk-core credentials logic to achieve what it needs to do. FWIW, the aws-mfa-secure decoration logic only triggers if you actually configure your ~/.aws/config to use it. Both should probably be updated so that it doesn't shell out to the aws cli at all. Believe Jets is the easier one to update actually. Will consider PRs for both.

@tongueroo
$ AWS_PROFILE=tung-west aws configure get region
us-west-2

This should be returning:
command not found: aws

Once you have verified that aws cli is NOT installed, try running jets deploy and see which region it deploys to.

Oh I see. So the aws cli is not installed and jets doesn't respect the region because it fails on shelling out. Make sense. Cool, will just add a note to make sure aws cli is installed.

Why not update the code too?

Sure. Will consider PRs. Otherwise will get to it in time. No sweat either way.


⚠ Samsung has degraded performance

In 0a58638, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2284 ms

Resolved: Samsung performance has improved in 1b5b047.


Lab 4 : Codecrunch Discrepancy in Running code

Summary
Why is my code not running properly on codecrunch?
Description:
I have been stuck on this for the entire time, codecrunch is seemingly running, I assume, compareTo method in level 3 differently than expected. Seemingly, the jsh script on codecrunch is not sorting at all. I have ran the code on jshell multiple times and yet I do not understand why this keeps happening.
Code Snippets
Attach code snippets (if any)
Screenshots

Seems like your output is sorted in increasing waiting time, followed by increasing fare

How do you know what is happening if you do not have the code to test3.jsh? Can you elaborate on how a test affects how the method is running in your code?
Is it possible that you have implemented the compareTo method incorrectly? (aka does b3.compareTo(b4) < 0 returns true in the level 3 test case.)

I dont know test3.jsh of course, but I expected the way that the sort happens in some direction.
My implementation of compareTo is simply comparing the cheapest fare of both bookings, say D = (fareA - fareB) and return a negative integer if D < 0, positive integer if D > 0, then similarly for waiting time for the tiebreaker.
Assuming the test3.jsh script is the same as the questions test case, yes b3.compareTo(b4) < 0 did return true.

The sort happens in the direction of the tiebreaker. So for (sort by cheaper -> then sort by faster), the sorting by fares should either return +ve, -ve or 0. The waiting time will be the tiebreaker when D == 0.

Oh I see it now, it was because my variable names were flipped, that caused the sort to happen with the tiebreaker first, then the fare, thank you so much


Adding AssetPath::resolve() method.

Objective
Fixes #9473
Solution
Added resolve() method to AssetPath. This method accepts a relative asset path string and returns a "full" path that has been resolved relative to the current (self) path.

Reviews: Please pay special attention to the doc comment. I've tried to explain the behavior as clearly as I can.
The motivation for this change is largely driven by the need for assets that internally contain references to other assets. Allowing a standardized way of representing relative relationships makes assets more portable, since they can be moved around in the asset directly without having to adjust all the paths.

Do we need anyone else to weigh in on this?

Do we need anyone else to weigh in on this?

In general, Bevy requires two community approvals for any non-trivial PR. In this particular case, I'm waiting for Cart to weigh in (or at least for his #8624 to merge) to avoid horrible merge conflicts.

@MrGVSV @alice-i-cecile I think once this gets committed I want to add an additional feature, which is to allow relative labels as well as relative paths. The syntax would be the same as JPointer (which is used by json-schema): "#./abc/def".
Here's the use case: Say I have a widget, such as a slider, which accepts a reference to a style asset. Now, the slider has internal parts such as a track, thumb, and so on. These all need styles, so each part needs an asset reference. I'm assuming that there's no CSS-like selector matching, so each part needs it's style specified explicitly.
At the same time, however, I want to be able to apply different styles to the slider, so the style asset needs to be a parameter to the slider.
In fact, I want to be able to ship a library of "unstyled" widgets that other people can use - I supply the widget behavior, and other people add a new coat of paint for their game.
However, it's too cumbersome if they have to pass in a separate asset path for each slider part.
The answer: relative labels. I design my slider so that all of the parts derive their styles from the base path of the style: "#./slider-thumb", "#./slider-track" and so on. So long as the user who is instantiating the slider has a style asset whose internal labels match the naming scheme, they only have to pass in a single asset path.

I like that design :) Agreed on leaving it for a follow-up PR though.

I've gone ahead and rebased the PR against the latest HEAD (including the Asset V2 work).
Also, I wanted to mention, that it is not intended that relative paths be supported automatically by the asset loading system. Rather, these methods are a convenience for people using assets and writing asset loaders, who can decide whether or not to support relative paths, and more importantly, can decide for themselves what they consider to be the "base path".
For example, in the use case I mentioned above, the path resolution happens at the time when a UI template is instantiated, not when it is loaded - because the base path would in that scenario be a parameter that is passed in to the template.

I've rebased the PR against the most recent changes to main.
I'm a bit concerned about the number of calls to .clone() and .to_owned() that I had to add to make this code work. I'm also not 100% convinced that I have the lifetime parameters correct - since it's effectively creating a new AssetPath every time, the lifetime of the result might not need to have any relation to the lifetime of self.

Given that we are covering "relative asset paths" here, I think its worth discussing the plans I have for Asset Paths in Bevy Asset V2 / Multiple Asset Sources to make sure this aligns.
We will need to be able to properly support and track "relative asset paths" to enable moving asset folders without reprocessing them. When possible, assets should load other assets using relative paths so we can encode that information during processing.
Relative asset paths are also important for Multiple Asset Sources because a loaded asset should not need to know the name of the asset source it currently exists in. Currently some asset stored in remote://foo/a.gltf referencing the foo/x.png asset path will look in the default (generally the filesystem) asset system because foo/x.png is considered an "absolute path". Not ideal!
Some basic principles I think we'll want to adopt for paths going forward:

foo/a.gltf: this is a relative path. It must be resolved to an absolute path based on the current context. The source is undefined so it will use the "default" asset source for the context. If the current context is custom://x/y, then it will be resolved to custom://x/y/foo/a.gltf
./foo/a.gltf: this is also a relative path. It must be resolved to an absolute path based on the current context. The source is undefined so it will use the "default" asset source for the context. If the current context is custom://x/y, then it will be resolved to custom://x/y/foo/a.gltf
/foo/a.gltf: this is an absolute path. This source is undefined so it will use the "default" asset source for the context
embedded://foo/a.gltf: this is an absolute path. The source is defined so it will use the embedded source.

Note that (1) is currently treated like an absolute path in the current implementation. Making this relative would be a breaking change, but in practice, given that it is generally used in the context of server.load("foo/a.gltf"), this will still behave identically (although we should probably encourage and adopt /foo/a.gltf in these cases for explicitness). I believe strongly that this should be treated as a relative path. Especially in the context of this resolve() implementation, because treating it like an absolute path would defy user expectations (based on their experience with other "file path" / "file system" apis).
From my perspective, "resolve" translates roughly to "for the current AssetPath context / working directory, resolve the final path given a path". Or put another way "path.resolve(input) behaves roughly like path commands in the linux terminal, where the path is the working directory and input is the path command".
With that context out of the way, lets consider the results of the resolve() implementation (pseudocode for brevity).
Here are some samples from the tests in this PR:

"alice/bob#carol".resolve("joe/next#dave") == "joe/next#dave"

These are two relative paths. I would interpret this as: for the relative path alice/bob#carol, navigate to joe/next#dave. "joe/next" is ambiguous as a file vs directory, but next is unambiguously a file here because of the #dave label. This feels "unresolvable" because "alice/bob#carol" is already a file / leaf. I find it resolving to "joe/next#dave" to be confusing. It does make some sense in the context of "joe/next#dev" being treated as an "absolute path" in the current asset implementation, but in the context of resolve() it makes no sense to me / defies my expectations as a user of filesystems. I don't think it is a good idea to encourage people to think about paths in this way, for this API.

"alice/bob#carol".resolve("./martin#dave") == "alice/martin#dave"

./martin#dave is a relative path, so I interpret this as "navigate to martin#dave within the context of alice/bob#carol. alice/bob#carol is already a leaf (a "relative file path leaf"), so there is no martin#dave to navigate to in this context. I would expect this to be a failure case rather than "swapping" the current file. If you can provide an example of a widely used path api that does this "swapping" behavior instead of treating this as a failure case, that would help convince me.

"alice/bob#carol".resolve("../martin#dave") == "martin#dave"

I interpret this as: for alice/bob#carol, go to the parent of bob#carol and then find martin#dave in it. I would expect this to resolve to alice/martin#dave, not martin#dave. Given that you have used Node's path.resolve as inspiration, here is how an equivalent path resolves there

Another test case that isn't covered: "alice/bob#carol".resolve("/joe/next#dave"). I would expect this to resolve to /joe/next#dave because it was an absolute path and that will always resolve to itself.

@cart So I agree with most of your points. As is typical with these kinds of cases, a contributor may have a certain timidity when it comes to making breaking changes, whereas a maintainer may have no such limitation :)
We will need to add a predicate function 'isAbsolute' or 'isFullPath' that returns true if a string begins with '/' or 'scheme:/'.
I think the greatest area of contention has to do with the treatment of leaf segments in the base path. Specifically, say I have a base path such as file:/x/y/z, and I have a relative path, './foo'. There are two choices:

Simply concatenate the two paths and strip out the extra './', leaving us with file:/x/y/z/foo. While this might seem the obvious approach, it's actually not what you want most of the time, because 'z' is a file, not a directory. (It might be different if the base path had a trailing slash, but it does not in this case). This also means that in order for a relative path embedded within 'z' to reference file 'foo' in the same directory, you'd have to go up an extra level, e.g. ../foo.

Remove the leaf portion of the base path before concatenating, which yields file:/x/y/foo.

Note that this second behavior is consistent with IETF RFC 1808, which defines the behavior of relative URL paths: https://datatracker.ietf.org/doc/html/rfc1808 . Specifically, in section 4, step 6 it says:

The last segment of the base URL's path (anything following the rightmost slash "/", or the entire path if no slash is present) is removed and the embedded URL's path is appended in its place.

(It then goes on to describe how the path is normalized by removing './' and '../' segments.)

Now that I have actually taken the time to read RFC 1808, I think that it should be the canonical reference for this feature, rather than node.resolve() or any other implementation.

I have updated the PR to change the behavior to more closely conform to RFC 1808. This addresses some of the issues raised by @cart . However, the PR doesn't yet handle the "asset source" syntax, mainly because I don't have a formal definition of what characters are valid in that case.
I suspect that AssetPath is going to need to be updated in several ways to support this. For example, the documentation says that an AssetPath consists of two parts: a path and a label. However, that's no longer true, it's now three parts.
At the very least, I need a function that can extract the "scheme" part of the path, given an &str as input.
Also, the latest changes means that the algorithm can no longer panic. According to RFC 1808, if there are insufficient segments in the base path to match the ".." segments in the relative path, then any leftover ".." elements are left as-is.

Remove the leaf portion of the base path before concatenating, which yields file:/x/y/foo.
Now that I have actually taken the time to read RFC 1808, I think that it should be the canonical reference for this feature, rather than node.resolve() or any other implementation.

I pretty strongly believe that the swapping behavior is going to confuse people. I've spent a lot of time working with "path code" in a variety of OS-es/languages/frameworks/terminals and this is the first time I've encountered this behavior. I believe others will feel the same way. Additionally, I think people will reach for resolve when they want to concatenate paths and the swap behavior will completely defy that expectation. We could have two separate apis, but I personally believe that would just confuse things more. I'd prefer resolve to be the one api people reach for when they want to work within the context of the current path and the "swap" behavior is too unintuitive to make it recommendable for concatenation scenarios (which I believe will be the most common scenario by a wide margin).

Also, the latest changes means that the algorithm can no longer panic. According to RFC 1808, if there are insufficient segments in the base path to match the ".." segments in the relative path, then any leftover ".." elements are left as-is.

I don't dislike this behavior. The alternative (which might be preferable) is to have two variants: resolve and try_resolve, where try_resolve returns Result<AssetPath, ResolveError>. resolve would just call try_resolve and panic on failure (with the helpful ResolveError message). RFC 1808 was written in a context where error messages (and error handling) aren't an option. Idk if we should conform to that at the cost of user experience.

I know we already discussed this on Discord, but for posterity: on the topic of "asset source": that code hasn't been merged yet. It exists in the Multiple Asset Sources PR.

@cart Here's a demonstration. Hover over this link and look at the resulting URL.
The actual link I just typed is ./test.
The resulting URL is https://github.com/bevyengine/bevy/pull/test.
If I were to implement the behavior that you describe, the resulting URL would be https://github.com/bevyengine/bevy/pull/9528/test.
Some other examples:

test
./test
../test
../../test

Im definitely not disputing that this is how the spec works. Only that it is not how most people are trained to think about paths generally in the context of foo, ./foo, and ../foo.
I think if you were to survey Bevy users (or programmers generally) "how do the following paths resolve", you would get the following answers:

"a/b".resolve("foo"): "a/b/foo"
"a/b".resolve("./foo"): "a/b/foo"
"a/b".resolve("../foo"): "a/foo"

Maybe we should just call it join instead to make this behavior clearer? Especially given that resolve has "always resolves to absolute" semantics in places like Node.js?

Although join does have strong "concatenation" vibes whereas resolve implies we're doing something more than that.

I think that the use case you are thinking of is different from the use case I am thinking of.
The fundamental problem I'm trying to solve is embedded paths within assets. I'm not interested in building a generic asset path resolver, except to the extent that it is necessary to solve the former problem. This means that I'm trying to optimize for user expectations around embedded paths, not user expectations around concatenating paths. These are different things, and, as I will argue, have very little overlap in terms of user experience and expectations.
I believe that for most programmers, their experience with embedded paths will derive from working with web technologies. This includes not just HTML hyperlinks, but JavaScript/TypeScript import statements and URLs embedded within CSS. Take for example, the following TypeScript code:
import { MyComponent } from "./MyComponent";

The path "./MyComponent" is resolved relative to the base path. But what is the base path? Is it the JavaScript file, or is it the directory containing the JavaScript file? MDN doesn't say, so let's put this question aside for a moment.
Similarly, in a CSS file:
.button {
  background-image: url(./images/button.png)
}

While not every web developer will have written a relative hyperlink, you would be hard-pressed not to find a web developer that has not embedded an image in a stylesheet, or imported a JavaScript module.
On the flip side, most users who work with filesystems and desktop APIs won't have nearly as much experience dealing with embedded paths, because there are few desktop file formats that contain embedded paths, and those that do are generally not human readable (e.g. Blender files). Contrast with the web, where the vast majority of content formats are both human readable, and can contain links ("hyperlinks") to other content.
Users of desktop APIs do have experience with concatenating paths, and I suspect much more so than web developers, because on the web most path concatenation is automatic and implicit. In JS, for example, it's very rare to call the browser API new URL(path, base), but it's common to call path.resolve() in Node.js.
So my first point is that I believe that embedded paths within assets should behave like embedded URIs on the web. I think that the "user expectations" argument favors this.
However, that alone does not specify how resolve() should work, because we haven't established what the "base path" for an asset is.
I'm assuming that the "base path" for resolving embedded paths is the same as the path to the asset itself, because that is what is currently available in LoadContext. However, in order to make the behavior of embedded paths consistent with the expectations outlines above, we need to resolve embedded paths relative to the directory the asset is in, not the asset itself. There are multiple ways to accomplish this. One way is by making the resolve() algorithm the same as the algorithm for relative URIs, which is what I've implemented. Another way is to manually compute the parent path before resolving, which is possible but cumbersome. A third way is to include the base path in the LoadContext, separate from the asset path.
OK, I'm going to stop here, because I want to know whether we agree on the behavior of embedded paths before we discuss the question of how resolve should work.

Similarly, in a CSS file:

Hmmm ok I see where your head is at. People have definitely been trained to think about paths/uris this way within the context of embedded paths / web resources.
That being said, I think theres a reason the Node.js path API behaves the way it does (without the "swapping" behavior): it is operating within the context of a file system. That is also why it is consistent with "command line apis" and how Rust handles Paths:
fn main() {
    std::fs::create_dir_all("foo/bar/baz");
    std::fs::create_dir_all("foo/baz");

    // prints Ok("/playground/foo/bar/baz")
    println!("{:?}", std::path::Path::new("foo/bar").join("./baz").canonicalize());
    // prints Ok("/playground/foo/baz")
    println!("{:?}", std::path::Path::new("foo/bar").join("../baz").canonicalize());
}

On the web URIs are not a filesystem. Consider https://example.com/foo/bar. bar is assumed to be a webpage / resource that can be resolved.  Likewise, if you go up a level, foo is also considered to be a resource that can be resolved, not a folder. Whenever using a "resolve" api you are guaranteed to be within the context of an actual resource (where it will often not make sense to go deeper down the hierarchy).
On the other hand, AssetPaths are a virtual filesystem.  When you see the AssetPath foo/bar/baz.json, you know with certainty that foo/bar is a folder hierarchy. bar cannot / will not be some arbitrary resource to resolve. When considering a given AssetPath, you are not guaranteed to be within the context of an actual resource ... you might be in a folder. I think it makes sense to employ the filesystem mindset to such an API.
I think we can reasonably cater to both use cases by making resolve behave "filesystem like" (making it useful and intuitive outside of the context of "embedded asset paths" ... remember that not all AssetPaths are local references in the context of an asset file). Then for "embedded cases" we can just use the parent as the base path:
// unwrap is necessary for AssetPath because not all paths will have a parent
load_context.asset_path().parent().unwrap().resolve("./foo");
// we can add an accessor. Internally asset_path.unwrap() is safe because the asset's path cannot be empty
load_context.parent_asset_path().resolve("./foo")

Note that Multiple Asset Sources makes it possible to express folder-only AssetPaths and adds support for asset_path.parent().

There's one case where calling 'parent' on the base path won't work, which is the case where the relative path begins with '#', meaning that we want to keep the exact same path, but replace the label. I expect this will be a very common case for asset files that have internal references.
Anyway, perhaps I should hold off on this PR until multiple asset sources are merged.
Note that the changes we are discussing are just one line of code (other than fixing all the unit tests).
We can also discuss other name choices besides "resolve" (e.g. join / concat). Although traditionally, "join" has meant concatenation without canonicalization, in other words all of the ".."s are left in. We definitely don't want or need that behavior.
I chose 'resolve' because that's the name used in Node.js, but I now think that was a mistake, because the behavior that I implemented is not the same. (Also node.js resolve takes a variable number of arguments.)

I realized that what I said earlier was incorrect, when I said that there were several ways to implement the "relative import" behavior being discussed. In fact, there is only one way, and that's the RFC 1808 algorithm. The other methods that I suggested (specifically, using the parent of the asset path as the base path) won't work. Specifically, they will work for some cases (./foo) but not others (#foo). The only way these can work is if the base path is the full asset path. My apologies for confusing the issue.
So, the question is how to resolve the impasse. I'm convinced that we need the RFC 1808 behavior for embedded asset paths in order to meet user expectations. However, I want to define this narrowly, and not conflate the issue of resolving embedded asset paths with the more general operation of path joining or concatenation. As you point out, there are other use cases for concatenating asset paths, which may have different expectations. So I'm OK with restricting the RFC 1808 behavior to only apply to this type of path calculation, assuming we can find an ergonomic way to do that.
I also realize that the name "resolve" may be problematic, because it connotes a similarity to the path.resolve() function of Node.js, which does have the behavior you are advocating, and therefore may cause some user confusion.
So I think that we need to come up with a better name, one that doesn't suggest that it's a general path concatenator. Unfortunately, I don't have any good ideas at the moment. I thought about .with_suffix() or .with_stem() or other with_ combinations, but those aren't precise or accurate. (As Mark Twain said, "Use the right word, not it's second cousin.") Other words such as combine, join, concat, compose, and conjoin imply more generality than we want.

Hmmm ok the #label case does probably justify a second API. Otherwise people building assets like this will need to do something like:
if is_label(value) {
    load_context.parent_asset_path().resolve(value)
} else {
    load_context.asset_path().resolve(value)
}

In addition to being boilerplatey and error-prone, it also would have weird corner case behaviors for things like ...
Consider me convinced that there is real justification in having a separate API that behaves this way! We do want to be able to cover cases like this consistently and cleanly. Thank you for holding your ground :)
I guess all that remains is picking a names for both apis: the "Node.js resolve" and the "RFC 1808 resolve" (note that the RFC does use "resolve" terminology internally). I believe we need to implement both and that they should be implemented together. They are almost identical, they should be designed + documented in a way that is aware of the other, and I'd like to avoid the situation where only one is available and someone looking for the other behavior is forced to use something almost correct.
Given that they will behave identically, other than the "local swapping behavior", I think it might make sense to use the resolve term in both cases.
I think my preference is for:

resolve: Rust/Node.js concat-style resolve
resolve_swap or resolve_local: RFC 1808 swap-style resolve

I don't have strong preferences for "swap" vs "local". They both describe important parts of the behavior.

This does a number of things:

It pairs the functionality together (in docs and autocomplete) and encourages the user to think about their choices
It gives the concat-style resolve precedence/preference by giving it the shorter name. I believe this is the better "default" as it is less surprising when doing asset-path ops in code (which will almost certainly be the more common use case).
It gives the concat-style resolve name parity with its counterpart in Node.js
It leaves the doors open for another join or concat api without the more complex "resolve" behaviors

How about "resolve_embed"?

In any case, I'll get to work on this, I have to rebase for multi-asset sources anyway.

I down for "embed". I especially like that its used front-and-center in the RFC.

OK I've updated to the latest head, resolved the conflicts, and coded two versions, resolve, and resolve_embed, both of which use a common implementation, resolve_internal.
I also changed the logic a little bit such as that ".." segments are now treated the same in both the base path and the relative path (this is more consistent, the reason for not doing it before is that I was lazy).
 canonicalization for you, for example if it sees something that looks like a windows drive letter. I don't like the idea that there could be unexpected behavior.
The one thing I have not done yet is added support for detecting asset sources as absolute paths. I'm trying to decide if I should leverage AssetPath::parse_internal or not. If I do, then I have to consider how to handle errors. Right now, resolve can never fail, but if I use parse_internal then I'll have to return a Result if the relative path is an invalid path.

BTW, Path and PathBuf are starting to scare me, they do a lot of stuff under the hood, like ignore trailing slashes, recognize backslashes, windows drive letters and such. I'm beginning to think that at some point (not now) we should consider creating our own implementation of this so that we can ensure that the behavior of asset paths are the same on all platforms - basically convert to Path only after all of the Bevy-specific path manipulations have been done.

Updated the PR. It now understands asset sources - any path string that begins with a source is considered absolute. This uses parse_internal under the hood.
AssetPath::resolve now can fail if the path syntax is invalid. This check may become more struct as the parse function evolves.
In this latest version, the leading slash / is stripped from the path argument - meaning that it is still considered absolute, but the slash is no longer present in the output. This is because all AssetPaths have a source (either a named source or the default) and a source always implies absoluteness. If this were not the case, then we'd have to use three slashes to indicate a truly absolute path source:///path.
Later we can introduce an AssetSourceId::Relative to allow for relative asset paths.

BTW, Path and PathBuf are starting to scare me, they do a lot of stuff under the hood, like ignore trailing slashes, recognize backslashes, windows drive letters and such. I'm beginning to think that at some point (not now) we should consider creating our own implementation of this so that we can ensure that the behavior of asset paths are the same on all platforms - basically convert to Path only after all of the Bevy-specific path manipulations have been done.

I'm very cool with this / feel similarly. Imo this would probably mean:

Implement our own Path-like thing (or just use String and str)
Use that directly in AssetPath
Also use that in the public AssetReader/Writer interfaces and only convert to path inside those implementations when the underlying impl (ex: filesystem apis) require it.

There are a number of refinements we can make...but "better is the enemy of good enough". I think we have convergence on the main points of the API, there's only nits left to pick.
BTW, I did not take your suggestion of adding a "try_resolve" - the reason is that AssetPath has enough methods already, and most of the time they should be handling the error - adding a panic version just saves them a call to unwrap(), something which I would discourage using when dealing with data read from files.

I added some doc examples (pictures are worth a thousand words) and resolved my resolve_internal comments.

@cart Release blurb: given that it's not actually used yet, I'm not sure it needs a release blurb. However, if you want one:
"AssetPath now has methods for resolving relative paths. This allow a group of linked assets containing embedded references to be moved around in the filesystem without breaking the links. Asset loaders can voluntarily choose to use these new methods."


Can't subscribe to YepAdvancementEvent

I'm trying to listen for YepAdvancementEvent but getting this error in console:
java.lang.ClassCastException: Cannot cast cc.unilock.yeplib.api.event.YepMessageEvent to cc.unilock.yeplib.api.event.YepAdvancementEvent

example of code:
@Subscribe
public void onYepAdvancement(YepAdvancementEvent event) {
    logger.info("Handling YepAdvancementEvent: {}", event.toString());
}

@Subscribe
public void onYepDeath(YepDeathEvent event) {
    logger.info("Handling YepDeathEvent: {}", event.toString());
}

That's strange; that should error should have been fixed as of https://github.com/unilock/YepLib/commit/ee3ebe3234ad79650cb22a553bcd79a55d90c299 (v2.1.0 and above).
I'm honestly not sure what the issue could be outside of making sure you're using the latest version of YepLib. (I asked about it in PaperMC's support Discord; even they have no idea what's going on...)

Looked in the logs, everything is working, it's just that for some reason velocity is sending all yep events to handlers that expect specific ones

This should be fixed with 2.3.0, god willing


[py2py3] Python3 does not have an 'exceptions' module

Impact of the bug
WMCore in general
Describe the bug
Some of the python3 unit tests crash when importing exceptions module, see the full traceback in [1]
How to reproduce it
nosetests test/python/WMCore_t/WMSpec_t/Steps_t/Executors_t/LogArch_t.py

Expected behavior
Exception handling should be functional and consistent in python2 and python3.
Additional context and error message
[1]
======================================================================
ERROR: testUnitTestBackend (WMCore_t.WMSpec_t.Steps_t.Executors_t.LogArch_t.LogArchiveTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dmwm/wmcore_unittest/WMCore/test/python/WMCore_t/WMSpec_t/Steps_t/Executors_t/LogArch_t.py", line 119, in setUp
    taskMaker.processWorkload()
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMSpec/Makers/TaskMaker.py", line 127, in processWorkload
    sandboxCreator.makeSandbox(self.workdir, self.workload)
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMRuntime/SandboxCreator.py", line 108, in makeSandbox
    fetcherInstances = list(map(getFetcher, fetcherNames))
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMSpec/Steps/StepFactory.py", line 196, in getFetcher
    return _FetcherFactory.loadObject(fetcherName)
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMFactory.py", line 58, in loadObject
    module = __import__(module, globals(), locals(), [classname])
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMSpec/Steps/Fetchers/PileupFetcher.py", line 17, in <module>
    from WMCore.Services.DBS.DBSReader import DBSReader
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/Services/DBS/DBSReader.py", line 9, in <module>
    from WMCore.Services.DBS.DBSErrors import DBSReaderError
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/Services/DBS/DBSErrors.py", line 4, in <module>
    from WMCore.Services.DBS.ProdException import ProdException
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/Services/DBS/ProdException.py", line 11, in <module>
    import exceptions
ModuleNotFoundError: No module named 'exceptions'

I remember having already encountered something like this. I can look for it, but if I am not mistaken simply removing the import and using directly Exception, which being a built-in should be readily available without any explicit import, should be enough.
Otherwise, I think that the solution provided by this answer in stackoverflow should work

grepping through the code I actually found few more imports of exceptions [1], but reading the code they seem to be just imported but never used... So I am fixing those too.
[1]
:WMCore$ grep -rin -E "(.*import.*exceptions|.*exceptions.*import)" --exclude=*~ --exclude=*.pyc src/python/ test/ |grep -i -v "from WMCore" |grep -v "import HTTPError" |grep -v "from dbs.exceptions" |grep -i exceptio
src/python/WMCore/Services/DBS/ProdException.py:11:import exceptions
src/python/WMCore/WMSpec/Makers/Handlers/MakeJob.py:14:import exceptions
src/python/WMCore/WMSpec/Makers/Handlers/MakeJobSlave.py:29:import exceptions
src/python/WMComponent/DBS3Buffer/Oracle/ListRunsWorkflow.py:11:import exceptions

Once I've passed beyond that error, I just hit another one... [1]. For which I have a vague memory about having it solved before, but for sure do not remember where and in what context.
FYI @amaltaro
[1]
======================================================================
ERROR: testOnWorkerNodes (WMCore_t.WMSpec_t.Steps_t.Executors_t.LogArch_t.otherLogArchiveTexst)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dmwm/wmcore_unittest/WMCore/test/python/WMCore_t/WMSpec_t/Steps_t/Executors_t/LogArch_t.py", line 288, in setUp
    taskMaker.processWorkload()
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMSpec/Makers/TaskMaker.py", line 127, in processWorkload
    sandboxCreator.makeSandbox(self.workdir, self.workload)
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMRuntime/SandboxCreator.py", line 108, in makeSandbox
    fetcherInstances = list(map(getFetcher, fetcherNames))
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMSpec/Steps/StepFactory.py", line 196, in getFetcher
    return _FetcherFactory.loadObject(fetcherName)
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMFactory.py", line 58, in loadObject
    module = __import__(module, globals(), locals(), [classname])
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/WMSpec/Steps/Fetchers/PileupFetcher.py", line 18, in <module>
    from WMCore.Services.Rucio.Rucio import Rucio
  File "/home/dmwm/wmcore_unittest/WMCore/src/python/WMCore/Services/Rucio/Rucio.py", line 17, in <module>
    from rucio.client import Client
  File "/home/dmwm/unittestdeploy/wmagent/1.4.9.pre3/sw/slc7_amd64_gcc630/external/py3-rucio-clients/1.24.2.post1-comp/lib/python3.8/site-packages/rucio/client/__init__.py", line 20, in <module>
    from .client import Client  # noqa: F401
  File "/home/dmwm/unittestdeploy/wmagent/1.4.9.pre3/sw/slc7_amd64_gcc630/external/py3-rucio-clients/1.24.2.post1-comp/lib/python3.8/site-packages/rucio/client/client.py", line 30, in <module>
    from rucio.client.accountclient import AccountClient
  File "/home/dmwm/unittestdeploy/wmagent/1.4.9.pre3/sw/slc7_amd64_gcc630/external/py3-rucio-clients/1.24.2.post1-comp/lib/python3.8/site-packages/rucio/client/accountclient.py", line 39, in <module>
    from rucio.client.baseclient import BaseClient
  File "/home/dmwm/unittestdeploy/wmagent/1.4.9.pre3/sw/slc7_amd64_gcc630/external/py3-rucio-clients/1.24.2.post1-comp/lib/python3.8/site-packages/rucio/client/baseclient.py", line 66, in <module>
    from rucio.common.utils import build_url, get_tmp_dir, my_key_generator, parse_response, ssh_sign, setup_logger
  File "/home/dmwm/unittestdeploy/wmagent/1.4.9.pre3/sw/slc7_amd64_gcc630/external/py3-rucio-clients/1.24.2.post1-comp/lib/python3.8/site-packages/rucio/common/utils.py", line 107, in <module>
    imp.find_module(extra_module)
NameError: name 'imp' is not defined

----------------------------------------------------------------------

Thanks @mapellidario


Metrics with Prometheus

Idea behind this change is described in https://github.com/badges/shields/issues/2068
Initially this change contains:

/metrics resource without access limit with default metrics (disabled by default)
METRICS_PROMETHEUS_ENABLED environment variable to enable metrics

Warnings

:warning:

This PR modified the server but none of the service tests. That's okay so long as it's refactoring existing code. Otherwise, please consider adding tests to the service: How-to

:warning:

This PR modified helper functions in lib/ but not accompanying tests. That's okay so long as it's refactoring existing code.

Messages

:book:

:sparkles: Thanks for your contribution to Shields, @platan!

:book:

Thanks for contributing to our documentation. We :heart: our documentarians!

  Generated by :no_entry_sign: dangerJS

I would like to have tests for this. But I do not know is it possible to run service tests with config.metrics.prometheus.enabled set to false and true.
My first attempt:
services/metrics/metrics.tester.js
'use strict'

const ServiceTester = require('../service-tester')

const t = new ServiceTester({ id: 'metrics', title: 'Prometheus metrics', pathPrefix: '' })
module.exports = t

t.create('metric are disabled by default')
  .get('/metrics')
  .expectStatus(404)

// below test will fail, we have to set config.metrics.prometheus.enabled = true
// const config = require('../../lib/server-config')
// config.metrics.prometheus.enabled = true

t.create('metric can be enabled by feature flag')
  .get('/metrics')
  .expectStatus(200)
  .expectBodyContains('nodejs_version_info')

Both tests requires change in services/service-tester.js (we have response which is not a JSON)
diff --git a/services/service-tester.js b/services/service-tester.js
index 91223be..17009af 100644
--- a/services/service-tester.js
+++ b/services/service-tester.js
@@ -62,12 +62,18 @@ class ServiceTester {
       })
       // eslint-disable-next-line mocha/prefer-arrow-callback
       .finally(function() {
+        let content;
+        try {
+          content = JSON.parse(this._response.body)
+        } catch(e) {
+          content = this._response.body;
+        }
         // `this` is the IcedFrisby instance.
         trace.logTrace(
           'outbound',
           emojic.shield,
           'Response',
-          JSON.parse(this._response.body)
+          content
         )
       })

I've tried with separate test files for both cases, but server is started only once for all tests.
@paulmelnikow maybe you have an idea how to test both cases?

The approach I'd probably take to testing this is to test it with a lower-level unit test. I just did something similar for the GitHub token acceptor in https://github.com/badges/shields/pull/2021/files#diff-e852a48eb7b4b8dcdf24b0bb30321a60.
The module exposes a function setRoutes, which gets a camp instance and adds routes to it. The unit test creates a new server, invokes setRoutes, and then fires some requests at it using got or fetch.
We have to trust that the wiring in server.js works, and don't get coverage of the config option, though that code is simple enough that it seems like a decent tradeoff.
There's some similar code for the github admin route and its test:
https://github.com/badges/shields/blob/master/services/github/auth/admin.js
https://github.com/badges/shields/blob/master/services/github/auth/admin.spec.js

I've refactored the code and I've added an access limit. @paulmelnikow thanks for the hints about testing! :-)

Can we merge it?

Let's merge it! I'll make sure @espadrine is aware of it before we turn it on in production.


Brand footer separator changed to hide from screen readers

Summary
Ref: #280. Moved the '/' separator out of CSS content to hide it from screen readers.
Closes #280
Additional Details
Added a theming variable the Brand Footer CSS to allow the separator to be re-colored.

Tested in VoiceOver on Mac and iOS as well as JAWS 2018 in IE 11, separators were not read by the screen readers. Nice work on this @amichaelparker!


Fix Ocelot release from github

https://ci.appveyor.com/project/TomPallister/ocelot-ayj4w

Fixed can now download 1.4.20 from nuget


Authorization does not work

Using this small sample code, I receive an error the the client is unauthorized at the nsq.pub call:
nsq = await open_connection(host=host, port=tcp)
await nsq.auth("writer:12345")
await nsq.pub("sample.topic7", "Hi, sample message!")

This can by setting nsq._is_authorized = True after the second line. I could not find any code that actually sets this variable. I guess this is not how it is designed to work.

Thanks for issue, i'll take a look at this bug next week.

AUTH does not yet implemented in ansq.


fix: convert runtime to ResponseStatusException to get status

Pull request checklist

[x] Please read our contributor guide
[ ] Consider creating a discussion on the discussion forum
first
[x] Make sure the PR doesn't introduce backward compatibility issues
[ ] Make sure to have sufficient test cases

Pull Request type

[ ] Bugfix
[ ] Feature
[x] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Other (please describe):

Changes in this PR
GraphQLClientException to extend ResponseStatusException instead of RuntimeException to solve for use cases which would require status code. (Some status codes need to have re-try implemented and some don't)
Describe the new behavior from this PR, and why it's needed
Issue #
No breaking changes. Client would have explicit access to status Code now
Alternatives considered
Describe alternative implementation you have considered
We can parse the message from GraphQLClientException and interpret status code but that doesn't seem like a clean solution

Thanks for the PR. Could you please rebase from master?

Never mind, I miss read the merge status.


Fix for BTS-852.

Scope & Purpose
3.10 backport for https://github.com/arangodb/arangodb/pull/16675

[x] :hankey: Bugfix
[ ] :pizza: New feature
[ ] :fire: Performance improvement
[ ] :hammer: Refactoring/simplification

Checklist

[ ] Tests

[ ] Regression tests
[ ] C++ Unit tests
[ ] integration tests
[ ] resilience tests

[x] :book: CHANGELOG entry made
[ ] :books: documentation written (release notes, API changes, ...)
[ ] Backports

[ ] Backport for 3.10: (Please link PR)
[ ] Backport for 3.9: (Please link PR)
[ ] Backport for 3.8: (Please link PR)

Related Information
(Please reference tickets / specification / other PRs etc)

[ ] Docs PR:
[ ] Enterprise PR:
[x] GitHub issue / Jira ticket: https://arangodb.atlassian.net/browse/BTS-852
[ ] Design document:

Only known unrelated failure in Jenkins.


Fix - Remove sms domain in htmlToText

Fixed Issues
$ https://github.com/Expensify/App/issues/47484
Tests

What unit/integration tests cover your change? What autoQA tests cover your change?
What tests did you perform that validates your changed worked?
Test:
Make the same update on userMention rule as in the current pr in node-modules
Now Open Expensify App
Tap on a report
Enter any phone mention
Send the message
Long press the message and copy to clipboard
Tap plus icon -- assign task
Paste it in title field
Verify sms domain doesn't exist in the text pasted

QA

What does QA need to do to validate your changes?
same as above
What areas to they need to test for regressions?
same as above

@aldo-expensify Thx Updated tests.

@ZhenjaHorbach PR ready for review

@aldo-expensify Thx Updated tests.

Sorry, I meant adding automated test :P


Changing the Throughput starting status

Is there a way to change the status that drives the Through Put currently its based on the ticket creation date but would like to trigger this when the ticket has been set to our internal Development status, is this possible?

You would have to use one of the queries that brings back all the work item history then take the timestamp of when an item first hits that particular "status". This isn't something I will be looking to create but could share brief (free) guidance on how to go about it.


How to implement Rijndael 256-bit block size and 256-bit key size

First of all, thanks so much for your open source code tiny AES. you made my life so much easier when dealing with it in C.
My knowledge in C/C++ is bad so i have to find open source code like this
I know this is not the best place to ask questions but i dont know how to do this. My problem at work is that they are not using AES but rather a Rijndael 256-bit block size and 256-bit key size. how can i implement Rijndael 256-bit block size in your code? what variable or constant that i have to change in order to do that?
Thanks so much for your work

Hi @bettafish15 and thanks for the compliments :)
AES is a version of Rijndael with 128-bit block size and 128/192/256 bit key size.
Unfortunately that means it won't easily convert to 256-bit Rijndael. You could hack the code if you are up for the task - but there is not a single flag or something similar you could change.
Over the years, the code has been improved for efficiency and smaller code size. Sometimes that has meant specializing some operations or unrolling a loop. The earliest versions of this project may be an easier fit for a conversion to 256-bit block size, at least compared to the current HEAD.
Sorry to disappoint you :(
If you decide to attempt the conversion, feel free to ask follow-up questions here.

Hi @kokke thanks for your response! i am having no others choice but hacking your code. Can i have the earliest version of this project?

Hi @bettafish15 - sure you can :)
I just checked the commit-history, and the code is a lot simpler in the first versions.
Check the files from around 2014, about here or so https://github.com/kokke/tiny-AES-c/commits/master?after=3f69a5899e58e2e398e8c32ce7b3a954dd593ed4+244&branch=master
The code from 2014 is easier to read and understand, but with fewer modes of operation available (initialt ECB only) and much fewer optimizations.
A major change should be something about changing from working on a 16 byte / 128 bit state -> 32 byte / 256 bit state.
If you are comfortable with the maths involved in Rijndael, the wiki page for the Rijndael algorithm might be helpful.

Thanks @kokke for your response! Luckily i am familiar with the operation in encrypting of Rijndael. It's just C/C++ coding. I will try out with your suggestions and hope that performance is not an issue :D

It definitely helps if you understand Galois-fields and how the arithmetic works there. Then you "only" have to struggle with the C/C++ implementation :+1:
If performance gets problematic, there are other much faster implementations of AES you could look into. A typical trick is to use lookup-tables, which takes up RAM but speeds up calculations (e.g. this section of the Rijndael-article on wikipedia).
Good luck @bettafish15 and let me know if you need help debugging the C/C++ code you end up with :)


Remove client00 from Mibbit URLs

By not going to client00 directly, users go through a load balancer that can put them on any available client server. It's not always guaranteed at client00 will be available.
(I'm an oper on the Mibbit network, I know these things.)

Thanks @Havvy !