Add parallel gzip support controlled by a daemon flag

Signed-off-by: Jon Botelho<EMAIL_ADDRESS>
- What I did
Improved image push performance for large images on fast networks, where compression is the bottleneck.
- How I did it
Added a --compression-threads flag on the daemon, which gzip compression pgzip for multi-threaded compression of the push of --compress-threads is set greater than 1 (eg. --compression-threads=2). When you set --compression-threads=1, it will fall back to the original behavior of using compress/gzip. This is pretty much a continuation of the work and discussion in #17674.
I'm new to this project, so I'm not sure the way I've implemented this flag is ideal. I could also use some guidance on what kind of test coverage this change would need.
- How to verify it
Start up the daemon in debug mode with --compression-threads=2 (or above), and push an image. Observe that it pushes correctly, and observe that the log mentions its using pgzip.
- Description for the changelog

TBD
- A picture of a cute animal (not mandatory but encouraged)
Containerized dog:

/cc @stevvooe @unclejack

Looks like build is failing
03:56:25 daemon/config.go:134: struct field tag `json:parallel-gzip,omitempty` not compatible with reflect.StructTag.Get: bad syntax for struct tag value

Do you have a benchmark result? (as in https://github.com/docker/docker/pull/17674#issuecomment-257680358 )

Also, can we control concurrency? ( https://github.com/docker/docker/pull/17674#issuecomment-261369407 )

@AkihiroSuda the flag is in, but I'll have to do some benchmarks tomorrow when I have some more time.
@thaJeztah is there a good way to replicate that build locally? I fixed the first error, but its complaining about my vendored library now:
00:01:18.181 distribution/push.go:14:2: cannot find package "github.com/klauspost/pgzip" in any of:
00:01:18.181 	/go/src/github.com/docker/docker/vendor/github.com/klauspost/pgzip (vendor tree)
00:01:18.181 	/usr/local/go/src/github.com/klauspost/pgzip (from $GOROOT)
00:01:18.181 	/go/src/github.com/klauspost/pgzip (from $GOPATH)

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "parallel-gzip" git@github.com:jbotelho2-bb/docker.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354252872
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

I think you did not use vndr (seems submoduled )

This would also require that we control the number of threads used. It's not a good idea to be able to choose only between a single thread and all cores.

That is how it works with the latest changes; you control the number of threads. Just updated the pull request description to reflect this.

I collected some benchmarks for image push using the new flag, using this script: https://gist.github.com/jbotelho2-bb/827cd5d00149b8f87d2ccd925a658c7f
Push time by layer size

Threads
10MB
100MB
500MB

Default
1.03s
4.89s
21.0s

2 threads
0.90s
2.12s
7.44s

4 threads
0.85s
2.13s
6.78s

*Each time is the average of ten test runs

Should the windowsRS1 build be passing? It seems that the error in the build log is unrelated to my changes, but I'm not too familiar with these tests:
00:25:13.472 FAIL: docker_cli_build_test.go:380: DockerSuite.TestBuildLastModified
00:25:13.472
00:25:13.472 docker_cli_build_test.go:395:
00:25:13.472     out, _ = dockerCmd(c, "run", name, "ls", "-le", "/file")
...
00:25:13.475 ... expected cmd.Expected = cmd.Expected{ExitCode:0, Timeout:false, Error:"", Out:"", Err:""}
00:25:13.475 ...
00:25:13.475 Command: d:\CI\CI-579c9ea38\binary\docker.exe run testbuildlastmodified ls -le /file
00:25:13.475 ExitCode: 1, Error: exit status 1
00:25:13.475 Stdout:
00:25:13.475 Stderr: BusyBox v1.27.0-FRP-1035-g74163a5 (2017-02-09 08:42:39 GMT) multi-call binary.
00:25:13.475
00:25:13.475 Usage: ls [-1AaCxdLHRFplinshrSXvctu] [-w WIDTH] [FILE]...
...
00:25:13.477 C:/busybox/busybox.exe: unknown option -- e

The number of threads should be validated. The default number of threads is supposed to be 1 to use the standard gzip implementation. The maximum number of threads shouldn't be higher than GOMAXPROCS. pgzip might perform its own validation of that value. We should perform our own validation to ensure everything is fine and to catch the error before we attempt to use pgzip.
I have a few concerns and questions related to this PR:

have you tested with a remote registry or with Docker Hub? at what point does it make sense to use more threads?
what about using too much CPU on the host where the Docker daemon runs? Have you tested to see how the high CPU usage on the Docker daemon affects the other workloads running on that host?
how thoroughly tested is this library? have you run into any problems with it? have you tested this code with huge layers?
does pgzip have any known gotchas or issues?

I've asked these questions because we've had support for something other than gzip for layers in the past. It was a major issue and we had to get rid of all images using that compression tool over time. I'd like to make sure we do the right thing from the beginning this time.

That validation makes sense. I can add that.
For the other concerns:

The benchmarks were done with a local docker registry, but I also tested with a remote registry on an internal network (Artiactory Pro). I've observed that the gains are relative to how fast the network is, but I can get some more exact info on the trade-off.
Figuring out how much this affects other processes on the system seems very subjective. What's an acceptable level of disruption? Should I test this on a very slow computer like a raspberry pi and ensure it's usable?
I'm not sure what would be considered as a huge layer, but the benchmarks tested it with several rounds of 500MB layers. I can go higher if you'd like. In general, I'm not in a great position to say how reliable the library is, but I'm sure @klauspost could chime in.
As far as I know, the only "gotcha" with pgzip is that the performance can be the same or slightly worse than the standard gzip library for very small layers, but it seems to be very minor since small layers are still fast on both systems.

Those are all valid concerns, and I can understand wanting to be cautious. However, keep in mind this isn't a different compression type so much as a different implementation of it, so it's unlikely to result in things needing to be removed from the registry. Could you link to some info on these past issues, for reference?

ping @unclejack @akihirosuda PTAL

The maximum number of threads shouldn't be higher than GOMAXPROCS.

That is not strictly true. There is likely an advantage of having the number of compressors > GOMAXPROCS. It will allow the compressor to keep working, if different blocks have different compression time. This will allow more in-flight blocks and give less blocking upstream. The only downside is bigger memory requirement.
So to keep the overall system impact small, staying <= GOMAXPROCS is good, but if you want fastest performance going above is fine.

As far as I know, the only "gotcha" with pgzip is that the performance can be the same or slightly worse than the standard gzip library for very small layers.

Correct. Huge layers are good. Small layers (< 1MB or so) will have little or no benefit. But they should still be fast and work fine.

That is not strictly true. There is likely an advantage of having the number of compressors > GOMAXPROCS. It will allow the compressor to keep working, if different blocks have different compression time. This will allow more in-flight blocks and give less blocking upstream. The only downside is bigger memory requirement.

Docker runs these push operations in the same long lived process. Compression speed is certainly not the best with a single process. The Docker daemon needs to do other things which aren't related to this compression and this is something we need to keep in mind.
There have been issues in the past which were pretty hard to debug by users (developers/administrators). Some of these issues were related to seeing CPU usage spike to 100% in the Docker daemon or extremely high memory usage. Nobody was doing something on purpose to introduce those issues either. This is why I'd like to see this limited to at least GOMAXPROCS.
Some machines have a lot of cores and threads. Nobody is looking to saturate 16/32+ CPU threads with compression processes for Docker images which are pushed.
Please give me a few days to run some tests and to provide some more useful feedback. I'll assign this to myself as well to be able to keep track of this.

@unclejack is there way to move this forward?

Some changes need to be made before we can implement something such as this. The issue is that the content addressability is affected by the use of gzip vs pgzip /w 2 threads vs pgzip /w 3 threads and so on. pkg/gzip will always produce the exact same gzip layer. pgzip with varying number of threads will produce different gzip archives. This is an issue because different machines produce different layers depending on the number of threads used during compression.
This means that we need to decouple this compression from the image layers themselves. It shouldn't matter whether they're compressed using gzip or pgzip in that case.
pgzip has much better performance than gzip. It might not be fast enough for certain systems which are slower (ARM, for example). It's also very likely some architectures might have special platform specific optimized gzip implementations which should have better efficiency (used resources/performance ratio, performance/power consumption).
It's most likely a good idea to use a different compression algorithm for huge layers.
Merging this PR before we make these changes would cause a bunch of issues. It'd be better to wait until we have implemented the changes needed to decouple the compression from the image layers and other related optimizations. This should enable us to implement these changes while ensuring everyone is able to enjoy these optimizations without breaking anything for anyone. Please keep in mind that I'm one of the first to want to implement optimizations and to improve performance. I'm saying this is the best path forward because optimizations must be implemented only when they don't take away safety and compatibility.

@unclejack is there any work currently happening towards this direction of de-coupling the compression from the layer IDs, or is this something new?
It seems like in theory, you should be able to address the uncompressed content, leaving the compression out of the equation. However, I'm not too familiar with the decisions that went into the current system.

So it seems like we need to close this for now as it is not able to be merged.

Let's close for now


Implemented diff for 'to have attributes' + to have class(es) assertion

/cc @sunesimonsen who came up with the idea of doing the diff like this.
Next thing I'd like to look into is a to satisfy for HTMLElement, but this PR is already too big.

Coverage increased (+2.04%) to 90.22% when pulling 14d30c9a618db1aebda72ce23a03a8aa8733fb57 on papandreou:feature/toHaveAttributesDiff into 4e4b9329fa971f81c580ad41852b7e07c218ca98 on Munter:master.

There are parts in here I don't quite get. Probably since I haven't been massaging custom output before and haven't followed along with the promises implementation in unexpected. Looks good though, so I'll merge

Published in v0.8.0

and haven't followed along with the promises implementation in unexpected

Yeah, when you cannot delegate directly to to satisfy, you need to reimplement parts of it. That includes supporting promise-returning expect.its everywhere.


If the wallet does not provide a SorobanRpcUrl we should handle the error

Currently, if the user does not have correcly set his network, and does not provide a SorobanRpcUrl, the user cannot sign the tx and does not know what is happening.
We should handle this error and pass the error to the frontend so the frontend shows a message to the user

This PR #52 is throwing an error if the soroban rpc url is not set. The error still needs to be handled in the frontend for better ui

@benjaminsalon please make that the boilerplate handles this error and then help us in soroswap to handle it the same way: https://github.com/soroswap/frontend/issues/131

Ok @esteblock !


cannot execute exe of tensorflow py compiled with pyinstaller

System information

I used the basic example of tensorflow installing procedure:
 import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Windows 10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
I don't know
TensorFlow installed from (source or binary):
installed via pip. pip install tensorflow and pip install --upgrade tensorflow
TensorFlow version (use command below):
2.4.0
Python version:
Python 3.8.7rc1
Bazel version (if compiling from source):
Not compiled from source
GCC/Compiler version (if compiling from source):
Not compiled from source
CUDA/cuDNN version:
Tried:
1- Without CUDA
2- CUDA 11.1, no CUDNN
3- CUDA 11.1,  CUDNN 8.1
4- CUDA 11.1,  CUDNN 8.1, CUDA 10.1 over CUDA 11.1
Same result
GPU model and memory:
NVIDIA GeForce MX250

Describe the current behavior
I run python with the script:
import tensorflow as tf
print(tf.reduce_sum(tf.random.normal([1000, 1000])))

The right output is:
tf.Tensor(-29.82965, shape=(), dtype=float32)
I launch pyinstaller --onefile prova.py and this is the output:
C:\Users\fabio\VSC>pyinstaller --onefile C:\Users\fabio\VSC\prova.py
103 INFO: PyInstaller: 4.1
103 INFO: Python: 3.8.7rc1
104 INFO: Platform: Windows-10-10.0.19041-SP0
106 INFO: wrote C:\Users\fabio\VSC\prova.spec
108 INFO: UPX is not available.
108 INFO: Extending PYTHONPATH with paths
['C:\Users\fabio\VSC', 'C:\Users\fabio\VSC']
131 INFO: checking Analysis
735 INFO: checking PYZ
1053 INFO: checking PKG
1061 INFO: Building because C:\Users\fabio\VSC\build\prova\prova.exe.manifest changed
1069 INFO: Building PKG (CArchive) PKG-00.pkg
22364 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
22370 INFO: Bootloader c:\windows\system32\venv\lib\site-packages\PyInstaller\bootloader\Windows-64bit\run.exe
22373 INFO: checking EXE
22374 INFO: Rebuilding EXE-00.toc because pkg is more recent
22376 INFO: Building EXE from EXE-00.toc
22383 INFO: Appending archive to EXE C:\Users\fabio\VSC\dist\prova.exe
22465 INFO: Building EXE from EXE-00.toc completed successfully.
Then I run the exe and this is the output:
(venv) C:\Users\fabio\VSC>dist\prova.exe
Traceback (most recent call last):
File "prova.py", line 1, in
import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))
File "c:\windows\system32\venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
exec(bytecode, module.dict)
File "tensorflow_init_.py", line 41, in
File "c:\windows\system32\venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
exec(bytecode, module.dict)
File "tensorflow\python_init_.py", line 41, in
File "c:\windows\system32\venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
exec(bytecode, module.dict)
File "tensorflow\python\eager\context.py", line 35, in
File "c:\windows\system32\venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
exec(bytecode, module.dict)
File "tensorflow\python\pywrap_tfe.py", line 29, in
ImportError: DLL load failed while importing _pywrap_tfe: Impossibile trovare il modulo specificato.
[20820] Failed to execute script prova
Describe the expected behavior
The exe file should return the tensor as the python script
Standalone code to reproduce the issue
Create a python script prova.py:
import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))
Run in the directory of the script
pyinstaller --onefile prova.py
Then run the exe resulting
How can I build a onefile exe from a script using tensorflow?
Thanks.

@fafabone
What is make/model of your cpu?
I suspect your cpu model does not support AVX instructions sets.See hardware requirements
Make sure to download the latest microsoft visual c++ redistributable from here.
.Also, please follow the instructions from to install from Tensorflow website.
Please, check Your CPU/Python is on 32 bits?Please, refer #36167
Thanks!

My CPU is Intel® Core™ i7-10510U Processor and it supports AVX 2 instruction set:
https://ark.intel.com/content/www/us/en/ark/products/196449/intel-core-i7-10510u-processor-8m-cache-up-to-4-90-ghz.html#:~:text=Instruction Set Extensions,-Instruction Set Extensions&text=These can include SSE (Streaming,AVX (Advanced Vector Extensions).
I have installed 2015-2019 C++ redistributable, 32bit and 64bit.
I installed tensorflow via pip.
Python is 64 bit as tensorflow ask in installation guide:
python -c "import platform;print(platform.architecture()[0])"
64bit
My script py runs as expected, my exe created via pyinstaller doesn't.

@fafabone
Just to verify are you able to import tensorflow. Thanks!

Any solution for my problem?

It seems it is a bug in pyinstaller not being able to properly load all needed DLLs. Not really a TF issue.

It seems it is a bug in pyinstaller not being able to properly load all needed DLLs. Not really a TF issue.

Please, could you suggest a way to produce an exe running in windows starting from python script that imports tensorflow?
Thanks

Please, could you suggest a way to produce an exe running in windows starting from python script that imports tensorflow?
Thanks


Update all native modules and their dependents to Nan 2

This is independent from #8779 and doesn't require an update of Electron. The good thing is all native modules are now update-to-date, and we no longer need to worry the build would break when we update a module or package to a later version.
The only change is to update the modules to use Nan 2, the build is green, but we might probably encounter bugs due to the update.
Fixes #8508
/cc @as-cii @nathansobo

Since this bumps all NaN dependencies to ^2, this should fix #8508 right?

Since this bumps all NaN dependencies to ^2, this should fix #8508 right?

Yeah it should be able to fix that.

This looks good to me, I'll try to build it later today and see if anything breaks. If after a first :eyes: it works fine I think we can just :ship: it.
Thanks, @zcbenz!

:clap:

I haven't noticed any regression so far, so I think this could be :ship:ped as soon as the merge conflicts get fixed.
Great job, @zcbenz! :star2:


Dynamic non-tiled map data

Dots on maps! Lines on maps! Wonders abundant!
These changes incorporate the very handy geojson-vt-cpp library from the good folks at Mapbox to enable two new ways of putting data on tangram maps:
1. Add geometry from a file of GeoJSON:
In the scene file, you can declare a data source from a non-tiled file location.
sources:
    my-data-file:
        type: GeoJSON
        url: path/to/my-data-file.json

The geometry from my-data-file.json will be added to the map and styled according to the scene file.
The usual layer property of the data block is ignored - all data from this source is one layer.
The url is currently limited to files in the bundled resources of the application.
2. Add points, lines, and polygons through C++ and Java interfaces
You can also create a data source at run-time in C++
int data_source_id = Tangram::addDataSource("my-cool-data");

or Java
MapData data = new MapData("my-cool-data");

Then add geometry however you fancy, in C++
double coords[2] = { coord_x, coord_y };
Tangram::addSourcePoint(data_source_id, coords);

or Java
LngLat point = new LngLat(coord_x, coord_y);
data.addPoint(point);

Again, data will be styled according to the scene file. The source property of a layer must match the name provided when creating the data source and the layer property is, again, ignored.
Some notes:

MapData is unsafe to instantiate before the Tangram MapView is initialized, which is not a well-defined point in the application life-cycle.  You can see from the current demo that I've just lazily instantiated it for now. I have in mind a way to remove this limitation, it'll just take a bit more time.
There are several ways to further optimize the usage of geojson-vt-cpp. I've left these for later in favor of making these features available as quickly as possible.
It could also be possible to add string properties to geometry that is programmatically added to the map (e.g. for filtering within the scene file) but this isn't implemented right now (again, for expediency).

@ecgreb I'd be interested in feedback on the Android interface (i.e. MapData)

Reviewing this now.

Have some unanswered questions/queries
master rebase
possible feature to add id/index to client data points.

Once these are done, this is good to merge.


i only can see the spotify logo when i start playing

I have the issue that if  i start to play a song on my phone , i only can see the spotify logo on the magic mirror screen. Can somebody help me?
Thx!

Are you using Spotify premium account?

Yes I do.

After MM is executed, the spotify logo is showing, Play any song on your iphone app (Same account with your module), At that time, what log says? (You can check it with pm2 show log or instead pm2, use npm start dev)

pi@SmartMirror:~ $ pm2 show log
[PM2][WARN] logs doesn't exist
pi@SmartMirror:~ $ npm start dev show log
npm WARN npm npm does not support Node.js v10.15.2
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 4, 6, 7, 8, 9.
npm WARN npm You can find the latest version at https://nodejs.org/
npm ERR! path /home/pi/package.json
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/home/pi/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR!  I'm just a beginner, so thank you for helping me.

Sorry for late answer. I was hospitalized.
npm start dev should be executed in your MagicMirror directory.
and should stop pm2 before.


Package Install with AdminsOnly option is changing all the profiles

Summary
When we run "sfdx force:package:install -p <*****> -b 0 -u <****> --apiversion=55.0 -r -s AdminsOnly --json" it should install and change the profile of the admin but it is changing the non admin profiles. When we do the same from a browser and selecting AdminsOnly option it is changing only the admin profile.
Steps To Reproduce:
Run  sfdx force:package:install -p <*****> -b 0 -u <****> --apiversion=55.0 -r -s AdminsOnly --json and see the profiles modified. It changes are non admin and admin profiles
Expected result
It should modify only the admin profiles
Actual result
Updating admin and non admin profiles
System Information
We have seen this behaviour starting from 7.180 to the latest 7.190
{
"cliVersion": "sfdx-cli/7.190.2",
"architecture": "darwin-x64",
"nodeVersion": "node-v18.14.1",
"pluginVersions": [
"@oclif/plugin-autocomplete 2.1.2 (core)",
"@oclif/plugin-commands 2.2.10 (core)",
"@oclif/plugin-help 5.2.5 (core)",
"@oclif/plugin-not-found 2.3.20 (core)",
"@oclif/plugin-plugins 2.3.2 (core)",
"@oclif/plugin-search 0.0.13 (core)",
"@oclif/plugin-update 3.1.5 (core)",
"@oclif/plugin-version 1.2.1 (core)",
"@oclif/plugin-warn-if-update-available 2.0.28 (core)",
"@oclif/plugin-which 2.2.16 (core)",
"alias 2.2.2 (core)",
"apex 2.2.0 (core)",
"auth 2.7.1 (core)",
"community 2.2.3 (core)",
"config 1.5.1 (core)",
"custom-metadata 2.1.3 (core)",
"data 2.3.1 (core)",
"info 2.4.4 (core)",
"limits 2.3.5 (core)",
"org 2.4.2 (core)",
"packaging 1.15.0 (core)",
"schema 2.3.1 (core)",
"signups 1.4.3 (core)",
"source 2.5.4 (core)",
"telemetry 2.1.1 (core)",
"templates 55.4.1 (core)",
"trust 2.4.2 (core)",
"user 2.3.2 (core)",
"@salesforce/sfdx-plugin-lwc-test 1.0.1 (core)",
"@salesforce/sfdx-scanner 3.10.0 (user)",
"sfdx-codescan-plugin 1.0.8 (user)"
],
"osVersion": "Darwin 22.3.0",
"shell": "zsh",
"rootPath": "/Users/srinivas.s/.local/share/sfdx/client/7.190.2-cd6cf69"
}

If you are using sfdx

Run sfdx version --verbose --json

If you are using sf

Run sf version --verbose --json

Paste the output here

Additional information
Feel free to attach a screenshot.

@ar-srinivas - can you please run the same command but without -s AdminsOnly.  AdminsOnly is the default and I'm curious if there is a difference in behavior.

Tried and it does the same
Get Outlook for iOShttps://aka.ms/o0ukef

From: Steve Hetzel @.>
Sent: Friday, March 3, 2023 1:45:03 AM
To: forcedotcom/cli @.>
Cc: Srinivas Salloori @.>; Mention @.>
Subject: Re: [forcedotcom/cli] Package Install with AdminsOnly option is changing all the profiles (Issue #1973)
@ar-srinivashttps://github.com/ar-srinivas - can you please run the same command but without -s AdminsOnly. AdminsOnly is the default and I'm curious if there is a difference in behavior.
—
Reply to this email directly, view it on GitHubhttps://github.com/forcedotcom/cli/issues/1973#issuecomment-1452487998, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHI55X43WAVZ2CAUQLWBKJLW2D5UPANCNFSM6AAAAAAVNO6LPE.
You are receiving this because you were mentioned.Message ID: @.***>
This E-Mail is strictly confidential and intended solely for the intended recipient only and may contain privileged, proprietary, or otherwise confidential information under applicable law. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful and AutoRABIT Inc. and/or any of its subsidiaries or associated companies shall not be held liable for any misuse of this message. If you have received this message by mistake, please notify the sender immediately and destroy all copies of this message including attachments without making any copy of any kind.

Ok good to know.  One last thing to try is the legacy install command.  Can you run sfdx plugins:install salesforce-alm, then run sfdx force:package:legacy:install -p <*****> -b 0 -u <****> --apiversion=55.0 -r -s AdminsOnly --json and see if that behaves correctly?  Then you can uninstall that plugin with sfdx plugins:uninstall salesforce-alm.  Thanks!

@shetzel I tried the steps you suggested to test it with legacy commands and the result is same.

@shetzel Please let us know if you have any updates on this issue.


Update Shared Components version

Update the scplus-shared-components & SQHooks dependencies to their latest versions

@20BBrown14 @lucas-homer @laurelbean Is this issue fine to delete since we are updating our scplus-shared-components version as part of the MUI5 migration?  Honestly not totally sure what the latest state of SQHooks is...  Just thinking we might want to do some cleanup of old issues if they are no longer relevant.

Currently on the latest.


xmlObject support

Hi,
I cannot find any information anywhere how to get the XML object(http://swagger.io/specification/#xmlObject) working in NSwag, is it a missing feature or am i doing something wrong?
I am after this section:
 "xml": {                 "name": "Pet"             }
From:
The pet store example JSON:
http://petstore.swagger.io/v2/swagger.json
Thanks for feedback,
Emil

The "xml" attribute is currently not supported.
What exactly would you expect? Generated .NET Xml* attributes?

Hi,
Ok, thanks for feedback!
I guess that what i was after was the same behaviour as the XMLSerializer in WebAPI e.g. property/type name from Xml* attribute and if that is not available go with the type/property name.
I guess this would complicate things even more if you would use different serializers.... :-(
Can you point me in a direction where in the source i can start looking, to make this change, or is there anyone that have looked into it earlier?

This is part of the JSON Schema and class generator which is implemented in http://njsonschema.org
We have to add a setting which specifies whether the "xml" properties should be generated and generate the required attributes in the code generator. The changes to the C# generator seem to be simple but the TypeScript part is trickier...

Hi,
What do you think is best, only support pure type names and properties in Njsonschema and let the WebAPI part in NSwag take care of the Xml* attributes? My take is that it would be sad to bind it to close to the serializer in the Njsonschema and instead limit it to only operate on the type and property names instead...
The other option as i see it is to only add the xmlObject to JsonSchema4 and not do anything in NJsonschema just to be able to use it from the outside. What is your take on it?

The problem is, that the schema and code for DTO classes are completely generated in NJS. If we add only the xml object data structures to NJS we need lots of extension points to hook into this generation process... I agree that it is not correct to add the whole functionality to NJS because it is not a JSON Schema feature but it is the simplest way to implement and later understand it...

I buy it completely, i have done the implementation on the NJsonSchema side for the non XML*Attribute scenarios, but they do not make sense due to that you need to use them to make well formated XML. So maybe it's best to implement a nice approach in there instead of relying partly on NSwag side as you say.
My implementation only covers Assembly => Swaggerfile, is that ok as a first step?
Should i make a pull request?

Thank you for the PR. The next step would be to generate Xml* attributes in the NJS code generators... I think for C# it should be simple, but for TypeScript I don't know an easy solution yet... So I think we should start with the C# generator.

Hi,
I have tested it all the way now with the latest NSwag => SwaggerUI and it looks good!
You are fast in integrating it all the way, i thought you would wait until i gave you a thumb up :-) 👍 .
Do you want me to start working on the generation also or do you want to do it yourself?

Because this is a new feature it is fine for me to release it that fast. I tested it and it looked good.
For the generation part: Again we need a setting GenerateXmlAttributes and if it is on, generate the appropriate attributes (full qualified).
I have a lot of other issues to work on in NSwag and NJS and if you have time i would really appreciate it if you can look into this.

Ok, I will look at it. I have not worked so much with Typescript so i will start with generating the attributes for the classes.

Typescript xml is a much bigger effort to implement. I think we should start with c# generation..

Any update on this?

Created new issue: https://github.com/RSuter/NSwag/issues/1735


React Native Authentication (OIDC) SDK Development

Description
As of now, Identity Server doesn't have any client-side tools like SDKs or official guides on how to integrate react native application with IS. #4
Description of the solution you would prefer
Develop a fully fledge React Native Authentication (ODC) SDK to provide secure authentication for React Native application with IS.
Task

[ ] Creating Infrastructure  for RN SDK

[ ] Adding Sign In Sign out Functionalities to the RN SDK

[ ] Add Extracting User Information Functionalities to RN app

[ ] Testing for SDK

Closing the issue as this repo is no longer maintained.


fix(input[radio]): use non-strict comparison for checkedness

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Reverting a Breaking Change
What is the current behavior? (You can also link to an open issue here)
Checkedness is determined by comparing $viewValue and value strictly.
What is the new behavior (if this is a feature change)?
Checkedness is determined by comparing $viewValue and value non-strictly.
Does this PR introduce a breaking change?
No
Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format
[x] Tests for the changes have been added (for bug fixes / features)
~[ ] Docs have been added / updated (for bug fixes / features)~

Other information:
This was introduced during the switch to ESLint, but it is a breaking change.

Can you mention the commit that introduced the regression in this PR or better yet in the commit message? Other than that LGTM

Commit is https://github.com/angular/angular.js/commit/b1bc25169e479ca276b839fe411a6b4ff6804773#diff-c244afd8def7f268b16ee91a0341c4b2R1691

So, it didn't make it into master, but it did make it into 1.5.x.
LGTM


fix(APMON-1429): apply security context to all init containers

What does this PR do?
https://datadoghq.atlassian.net/browse/APMON-1429
Ensures that initSecurityContext parameters are applied to all init containers that are injected via auto-instrumentation.
Motivation
During QA testing of https://datadoghq.atlassian.net/browse/APMON-1368 I noticed that we didn't propagate the security context to the inject init container.
Additional Notes
This passes through the same initContainerMutators to the injector requirements as we do for the rest of the library requirements.
Possible Drawbacks / Trade-offs
Describe how to test/QA your changes

[x] Added regression tests to make sure that all init containers get the security context applied.
[x] Manual QA with built image and validating security contexts applied to all initContainers.

/merge

/merge


Update showdown.js

Fixed a couple typos and adding a better check for fs. When we are using showdown with webpack, fs is returned as an empty object, which passes the test erroneously.

thanks for your contribution. You're changes have been used in version 0.3.3


[#119089085] Stop graphite nozzle sg cleanup

What
After #727 is merged and applied, we no longer need to keep unbinding the graphite nozzle sg. This PR fixes that.
How to review
Checkout, make dev pipelines, run cf-deploy job, observe that graphite-nozzle group is no longer being removed from global running and staging SGs.
Who can review
not @mtekel

:+1: This looks good to merge once #727 has reached production.


Question Regarding the Experimental Setup for the Reported Results

Hi!
This is an outstanding piece of work—not only as a user-friendly, integrated tool but also as a comprehensive analysis of Knowledge Editing for LLMs!
While reading your paper, "A Comprehensive Study of Knowledge Editing for Large Language Models," I found myself a bit unclear about the detailed experimental setup. For example, when reviewing the result tables, I was curious about specifics such as the number of edits, method-specific parameters (e.g., layers, v_num_grad_steps in ROME), and the selection of edit data (although the source is noted, was the data chosen randomly, and were the results averaged across multiple tests?).
Having a complete description of the experimental settings that correspond to the reported results in your paper would be incredibly helpful—not only for me but also for future readers and researchers looking to build on this work.
Looking forward to your response! :)

Thank you very much for your interest in EasyEdit! Our parameters are the default hyperparameters in EasyEdit. We did not modify them; everything was run according to the parameters specified in the original method.

OK i see, thanks!

After the code has been updated, the results in the paper are outdated and we're currently checking the results, but it seems to have some random results. Please follow #390 to keep the update.

I think the problem is solved and we will update the arxiv this week. The setting to get the result is the same as the hparam files in our repository and the data we used is the KnowEdit data in the huggingface.
You can just check the run_knowedit_llama2.py  to see how we compute the results.
Here, we edit each case in the test json and average them.
Leave any message if you have further question and please help us close the issue if it pans out.

I close this issue and you can reopen if you have further questions.

Dear 5456es,
We have fixed the bug and will update the paper on arXiv tomorrow (the README has been updated). We have written a pinned issue statement explaining the cause of this issue and included an announcement in the News. We apologize for any inconvenience caused by this bug.
The following is a statement.
Dear all:
Recently, with the help from the community (special thanks to @StarLooo), we will update the KnowEdit results (Llama2-7b-chat) in Table 4 of the paper ‘A Comprehensive Study of Knowledge Editing for Large Language Models’. Overall, the results have improved, primarily due to the following reasons:
1. AdaLoRA Optimization: we follow the FT-M  instead of the FT-L, which trains the same FFN layer as FT-L using cross-entropy loss on the target answer while masking the original text. This approach not only yields better results but also highlights the optimal performance of AdaLoRA. Meanwhile, the peft version would also affect the performance.
2. ROME and MEMIT Updates: The results are updated after identifying missing components in the local version of the Llama2-7b-chat files (specifically, the legacy feature in the tokenizer_config.json). If you are using the official Llama2-7b-chat model downloaded directly from HF, this issue should not affect your results. Also, we fix a bug related to the padding_size for these two methods, which will influence the performance when you compute the results for batch inputs.
We deeply apologize for any inconvenience caused by this bug.
We will continue improving EasyEdit, updating this paper, and welcome everyone to engage in discussions and share ideas.
EasyEdit Team


ci(windows): build x86_64-windows takes a long time.

In build (windows-latest, windows, x86_64, nightly-x86_64-pc-windows-gnu) / build x86_64-windows, even though the test is completed, it does not proceed to the next STEP for a long time.

I have no idea what I'm waiting for...

Maybe need to add exit function
  shutdown() {
    this.server?.close();

  //     process.exit(0)
  }

I think there is a race condition here doesn't close the server or child processes properly. We need to investigate.

This has been fixed in #721 so I will close it.


How to use the app with REVERSE_PROXY_AUTH

Hi,
I am using the baby buddy web app configured with the REVERSE_PROXY_AUTH setting, to use my Authelia user header.
How can I use this with the mobile app as well? It would need to know how to sign in to Authelia, and extract the header, I guess.
Any ideas how I could setup this?
Thanks for this great app!

Hello! First of all let me say that your setup seems to be very cool and advanced. I assume that you had to use a custom Django-config for this to work as well, is that correct?
Second, I unfortunately must say that I am not properly equipped to answer your question. I do not know how Authelia is handling authentication exactly and if the regular baby buddy authentication still is in-place? Normally, the app uses the username/password combo to do an initial login and then scrape the baby-buddy apptoken which then is used to access the app in the future.
The app does not (appear) to offer entering the apptoken directly, however, the QR-code scanner actually does this! So this is, if token authentication is still working for you(?), probably your best bet:

You could even manufacture your own QR code if you can provide some other means of authentication that is compatible with the babybuddy API access. The QR code is very much readable (JSON-)text. Here an example from my testservers:
BABYBUDDY-LOGIN:{"url":"http://localhost:9002/","api_key":"499856ac563097b26a161215f12e5913eb78637e","session_cookies":{}}

api_key is used for the traditional auth-header:
Authorization: Token 499856ac563097b26a161215f12e5913eb78637e

session_cookies is a bit of a crutch to allow the app to authenticate with homeassistant-ingress_servers. Kind of a crutch. But you can supply cookies in that section.
So, what I suggest is that you can try and populate your own QR-code here with the correct tokens etc and then try to login that way! I think it is unlikely at this point that I will have time or motivation to support a specialized authentication system that does not ship with babybuddy by default.


Add common patterns / recipies to documentation

The Globbing syntax is not the easiest in the world, it would be nice to add some documentation and / or code samples to show how to do various common tasks with moveup, for instance:

Flatten the whole destination folder
Match only a single folder
Match all files of a single type
Match all files with the same partial name
Negate matches
Match by conditional or filtered pattern

@rbardini This is pretty much all that was left to do when I left it, feel free to do or close.


0.7.1

@allewun @kevgo @ricmatsui

Good from my end

Manually merged in 0cc716304800848b4241212c0aef7471d1308e81 as I couldn't ship it :disappointed:

Wow, a lot of changes in a very short time! Nice!


Properly award new powers and additional inspiration slots on level up

We already have /levelup so the actual level up process is at least done.
The things that should be done to solve this issue should be:

Check if character's exp is enough to level up (should be handled by client already?), but this might be for another issue if needed
On each level up, check if the character level is enough to award additional inspiration slots
On each level up, check if the character level reaches the amount required to be granted certain powers

So some insight into the client side here:
The client already checks for these things, but on the server side we need to verify (as server has authority). There's also a couple of opcodes thrown back to the server when running /levelup (possibly not every time?) so we'll need to handle those as well.
Another thing you may want to add to this: we can adjust max-attribs based upon level. It may rely upon #502 to get per-class by level data.
I'll be more helpful with information after I've had some hands-on over the next couple of days.

A note on this: we're not currently handling adding new empty enhancement slots to powers correctly, and the server just forces every power to have 3 empty enhancement slots

We should add something like "trained level" to keep track while you are training. According to the wiki it was "security level"


idle notifier: add crystal felling axe (inactive) animation

The crystal felling axe (inactive) uses a different animation id than the charged one for some reason, as outlined in: #17218
Please also refer to that issue for a picture of the animation id.

This needs to be added to at least: wc, wintertodt, idle notifier plugins

Apologies, I should have used ctrl+B instead of rushing out a PR before I had to leave. Fixed it now.
Obviously feel free to squash the commits if you prefer to.


add test for mongoose schema, see #17

Current coverage is 42.93% (diff: 0.00%)

Merging #28 into master will decrease coverage by 12.01%

@@             master        #28   diff @@
==========================================
  Files            13         15     +2
  Lines           182        191     +9
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits            100         82    -18
- Misses           82        109    +27
  Partials          0          0

Powered by Codecov. Last update bab9c01...98dfe10

@sibelius: yeoman seems to have some sort of cache when it runs the generator so I separated the use of the same typeGenerator with helper.run for each test.
Also, I moved the fixtures to the root of the generator package so they can be used by other tests as well.
What do you think?


fix: filter images by image-list annotation

Try to fix #650
@jannfis

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (7d93c7a) 66.27% compared to head (8b13249) 66.35%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #651      +/-   ##
==========================================
+ Coverage   66.27%   66.35%   +0.07%
==========================================
  Files          22       22
  Lines        2150     2155       +5
==========================================
+ Hits         1425     1430       +5
  Misses        591      591
  Partials      134      134

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Icons folder is missing causing unability to replay recorded script

when you install pywinauto_recorder using:
pip install -U pywinauto_recorder
and try to run previously recorded script, this crash occurred:

When you install pywinauto_recorder using:
pip install -U pywinauto_recorder
It does not install the Pywinauto recorder GUI.
Indeed, the icons directory is not in the package. To get it, you have to download the whole project which also contains the main GUI of pywinauto recorder.
But it is indeed possible to record a Python script by calling the start_recording() function without going through the GUI.
To fix this bug I will move the Icons directory in the package in the next version of Pywinauto recorder.

It is fixed on version 0.3.8. Can you retry with this version to confirm it is fixed for you?

Hi, I installed the 0.3.8 version via pip, and it didn't help. I had to copy Icons into pywinauto package folder manually.
Maybe I got the workflow somehow wrong:

I downloaded the installer and installed it
Recorded script with executable
I ran the generated script in my Python venv , where I installed the pywinauto_recorder package via pip.

If the package is not intended to run scripts and it only serves a use for some API stuff, then my bug maybe is not a bug and is not relevant.

@RolandMisica  The Icons directory should have been moved into the package and it should have worked but I confess I did not test it. In fact, I forgot to commit the Icons directory. I need to take the time to do it properly but I won't be able to do it until next week.
Your bug is relevant but I think that we should not put the icons in the package because there is no need to display the icons in replay mode.
So I need to make sure that the package can run a script without requiring the icons.

I just committed and pushed the Icons directory and published another package. It should work now.

@RolandMisica Can I close this issue?

@RolandMisica Having no news, I take the liberty of closing this issue. Do not hesitate to reopen it if needed.

Even in version 3.9.0 the icon directory is still not present in the package but I don't know how to solve this problem for the moment.

Hi, sorry for not answering, I was off...
I tested it , varsion 0.3.9 isn't working from me either
....but you managed to fix it in new (0.4.0) version.
So this issue can be closed now


Revert "Refactor helpers and change lifecycle ts folder names"

Reverts test-network-function/cnfcert-tests-verification#92

@kononovn may I know why this PR was reverted? Can you provide a more descriptive text/reason than "Reverts #XX", please?


NextJS Landing Page

Please connect content to the schema (bottom sections). Change top CTA button back to Zesty dark blue with white text. Make the bottom CTA buttons match the "try free" and "schedule a demo" ones on the homepage.

Hey Darwin, I mistyped the CTA button directions. Can we actually change the bottom button to match the line of code at the top? Make it orange instead of dark blue. Can we also remove the orange outline from the button at the top? Thanks!

Approved, thank you!


Minimized the number of `stylesheet_link_tag˙ calls

#5799

@epwinchell @himdel @abonas please review

topology looks fine

LGTM, but didn't you forget
app/views/layouts/_my_code_mirror.html.haml
23:  = stylesheet_link_tag "codemirror/themes/#{theme}"

?

@himdel yes!

LGTM

Please merge!

@skateman .. this PR should now also remove those files from config/initializers/assets.rb (because #6070), right?

@himdel I will do it in a separate PR because these are JS stuff


Blazor windows authentication authorization issues with ClaimsTransformation

Steps to reproduce the behavior:

Create a new Blazor server-side app with the preview 6-012264 using the blazor extension in Visual Studio and add windows authentication.
Add the following class in the app with the following code:

    public class ClaimsTransformerTest: IClaimsTransformation
    {
        public async Task<ClaimsPrincipal> TransformAsync(ClaimsPrincipal principal)
        {
            ClaimsIdentity identite = (ClaimsIdentity)principal.Identity;
            if (identite.IsAuthenticated && identite is WindowsIdentity)
            {
                    identite.AddClaim(new Claim("Admin", "Role1"));
            }
            await Task.Delay(10);
            return new ClaimsPrincipal(identite);
        }
    }

And add the following lines in the ConfigureServices method in the startup class:
services.AddAuthentication(IISDefaults.AuthenticationScheme);
services.AddAuthorization(options =>
{
    options.AddPolicy("AdminRole1", policy =>
        policy.RequireClaim("Admin", "Role1"));
});
services.AddScoped<IClaimsTransformation, ClaimsTransformerTest>();

And in the Configure method add these lines:
 app.UseAuthentication();
 app.UseAuthorization();

And finally add the Authorize attribute to the FetchData.razor or any page that you want to access:
@using Microsoft.AspNetCore.Authorization;
@attribute [Authorize(Policy = "AdminRole1")]

Run the app in debug and try to access the page in which you added the authorization policy
I get the following error:

System.ObjectDisposedException: Safe handle has been closed.
Object name: 'SafeHandle'.
   at System.Runtime.InteropServices.SafeHandle.DangerousAddRef(Boolean& success)
   at System.StubHelpers.StubHelpers.SafeHandleAddRef(SafeHandle pHandle, Boolean& success)
   at Interop.Advapi32.GetTokenInformation(SafeAccessTokenHandle TokenHandle, UInt32 TokenInformationClass, SafeLocalAllocHandle TokenInformation, UInt32 TokenInformationLength, UInt32& ReturnLength)
   at System.Security.Principal.WindowsIdentity.GetTokenInformation(SafeAccessTokenHandle tokenHandle, TokenInformationClass tokenInformationClass, Boolean nullOnInvalidParam)
   at System.Security.Principal.WindowsIdentity.get_User()
   at System.Security.Principal.WindowsIdentity.<GetName>b__51_0()
   at System.Security.Principal.WindowsIdentity.<>c__DisplayClass67_0.<RunImpersonatedInternal>b__0(Object <p0>)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Security.Principal.WindowsIdentity.RunImpersonatedInternal(SafeAccessTokenHandle token, Action action)
   at System.Security.Principal.WindowsIdentity.RunImpersonated(SafeAccessTokenHandle safeAccessTokenHandle, Action action)
   at System.Security.Principal.WindowsIdentity.GetName()
   at System.Security.Principal.WindowsIdentity.get_Name()
   at System.Security.Principal.WindowsIdentity.InitializeClaims()
   at System.Security.Principal.WindowsIdentity.get_Claims()+MoveNext()
   at System.Security.Claims.ClaimsPrincipal.get_Claims()+MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source, Func`2 predicate)
   at Microsoft.AspNetCore.Authorization.Infrastructure.ClaimsAuthorizationRequirement.HandleRequirementAsync(AuthorizationHandlerContext context, ClaimsAuthorizationRequirement requirement)
   at Microsoft.AspNetCore.Authorization.AuthorizationHandler`1.HandleAsync(AuthorizationHandlerContext context)
   at Microsoft.AspNetCore.Authorization.Infrastructure.PassThroughAuthorizationHandler.HandleAsync(AuthorizationHandlerContext context)
   at Microsoft.AspNetCore.Authorization.DefaultAuthorizationService.AuthorizeAsync(ClaimsPrincipal user, Object resource, IEnumerable`1 requirements)
   at Microsoft.AspNetCore.Components.AuthorizeViewCore.IsAuthorizedAsync(ClaimsPrincipal user)
   at Microsoft.AspNetCore.Components.AuthorizeViewCore.OnParametersSetAsync()
   at Microsoft.AspNetCore.Components.ComponentBase.CallStateHasChangedOnAsyncCompletion(Task task)
   at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
Microsoft.AspNetCore.Components.Server.ComponentHub: Warning: Unhandled Server-Side exception

System.ObjectDisposedException: Safe handle has been closed.
Object name: 'SafeHandle'.
   at System.Runtime.InteropServices.SafeHandle.DangerousAddRef(Boolean& success)
   at System.StubHelpers.StubHelpers.SafeHandleAddRef(SafeHandle pHandle, Boolean& success)
   at Interop.Advapi32.GetTokenInformation(SafeAccessTokenHandle TokenHandle, UInt32 TokenInformationClass, SafeLocalAllocHandle TokenInformation, UInt32 TokenInformationLength, UInt32& ReturnLength)
   at System.Security.Principal.WindowsIdentity.GetTokenInformation(SafeAccessTokenHandle tokenHandle, TokenInformationClass tokenInformationClass, Boolean nullOnInvalidParam)
   at System.Security.Principal.WindowsIdentity.get_User()
   at System.Security.Principal.WindowsIdentity.<GetName>b__51_0()
   at System.Security.Principal.WindowsIdentity.<>c__DisplayClass67_0.<RunImpersonatedInternal>b__0(Object <p0>)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Security.Principal.WindowsIdentity.RunImpersonatedInternal(SafeAccessTokenHandle token, Action action)
   at System.Security.Principal.WindowsIdentity.RunImpersonated(SafeAccessTokenHandle safeAccessTokenHandle, Action action)
   at System.Security.Principal.WindowsIdentity.GetName()
   at System.Security.Principal.WindowsIdentity.get_Name()
   at System.Security.Principal.WindowsIdentity.InitializeClaims()
   at System.Security.Principal.WindowsIdentity.get_Claims()+MoveNext()
   at System.Security.Claims.ClaimsPrincipal.get_Claims()+MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source, Func`2 predicate)
   at Microsoft.AspNetCore.Authorization.Infrastructure.ClaimsAuthorizationRequirement.HandleRequirementAsync(AuthorizationHandlerContext context, ClaimsAuthorizationRequirement requirement)
   at Microsoft.AspNetCore.Authorization.AuthorizationHandler`1.HandleAsync(AuthorizationHandlerContext context)
   at Microsoft.AspNetCore.Authorization.Infrastructure.PassThroughAuthorizationHandler.HandleAsync(AuthorizationHandlerContext context)
   at Microsoft.AspNetCore.Authorization.DefaultAuthorizationService.AuthorizeAsync(ClaimsPrincipal user, Object resource, IEnumerable`1 requirements)
   at Microsoft.AspNetCore.Components.AuthorizeViewCore.IsAuthorizedAsync(ClaimsPrincipal user)
   at Microsoft.AspNetCore.Components.AuthorizeViewCore.OnParametersSetAsync()
   at Microsoft.AspNetCore.Components.ComponentBase.CallStateHasChangedOnAsyncCompletion(Task task)
   at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
Microsoft.AspNetCore.Routing.EndpointMiddleware: Information: Executed endpoint '/_blazor'
Microsoft.AspNetCore.Hosting.Diagnostics: Information: Request finished in 1487.5535ms 101

Looks similar to some earlier issues with Windows authentication in non-Blazor cases:

https://github.com/aspnet/Logging/issues/543
https://github.com/aspnet/HttpSysServer/issues/426
https://github.com/aspnet/SignalR/issues/2948

I see that https://github.com/aspnet/SignalR/issues/2948 was fixed by @BrennanConroy making a change to clone the ClaimsPrincipal at some point in the flow.
@BrennanConroy, do you still remember what was going on with that SignalR issue? Do you know if we need to do something equivalent in Blazor? In Blazor we're holding a reference to the User that comes from HubConnectionContext - is there some reason why we can't hold and use that for the duration of the connection? When and why does it have to be cloned?

I don't think you can change a Windows identity like this, because a windows identity is somewhat special, it's not a normal claims identity at all, it just maps to one via magic, and it will get its claims from AD.
Take the claims out of the windows identity, put them into a new ClaimsIdentity via the IEnumerable constructor and then build a new principal off that.

I don't think you can change a Windows identity like this, because a windows identity is somewhat special, it's not a normal claims identity at all, it just maps to one via magic, and it will get its claims from AD.
Take the claims out of the windows identity, put them into a new ClaimsIdentity via the IEnumerable constructor and then build a new principal off that.

I tried what you suggested and the problem is gone, thanks!

Thanks for the info, @blowdart. Closing as "working as expected" then.

In Blazor we're holding a reference to the User that comes from HubConnectionContext - is there some reason why we can't hold and use that for the duration of the connection?

As long as it's only for the duration of the connection you're good. SignalR will handle any silliness like the windows auth stuff.

@BrennanConroy

As long as it's only for the duration of the connection you're good

Well, that's not completely the case. If the SignalR connection drops, we move the circuit into a "pending shutdown" state where it stays for a while, as we wait to see if the user reconnects. It's possible that some server-side code chooses to run while the circuit is in "pending shutdown" and such user code could interact with the User object.
What are the extra guarantees that SignalR offers while the connection is open that it stops doing later? How is SignalR actually affecting the ClaimsPrincipal object for some other code that is holding a reference to that object (and not just using HubConnectionContext.User)?

Well the User object isn't disposed while the SignalR connection is around. We don't explicitly dispose it, but it could be disposed by the server when the connection is gone.
@Tratcher I'm assuming it's a bad idea to hold onto the User from HttpContext after the underlying server connection is gone?

Right, it's disposed at the end of each request. Clone it if you need to hold onto it.

I'm getting the following using Windows Authentication:
'user.Identity.Name' threw an exception of type 'System.ObjectDisposedException'
You suggests that I should clone the claims, which I do not know where or how to do. Please can you help.

This is an issue that will be fixed in the 3.0 Preview 8 release (the code is already in master). Until then there isn't an easy fix, though technically it's possible to implement your own AuthenticationStateProvider that does whatever you want.

Thanks Steve


adapt to use ocw-studio generated ocw-www content

Pre-Flight checklist

[ ] Testing

[ ] Changes have been manually tested

What are the relevant tickets?
Closes https://github.com/mitodl/ocw-hugo-themes/issues/125
What's this PR do?
In mitodl/ocw-hugo-projects#8, we set up the ocw-www studio config to have a collection of resources with uploaded files and metadata. Now that we are moving to source ocw-www content from ocw-studio, we will need to update the way we source image URLs for promos and testimonials to reflect this change. Currently, in the static version of ocw-www at https://github.com/mitodl/ocw-www we simply store the image URL under the image front matter property and the alt text in image_alt.
This PR adds the resource_url.html partial to the www theme.  It takes two arguments, the first being filetype and second being uid.  The partial retrieves a resource by first filtering the resources by filetype and then matching on uid.  It then returns the URL for the image minus the hostname and schema, leaving a relative URL.  ocw-studio currently writes in fully qualified S3 URLs into the file field, and these need to be stripped down to just be a relative URL in the final output as it will be served from a CDN.  This partial is then used in the various locations that promo and testimonial images are displayed throughout our templates.
How should this be manually tested?

If you have never run this project before, run yarn install
Run npm run start:site:webpack
Clone https://github.com/mitodl/ocw-hugo-projects locally
Clone https://github.mit.edu/ocw-content-rc/ocw-www_eb8b70174c714752a91dbe730af76769 locally and open a terminal there
Run hugo server --config /path/to/ocw-hugo-projects/ocw-www/config.yaml --themesDir /path/to/ocw-hugo-themes, replacing /path/to with your path to these respective repos
Browse to http://localhost:1313 and check the URLs on the images for promos and testimonials at:

http://localhost:1313
http://localhost:1313/testimonials/
http://localhost:1313/testimonials/clinton-blackburn/

Note that the images won't load locally as they are being stripped of their hostname down to relative URLs for use behind the CDN.  What you should expect to see for each URL is a pattern of /ocw-www/filename.  Check the markdown source for any given testimonial or promo and make sure that the URL written into the outputted HTML is essentially the file property from the markdown minus the schema and domain (https://ol-ocw-studio-app-qa.s3.amazonaws.com/)

PS The lighthouse test failed:
Error: Error building site: failed to render pages: render of "page" failed: "/home/runner/work/ocw-hugo-themes/ocw-hugo-themes/www/layouts/testimonials/single.html:12:89": execute of template failed: template: testimonials/single.html:12:89: executing "main" at <$testimonial.Params.image.content>: can't evaluate field content in type string

but it deployed fine for me locally.

@mbertrand should be all set


[WIP] Implement Keystroke Recordings for Assessments

Description
Implements Simple Keystroke Recording for the AssesmentWorkspace
This is done using local state management within the AssessmentWorkplace of Source Academy
Fixes # (issue)
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Code quality improvements

How to Test
Making Changes within the Editor and Clicking Save should allow a payload to be logged into your console.
TODO: Add handlers to send payload to backend.
Checklist
Please delete options that are not relevant.

[ ] I have tested this code
[ ] I have updated the documentation

Pull Request Test Coverage Report for Build 6811

17 of 59 (28.81%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.1%) to 32.236%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/commons/assessmentWorkspace/AssessmentWorkspace.tsx
17
59
28.81%

Files with Coverage Reduction
New Missed Lines
%

src/commons/assessmentWorkspace/AssessmentWorkspace.tsx
1
55.2%

Totals

Change from base Build 6810:
-0.1%

Covered Lines:
1939

Relevant Lines:
5640

💛  - Coveralls


Update manifest version to MV3

Chrome will eventually depreciate manifest version 2. Consider allowing an option to use MV3 instead.

I'm going to move to v3 by default because you can't even create new v2 extensions in the chrome webstore.


[BUG] Long tweet of blue tick have text mixed with URL

Describe the bug
Since the fix of the bug #709, long tweet from blue tick user have text which is mixed with its URL
To Reproduce
Steps to reproduce the behavior:

Go to https://twitter.com/YusufCatStevens/status/1655892996544421889
See error

Screenshots
Fritter:

Browser:

Device
Model: Samsung Galaxy S20FE
OS: Android 10
Language: English
Fritter Version: v3.0.0-beta9+300001014 with the commit 63675c9989cfe8d6cf8ddde18e4f58b86758db01

I also noticed similarly that these long tweets in the profile feed cut off prematurely and end with that same partial URL text instead.


Use qpy.load from qiskit/terra

Summary
Details and comments

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.007%) to 51.287%

Totals

Change from base Build<PHONE_NUMBER>:
0.007%

Covered Lines:
3367

Relevant Lines:
6565

💛  - Coveralls


Reduce rounding errors in l2_sqr_float()

This change improves the floating point accuracy of the l2_sqr operation on float32 values. With divide and conquer summation, we reduce roundoff errors to logarithmic growth. For example, on an array with 32768 floats of random values on the interval (-1,1), the accuracy of naïve summation degrades to 1e-3 whereas this algorithm maintains 1e-7 accuracy compared to an implementation that uses a float 64 compute type. Divide / conquer also enables better vectorization.
1.10731 ns 100x l2_sqr_float_avx
1.26103 ns 100x l2_sqr_float in ANSI C (NEW implementation)
1.65579 ns 100x l2_sqr_float in ANSI C (old implementation)
1.87289 ns 100x l2_sqr_double

The benchmarks above, measured in nanoseconds per float show we can gain nearly all the performance benefits of intrinsics without the complexity therefore this change removes the AVX and NEON implementations, since it isn't worth compromising standards conformance and clarity for just 10%.

Hey @jart , thanks for the PR! Definitely interested in making the L2 calculations more accurate.
However I'm seeing quite a large slowdown (2-3x) compared to what's currently there. Here's a sample of what I'm getting before:
➜ python bench.py sift.suite
Config(name='sift1m', input='data/sift/sift_base.fvecs', k=20, queries='data/sift/sift_query.fvecs', qsample=100, tests=['faiss', 'usearch', 'duckdb', 'vec-static', 'vec-vec0.8192.1024', 'vec-vec0.8192.2048', 'vec-scalar.8192', 'numpy'], sample=None)
Starting benchmark suite: sift1m (1000000, 128), k=20
faiss...
duckdb...
sqlite-vec static...
sqlite-vec 8192 1024...
sqlite-vec 8192 2048...
sqlite-vec-scalar...
numpy...
           sift1m: 1,000,000 128-dimension vectors, k=20
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Tool                        ┃ Build Time (ms) ┃ Query time (ms) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ sqlite-vec static           │             0ms │            10ms │
│ faiss                       │            88ms │            18ms │
│ sqlite-vec vec0 (8192|2048) │          4610ms │            31ms │
│ sqlite-vec vec0 (8192|1024) │          3979ms │            34ms │
│ duckdb                      │           598ms │            44ms │
│ usearch numpy exact=True    │             0ms │            61ms │
│ sqlite-vec-scalar (8192)    │          1029ms │            75ms │
│ numpy                       │             0ms │           140ms │
└─────────────────────────────┴─────────────────┴─────────────────┘

And with this change:
alex sqlite-vec/benchmarks/exhaustive-memory
➜ python bench.py sift.suite
Config(name='sift1m', input='data/sift/sift_base.fvecs', k=20, queries='data/sift/sift_query.fvecs', qsample=100, tests=['faiss', 'usearch', 'duckdb', 'vec-static', 'vec-vec0.8192.1024', 'vec-vec0.8192.2048', 'vec-scalar.8192', 'numpy'], sample=None)
Starting benchmark suite: sift1m (1000000, 128), k=20
faiss...
duckdb...
sqlite-vec static...
sqlite-vec 8192 1024...
sqlite-vec 8192 2048...
sqlite-vec-scalar...
numpy...
           sift1m: 1,000,000 128-dimension vectors, k=20
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Tool                        ┃ Build Time (ms) ┃ Query time (ms) ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ faiss                       │           183ms │            22ms │
│ sqlite-vec static           │             1ms │            37ms │
│ duckdb                      │           554ms │            44ms │
│ usearch numpy exact=True    │             0ms │            60ms │
│ sqlite-vec vec0 (8192|1024) │          4113ms │            66ms │
│ sqlite-vec vec0 (8192|2048) │          4675ms │            68ms │
│ sqlite-vec-scalar (8192)    │          1080ms │           126ms │
│ numpy                       │             0ms │           136ms │
└─────────────────────────────┴─────────────────┴─────────────────┘

(Ran on a Mac M1 mini with -DSQLITE_VEC_ENABLE_NEON)
The benchmarks are under benchmarks/exhaustive-memory, requires python to run. They're a bit of a mess, so my apologies. But the most important test for this is sqlite-vec static, which holds the entire vector dataset in contiguous memory (like Faiss/usearch/numpy), so that's the best test for l2_sqr_float(). The other vec-related tests (sqlite-vec scalar and sqlite-vec vec0 have different storage strategies for better persistence/write performance, so it's not the best test of l2_sqr_float()
That's with the sift1m dataset (download with make data/sift in that directory), which is 1 million vectors with 128 dimensions. Not the biggest (most text embeddings are 384/768/1024 dimensions), but good enough for a quick benchmark.
Though I am mostly ok with rounding errors here, as other vector libraries (faiss, hnswlib, usearch, etc) also make the same tradeoff. Though having a compile-time or runtime setting to enable better accuracy at the expense of performance might be interesting...

Yeah you're right. Benchmarking it again, I'm noticing a larger disparity than I measured before. I don't think there needs to be a tradeoff on accuracy. I'll circle back later when I have time to write this change using intrinsics.


ex5_19

hello, maybe
error:  cout << (str1 <= str2 ? str1 : str2) << " is less than the other. "
right:  cout << (str1.size() <= str2.size() ? str1 : str2) << " is less than the other. "

Hi, @WuPeng2016 .
The question is less than, rather than shorter than. Thus, we should compare by alphabet order, not the length.

Yes, Simple comparison with '<' operator on string should do.

cout<<(s1<s2?s1:s2);


update rockspec to be more dynamic, added latest release

@alerque not sure this will upload the rockspecs since the tag and 1.4 rockspec is not in the same commit (requirement iirc)

Actually with our current logic I think it will try to upload when you merge this. The commit being pushed doesn't match a tag, but you'll be pushing a new rockspec to the default branch. Checking for those is an OR condition. Since the tag has never been uploaded to LuaRocks before it should succeed.

here goes ....

Seems to have worked as expected to me.

yep worked! I really like that stuff ❤


[New] Show scale bar

Description
This PR implements Show scale bar  in Maps category.
swift/issues/5465
How To Test
Load map and check that the scale bar is present on the bottom of the screen. Zoom in on location on map and check to see that the scale is adjusted for the level of zoom.
Screenshots

Looks good. Can you update the sample screenshot as well?

Pushed the updated screenshot