Azure single sign on

Local development is still possible despite integration of Azure single sign on.
SSO is turned on in production by definition of the following env variables: OPENPECTUS_DOMAIN_NAME, OPENPECTUS_APPLICATION_CLIENT_ID, OPENPECTUS_DIRECTORY_TENANT_ID and OPENPECTUS_CLIENT_SECRET.
User information is available to the frontend at /id and offers json:
{
"full_name":"Not Applicable",
"initials":"NA",
"profile_picture":"data:image/jpeg;base64, /9j/4AAQSkZJ[..]",
"roles":["User"]
}

This PR uses the ìdentity` python package (https://pypi.org/project/identity/) which is only in version number 0.2.0. Additionally, this package only covers Flask and not FastAPI.
This constraint means that API endpoints are not covered by authentication.
It would likely be a batter basis to use fastapi-azure-auth (https://intility.github.io/fastapi-azure-auth/) which is much more mature. However, this library is not really meant for authentication. Instead it accepts a token and can use that to access Azure endpoints.
So at this time, I think the current solution is the lesser evil.

We can definitely do OAuth 2 authentication on the frontend, and I have done so before. But it is a lot more complicated than handling it through the backend and cookies (still using an idp and OAuth 2, just through the backend).
I'm also concerned that fastapi's own documentation https://fastapi.tiangolo.com/tutorial/security/first-steps/ more or less only talks about implementing OAuth2 against itself as the idp, which isn't what we want. And even when they mention 3rd party idps https://fastapi.tiangolo.com/advanced/security/oauth2-scopes/#about-third-party-integrations they recommend the "implicit flow" which is not the recommended flow any longer https://oauth.net/2/grant-types/password/ So I'm unsure of how well it will support the recommended "auth code with pkce" flow, as I can't remember right know how much is different between those two flows wrt. the backend. I think it might be the same and just require the auth bearer token.
Alternatively, because we host our frontend application on the same domain as our API, we should be able to use OpenID Connect to authenticate with an idp and yet use cookies to simplify the code: https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-app-types#web-apps
This requires a small, simple, non-spa frontend application for the small login flow itself, before browser is redirected to the SPA. And maybe we need some communication with the idp from the backend as well: https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-app-types#web-apis which I don't know if we need any python library for.

Hi
I had a closer look at fastapi-azure-auth and found the following very informative closed issue: https://github.com/Intility/fastapi-azure-auth/issues/41#issuecomment-999705634
Basically, the package is only useful for SPA-like PKCE flows. I will try to see if I can cook this up in Flask.
Br Eskild

Hmm, yeah, it's probably the sessions stuff we need to be able to use cookies. That is, I think we need something in fastapi to handle user sessions and then only use cookies as a session id. I don't think we can do it in the flask login-only application. If we don't have anything that easily supports user sessions in fastapi, we should probably go with the SPA auth code flow with pkce.
But maybe this can be used? https://pypi.org/project/fastapi-sessions/

Closed since we decided to place authentication in the frontend. The frontend is responsible for getting tokens via PKCE flow and the backend is responsible for validating the token and verifying access to specific resources.


cannot install requirements.txt

I am having a great deal of difficulty installing the requirements in requirements.txt.
I am using Python 3.10.7
When trying to install the requirements.txt file I am informed the following packages cannot be found:

altair-pandas==0.1.0.dev0   [just does not exist]
cctx==1.54.19    [version unavailable, newer version available]
dataclasses==0.8   [version unavailable, newer version available]
pycaret==3.0.0.rc4   [version unavailable, newer version available]
pyfolio==0.9.2+75.g4b901f6   [version unavailable, newer version available]

I ultimately install pip-tools, created a requirements.in without any specified version and then executed:
pip-compile to let pip-tools package pick any combination of package versions that would work.  I received error message:
There are incompatible versions in the resolved dependencies:
numba (from -r requirements.in (line 183))
numba>=0.51.2 (from pynndescent==0.5.8->-r requirements.in (line 241))
numba>=0.51 (from pyod==1.0.7->-r requirements.in (line 242))
numba>=0.55 (from sktime==0.15.0->-r requirements.in (line 290))
numba>=0.49 (from umap-learn==0.5.3->-r requirements.in (line 344))
numba>=0.52.0 (from kats==0.2.0->-r requirements.in (line 146))
numba<0.55 (from pycaret==2.3.10->-r requirements.in (line 228))
numba (from shap==0.41.0->-r requirements.in (line 285))
I am not really sure how to get the requirements installed at this moment.  Any help or pointers would be appreciated.  If I find the solution I will update that there.
Thanks

Hi both, sorry for the late reply, I totally missed those messages. For the book, I was using Python 3.9.10. Please le time know if using that version solves the issues!


When creating a new tag, make sure it's strictly numeric (i.e. 1.1.0 instead of v1.1.0)

Discovered that the new tag of Camel K causes issues when the tooling tries to download it.
The new 1.1.0 version is tagged as "v1.1.0" whereas the rest of the versions are all 1.0.1, 1.0.0, etc. with no "v" - https://github.com/apache/camel-k/releases/tag/v1.1.0 vs. https://github.com/apache/camel-k/releases/tag/1.0.1
That extra "v" creates an issue in the url we construct to download the runtime CLI
Not found - https://github.com/apache/camel-k/releases/download/v1.1.0/camel-k-client-v1.1.0-linux-64bit.tar.gz
Found - https://github.com/apache/camel-k/releases/download/v1.1.0/camel-k-client-1.1.0-linux-64bit.tar.gz
We will work on avoiding this kind of problem in the tooling, but it caught us off guard and broke the download process.

Looks like this is a necessary change. I was not aware of apache/camel-k#1603 and apache/camel-k#1601. Will close and we are adapting the version checking to be better in the tooling


Low Latency HLS support

Any plans to add support for Low Latency HLS and MPEG-Dash on the road map?
Thanks for the very cool project, RTSP functionality is excellent.

Hello, LL HLS can be planned, extending the current implementation shouldn't be difficult.
Next time please open a issue for each new feature.

This would be a most excellent feature! There is pretty much nothing open source that does LL-HLS effectively, would make this one of the most popular open source projects out there.

+1
Perhaps Home Assistant's Stream component could be helpful with the implementation, as they have recently introduced support for it.

Low-Latency has been implemented and is being tested here: #938 .

added in v0.19.0 🎉


[Feature] Simplify restartStrategy related configuration

What problem does this PR solve?
Issue Number: close #1743
Problem Summary:
Click  #1739 to get detailed background.
Simplify restartStrategy related configuration.

StreamPark Configuration
Flink Configuration
Flink 1.12 doc
Flink 1.15 doc

flink.restart-strategy.value
restart-strategy
https://nightlies.apache.org/flink/flink-docs-release-1.12/deployment/config.html#restart-strategy
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/config/#restart-strategy

flink.restart-strategy.failure-rate.max-failures-per-interval
restart-strategy.failure-rate.max-failures-per-interval
https://nightlies.apache.org/flink/flink-docs-release-1.12/deployment/config.html#restart-strategy-failure-rate-max-failures-per-interval
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/config/#restart-strategy-failure-rate-max-failures-per-interval

flink.restart-strategy.failure-rate.failure-rate-interval
restart-strategy.failure-rate.failure-rate-interval
https://nightlies.apache.org/flink/flink-docs-release-1.12/deployment/config.html#restart-strategy-failure-rate-failure-rate-interval
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/config/#restart-strategy-failure-rate-failure-rate-interval

flink.restart-strategy.failure-rate.delay
restart-strategy.failure-rate.delay
https://nightlies.apache.org/flink/flink-docs-release-1.12/deployment/config.html#restart-strategy-failure-rate-delay
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/config/#restart-strategy-failure-rate-delay

flink.restart-strategy.fixed-delay.attempts
restart-strategy.fixed-delay.attempts
https://nightlies.apache.org/flink/flink-docs-release-1.12/deployment/config.html#restart-strategy-fixed-delay-attempts
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/config/#restart-strategy-fixed-delay-attempts

flink.restart-strategy.fixed-delay.delay
restart-strategy.fixed-delay.delay
https://nightlies.apache.org/flink/flink-docs-release-1.12/deployment/config.html#restart-strategy-fixed-delay-delay
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/config/#restart-strategy-fixed-delay-delay

What is changed and how it works?
Proposal: xxx
Feel free to ping committers for the review!
Contribution Checklist

Make sure that the pull request corresponds to a GITHUB issue.

Name the pull request in the form "[Feature] [component] Title of the pull request", where Feature can be replaced by Hotfix, Bug, etc.

Minor fixes should be named following this pattern: [hotfix] [docs] Fix typo in README.md doc.

Purpose of this pull request
Simplify restartStrategy related configuration.

hi fanrui:
It looks good, Thanks you for your contribution, BTW the document also needs to be updated

hi fanrui: It looks good, Thanks you for your contribution, BTW the document also needs to be updated

Good idea, I create an issue #1751 , I will update the doc when all configurations are done.


Interface to MathOptInterface

This will allow us to access the predefined sets, projections from MathOptSetDistances.jl.
From a user point of view, this means one can potentially use the algorithms from the package with the https://jump.dev interface

wait until nonlinear interface in jump is stable.


continuous running can not be terminated once its started

As stated here
The extension's creator believes its a vscode issue, so I repost here.

VS Code Version: 1.83.1
OS Version: Windows 11

Steps to Reproduce:

write a program with an infinite loop
write a cmakelist.
enable continuous running
compile and then try to stop it with the stop button.

Looks related to testing https://github.com/matepek/vscode-catch2-test-adapter/issues/398

Yea, it's on the extension, will follow up on that issue. Thanks Rob.

This issue is caused by an extension, please file it with the repository (or contact) the extension has linked in its overview in VS Code or the marketplace for VS Code. See also our issue reporting guidelines. If you don't know which extension is causing the problem, you can run Help: Start extension bisect from the command palette (F1) to help identify the problem extension.
Happy Coding!

When I click on "stop" button onCancellationRequested is not fired at https://github.com/matepek/vscode-catch2-test-adapter/blob/master/src/main.ts#L177.
I think it is a vscode issue.


fix(rbac): add missing permissions to helm clusterrole

This PR fixes missing permissions for the helm chart (recent addition of schedules support and configmap deletion).

Thanks @nerdingassendible !

Now released in: https://github.com/alexandrevilain/temporal-operator/releases/tag/helm-chart-0.5.0


Post-GRT, resizer fails with cryptic error

Describe the bug
Running post-GRT timing optimizations fails with a cryptic error and later a stack dump (see log), but only in the event that the database has been written after global routing, OpenROAD exited out of, then the database is re-read.
If GRT and resizing happens in the same script, this works fine. However, this may be a waste of compute for some designs where GRT takes a non-trivial amount of time.
Expected Behavior
Whatever data global routing saves should be in Odb/more actionable error
Environment
Git commit: 14b1cbdf0c3160dd34ff79cf413d1f66f9ec4eac
kernel: Darwin 23.5.0
os: macOS 14.5
cmake version 3.29.2
CMake Warning (dev) at src/sta/CMakeLists.txt:32 (option):
  Policy CMP0077 is not set: option() honors normal variables.  Run "cmake
  --help-policy CMP0077" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  For compatibility with older versions of CMake, option is clearing the
  normal variable 'USE_TCL_READLINE'.
This warning is for project developers.  Use -Wno-dev to suppress it.

Number of processor cores: 24
CMake Warning:
  Manually-specified variables were not used by the project:

    VERBOSE

-- The CXX compiler identification is Clang 16.0.6
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/c1ib07kj7baid3s3wf5m98hcpbvgs8mk-clang-wrapper-16.0.6/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- OpenROAD version: v2.0-15221-g14b1cbdf0
-- System name: Darwin
-- Compiler: Clang 16.0.6
-- Build type: RELEASE
-- Install prefix: /var/empty/local
-- C++ Standard: 17
-- C++ Standard Required: ON
-- C++ Extensions: OFF
-- The C compiler identification is Clang 16.0.6
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/c1ib07kj7baid3s3wf5m98hcpbvgs8mk-clang-wrapper-16.0.6/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test C_COMPILER_SUPPORTS__-Wall
-- Performing Test C_COMPILER_SUPPORTS__-Wall - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wall
-- Performing Test CXX_COMPILER_SUPPORTS__-Wall - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-array-bounds
-- Performing Test C_COMPILER_SUPPORTS__-Wno-array-bounds - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-array-bounds
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-array-bounds - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-nonnull
-- Performing Test C_COMPILER_SUPPORTS__-Wno-nonnull - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-nonnull
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-nonnull - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-maybe-uninitialized
-- Performing Test C_COMPILER_SUPPORTS__-Wno-maybe-uninitialized - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-maybe-uninitialized
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-maybe-uninitialized - Failed
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format-overflow
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format-overflow - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format-overflow
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format-overflow - Failed
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-variable
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-variable - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-variable
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-variable - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-function
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-function - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-function
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-function - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-write-strings
-- Performing Test C_COMPILER_SUPPORTS__-Wno-write-strings - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-write-strings
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-write-strings - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-sign-compare
-- Performing Test C_COMPILER_SUPPORTS__-Wno-sign-compare - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-sign-compare
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-sign-compare - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-deprecated
-- Performing Test C_COMPILER_SUPPORTS__-Wno-deprecated - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-deprecated
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-deprecated - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-c++11-narrowing
-- Performing Test C_COMPILER_SUPPORTS__-Wno-c++11-narrowing - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-c++11-narrowing
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-c++11-narrowing - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-register
-- Performing Test C_COMPILER_SUPPORTS__-Wno-register - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-register
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-register - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format
-- Performing Test C_COMPILER_SUPPORTS__-Wno-format - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-format - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal
-- Performing Test C_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-reserved-user-defined-literal - Success
-- Performing Test C_COMPILER_SUPPORTS__-fpermissive
-- Performing Test C_COMPILER_SUPPORTS__-fpermissive - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-fpermissive
-- Performing Test CXX_COMPILER_SUPPORTS__-fpermissive - Success
-- Performing Test C_COMPILER_SUPPORTS__-x
-- Performing Test C_COMPILER_SUPPORTS__-x - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-x
-- Performing Test CXX_COMPILER_SUPPORTS__-x - Failed
-- Performing Test C_COMPILER_SUPPORTS__c++
-- Performing Test C_COMPILER_SUPPORTS__c++ - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__c++
-- Performing Test CXX_COMPILER_SUPPORTS__c++ - Failed
-- Performing Test C_COMPILER_SUPPORTS__-std=c++17
-- Performing Test C_COMPILER_SUPPORTS__-std=c++17 - Failed
-- Performing Test CXX_COMPILER_SUPPORTS__-std=c++17
-- Performing Test CXX_COMPILER_SUPPORTS__-std=c++17 - Success
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-but-set-variable
-- Performing Test C_COMPILER_SUPPORTS__-Wno-unused-but-set-variable - Success
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-but-set-variable
-- Performing Test CXX_COMPILER_SUPPORTS__-Wno-unused-but-set-variable - Success
-- TCL library: /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/lib/libtcl.dylib
-- TCL header: /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/include/tcl.h
-- TCL readline library: /nix/store/0cwgrwkw5kfshdz7bkqjmhm9clas9fqs-tclreadline-2.3.8/lib/libtclreadline.dylib
-- TCL readline header: /nix/store/0cwgrwkw5kfshdz7bkqjmhm9clas9fqs-tclreadline-2.3.8/include
-- Found SWIG: /nix/store/q6jgc1rpam7xqrylwq6gxq2f3arcmhvs-swig-4.0.2/bin/swig (found suitable version "4.0.2", minimum required is "4.0")
-- Found Boost: /nix/store/67hsz87y3shaq2qsql57w5gg8wih8g2m-boost-1.83.0-dev/lib/cmake/Boost-1.83.0/BoostConfig.cmake (found version "1.83.0")
-- boost: 1.83.0
-- Found Python3: /nix/store/0hhzvkw889bsybhqxy12ky4jx6a95p2d-python3-3.11.9/include/python3.11 (found version "3.11.9") found components: Development Development.Module Development.Embed
-- Found ZLIB: /nix/store/5g9k8zyjv029pw87xf95pci414r8pw8j-zlib-1.3.1/lib/libz.dylib (found version "1.3.1")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- spdlog: 1.13.0
-- Found BISON: /nix/store/zz65ns9x0hczhhdq74905dk297z29zvn-bison-3.8.2/bin/bison (found version "3.8.2")
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- STA version: 2.6.0
-- STA git sha: 4fe6f220a7e3fbf18aff5b59aa1c16295f7bb702
-- System name: Darwin
-- Compiler: Clang 16.0.6
-- Build type: RELEASE
-- Build CXX_FLAGS: -O3 -DNDEBUG
-- Install prefix: /var/empty/local
-- Found FLEX: /nix/store/n0kkb647sybl2320a1pk85681jbb05p6-flex-2.6.4/bin/flex (found version "2.6.4")
-- TCL library: /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/lib/libtcl.dylib
-- TCL header: /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/include/tcl.h
-- TCL readline library: /nix/store/0cwgrwkw5kfshdz7bkqjmhm9clas9fqs-tclreadline-2.3.8/lib/libtclreadline.dylib
-- TCL readline header: /nix/store/0cwgrwkw5kfshdz7bkqjmhm9clas9fqs-tclreadline-2.3.8/include/tclreadline.h
-- CUDD library: /nix/store/yklam2c2z7i3msfg106r2v3g46wprjwn-cudd-3.0.0/lib/libcudd.a
-- CUDD header: /nix/store/yklam2c2z7i3msfg106r2v3g46wprjwn-cudd-3.0.0/include/cudd.h
-- SSTA: 0
-- Found SWIG: /nix/store/q6jgc1rpam7xqrylwq6gxq2f3arcmhvs-swig-4.0.2/bin/swig (found suitable version "4.0.2", minimum required is "3.0")
-- STA executable: /Users/donn/efabless/openroad/src/sta/app/sta
-- Found Protobuf: /nix/store/a4n8n8wh80wllzrcgrsi94akgj22417g-protobuf-21.12/lib/libprotobuf.dylib (found version "3.21.12")
-- Found re2: /nix/store/ha95iybvpwxg89q3d7q0453fxccxx6dl-re2-2024-05-01-dev/lib/cmake/re2/re2Config.cmake (found version "11.0.0")
-- Found GLPK: /nix/store/zv2pdp1np8n19qgzk0x10kdp2pwfahja-glpk-5.0/lib/libglpk.dylib
-- Found OpenMP_CXX: -fopenmp=libomp (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0")
-- TCL library: /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/lib/libtcl.dylib
-- TCL header: /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/include/tcl.h
-- Found OpenMP_C: -fopenmp=libomp (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0")
-- GUI is enabled
-- Charts widget is enabled
-- Found Boost: /nix/store/67hsz87y3shaq2qsql57w5gg8wih8g2m-boost-1.83.0-dev/lib/cmake/Boost-1.83.0/BoostConfig.cmake (found version "1.83.0") found components: serialization
-- Could NOT find VTune (missing: VTune_LIBRARIES VTune_INCLUDE_DIRS)
-- Found Boost: /nix/store/67hsz87y3shaq2qsql57w5gg8wih8g2m-boost-1.83.0-dev/lib/cmake/Boost-1.83.0/BoostConfig.cmake (found suitable version "1.83.0", minimum required is "1.78")
-- TCL library: /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/lib/libtcl.dylib
-- TCL header: /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/include/tcl.h
-- Found Boost: /nix/store/67hsz87y3shaq2qsql57w5gg8wih8g2m-boost-1.83.0-dev/lib/cmake/Boost-1.83.0/BoostConfig.cmake (found version "1.83.0") found components: serialization system thread
-- Found Eigen3: /nix/store/2arrr3d4ixycdq5lmpqxydz30pip7h7m-eigen-3.4.0/share/eigen3/cmake/Eigen3Config.cmake (found version "3.4.0")
-- TCL readline enabled
-- Tcl Extended disabled
-- Python3 enabled
-- Configuring done (27.3s)
-- Generating done (0.8s)
-- Build files have been written to: /tmp/nix-shell.w0nava/tmp.GMzJwUoQd6

To Reproduce
Concatenate the file to extract it.
./run_good.sh is the example with both GRT and RSZ in the same script that works.
./run_bad.sh is the example with GRT and RSZ in separate scripts that fails.
repro.partaa.tgz
repro.partab.tgz
Relevant log output
Reading OpenROAD database at './intermediate_bad.odb'…
Reading design constraints file at './files/Users/donn/efabless/caravel_openframe_project/openlane/picosoc/base.sdc'…
[INFO]: Setting output delay to: 22
[INFO]: Setting input delay to: 0
[WARNING STA-0441] set_input_delay relative to a clock defined on the same port/pin not allowed.
[WARNING STA-0441] set_input_delay relative to a clock defined on the same port/pin not allowed.
[WARNING STA-0441] set_input_delay relative to a clock defined on the same port/pin not allowed.
[INFO]: Cap load range: 0.5 : 1.0
[INFO]: Input transition range: 1 : 1.19
[INFO]: Setting derate factor to: 15.0 %
[INFO] Setting RC values…
[INFO RSZ-0094] Found 557 endpoints with setup violations.
[INFO RSZ-0099] Repairing 557 out of 557 (100.00%) violating endpoints...
[ERROR GRT-0226] Type2 ripup not type L.
Error: rsz_timing_postgrt.tcl, 36 GRT-0226
Signal 11 received
Stack trace:
 0# handler(int) in /nix/store/x1n09k7mal26ln8z7npr0nhd8qq21846-openroad/bin/.openroad-wrapped
 1# _sigtramp in /usr/lib/system/libsystem_platform.dylib
 2# std::__1::__tree<std::__1::__value_type<sta::Instance*, std::__1::tuple<sta::LibertyPort*, sta::LibertyPort*>>, std::__1::__map_value_compare<sta::Instance*, std::__1::__value_type<sta::Instance*, std::__1::tuple<sta::LibertyPort*, sta::LibertyPort*>>, std::__1::less<sta::Instance*>, true>, std::__1::allocator<std::__1::__value_type<sta::Instance*, std::__1::tuple<sta::LibertyPort*, sta::LibertyPort*>>>>::destroy(std::__1::__tree_node<std::__1::__value_type<sta::Instance*, std::__1::tuple<sta::LibertyPort*, sta::LibertyPort*>>, void*>*) in /nix/store/x1n09k7mal26ln8z7npr0nhd8qq21846-openroad/bin/.openroad-wrapped
 3# std::__1::__tree<std::__1::__value_type<sta::Instance*, std::__1::tuple<sta::LibertyPort*, sta::LibertyPort*>>, std::__1::__map_value_compare<sta::Instance*, std::__1::__value_type<sta::Instance*, std::__1::tuple<sta::LibertyPort*, sta::LibertyPort*>>, std::__1::less<sta::Instance*>, true>, std::__1::allocator<std::__1::__value_type<sta::Instance*, std::__1::tuple<sta::LibertyPort*, sta::LibertyPort*>>>>::destroy(std::__1::__tree_node<std::__1::__value_type<sta::Instance*, std::__1::tuple<sta::LibertyPort*, sta::LibertyPort*>>, void*>*) in /nix/store/x1n09k7mal26ln8z7npr0nhd8qq21846-openroad/bin/.openroad-wrapped
 4# rsz::Resizer::~Resizer() in /nix/store/x1n09k7mal26ln8z7npr0nhd8qq21846-openroad/bin/.openroad-wrapped
 5# rsz::Resizer::~Resizer() in /nix/store/x1n09k7mal26ln8z7npr0nhd8qq21846-openroad/bin/.openroad-wrapped
 6# ord::OpenRoad::~OpenRoad() in /nix/store/x1n09k7mal26ln8z7npr0nhd8qq21846-openroad/bin/.openroad-wrapped
 7# __cxa_finalize_ranges in /usr/lib/system/libsystem_c.dylib
 8# exit in /usr/lib/system/libsystem_c.dylib
 9# ord::tclAppInit(Tcl_Interp*) in /nix/store/x1n09k7mal26ln8z7npr0nhd8qq21846-openroad/bin/.openroad-wrapped
10# Tcl_MainEx in /nix/store/7cpj5zdlz28y6ipg2h4cw6qagvqycm0i-tcl-8.6.13/lib/libtcl8.6.dylib
11# main in /nix/store/x1n09k7mal26ln8z7npr0nhd8qq21846-openroad/bin/.openroad-wrapped
./run_bad.sh: line 17: 37378 Segmentation fault: 11  openroad -exit -no_splash -metrics ./or_metrics_out.json ./scripts/openroad/rsz_timing_postgrt.tcl

Screenshots
No response
Additional Context
No response

[ERROR GRT-0226] Type2 ripup not type L. suggest the problem is in the fastroute core.

The current incremental grt relies on running grt and rsz in the same script. This is because the internal structures that grt uses to estimate the parasitics and to define what nets are already routed are not present in the database.
To fix this issue, we would need to save the route segments of each net in addition to the route guides. This way rsz will get the correct data about connectivity between the nets.

If GRT and resizing happens in the same script, this works fine. However, this may be a waste of compute for some designs where GRT takes a non-trivial amount of time.

Most likely any fix is going to involve re-running both.  Storing all the extra data in the db seems like a lot of overhead for a small benefit.
The error message should be better regardless.

fyi @kareefardi

If GRT and resizing happens in the same script, this works fine. However, this may be a waste of compute for some designs where GRT takes a non-trivial amount of time.

Most likely any fix is going to involve re-running both. Storing all the extra data in the db seems like a lot of overhead for a small benefit.
The error message should be better regardless.

How much is a lot of overhead? Also, how much would rerunning GRT modify existing GRT data (guides only I suppose) in the db?

You are storing the routing of each net twice - once in guides and again in segments.  I don't have a quotative value but that isn't trivial.
I'm not sure what the value of this use model is.  Would you explain further how this is a benefit.

I'm personally unconcerned. Would you like me to keep this issue open so you can track "The error message should be better regardless"?

Sure the message should be fixed.

@eder-matheus @maliberty I just noticed the commands read_global_route_segments and write_global_route_segments. Would rsz still need an iteration of GRT even with these commands

Those were added for a contest and I don't know we'll keep them long term.

@kareefardi This PR will make it possible to estimate parasitics from the guides stored in odb: https://github.com/The-OpenROAD-Project/OpenROAD/pull/5758


Cygwin setup update

Codecov Report

Merging #1322 into master will decrease coverage by 0.54%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1322      +/-   ##
==========================================
- Coverage   93.67%   93.12%   -0.55%
==========================================
  Files          45       45
  Lines        4696     4774      +78
  Branches      513      807     +294
==========================================
+ Hits         4399     4446      +47
- Misses        297      301       +4
- Partials        0       27      +27

Impacted Files
Coverage Δ

browser/model/devstudio-autoinstall.js
97.29% <0%> (-2.71%)
:arrow_down:

browser/services/request.js
97.61% <0%> (-2.39%)
:arrow_down:

browser/pages/install/controller.js
91.86% <0%> (-1.75%)
:arrow_down:

browser/model/helpers/downloader.js
95.53% <0%> (-1.68%)
:arrow_down:

browser/model/che.js
95% <0%> (-1.62%)
:arrow_down:

browser/services/platform.js
97.05% <0%> (-1.41%)
:arrow_down:

browser/pages/account/controller.js
98.61% <0%> (-1.39%)
:arrow_down:

browser/model/cdk.js
98.74% <0%> (-1.26%)
:arrow_down:

browser/model/devstudio-p2-zip.js
64.4% <0%> (-1.12%)
:arrow_down:

browser/model/installable-item.js
92.93% <0%> (-1.04%)
:arrow_down:

... and 23 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7d8f38a...e1ce8aa. Read the comment docs.


@conda-forge-admin, please rerender

[ ] I read the conda-forge documentation and could not find the solution for my problem there.

Issue:

Environment (conda list):

$ conda list

Details about  conda and system ( conda info ):

$ conda info

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I rerendered the recipe in conda-forge/boto3-stubs-feedstock#11.


Enhancer doesn't work on Windows 10 21H2

What is happening?
Clean install of Notion-Enhanced-Setup-2.0.18-1.exe doesn't give Enhancer features, it only runs standard Notion app, no Enhancer menu in the task bar to customize
What should be happening?
notion enhancer icon to be present in the taskbar
Operating system/browser name & version:
Windows 10 Pro 10.0.19044 Build 19044
notion-enhancer and/or notion-repackaged version:
Notion-Enhanced-Setup-2.0.18-1.exe

Solved by cleaning, removing all prior notion enhancer files in C:\Users\


feat(verifier): don't retry verifying permanently ignored batches

Currently, the TEE verifier – the tool for continuous SGX attestation and batch signature verification – is stuck on batches that failed to be proven and are marked as permanently_ignored. The tool should be able to distinguish between batches that are permanently ignored (and should be skipped) and batches that have failed but will be retried. This PR enables that distinction.
This commit goes hand in hand with the following PR: https://github.com/matter-labs/zksync-era/pull/3321

I just don't understand why the check-spdx-headers CI check is failing. I haven't created any new files that need a license header. :thinking_face:


Move /usr/bin/env to own module

Move the /usr/bin/env script ot its own module instead of being hardcoded on the nodeos-initramfs module, and replace it by a symlink. Their parent folders could get removed the write & execution permissions, and also they could be removed for OverlayFS and being still available, so they don't appear in users filesystem.

I've done this on https://github.com/piranna/usrbinenv, clossing.


Have upstart wait for Monit to be up and running

grep has to be done since monit always exits with 0. -v is an inverse, so that means that it does exit 0 as soon as it stop seeing the error
Using start delay as it should be enough to only do one iteration but doing a loop just in case
This will allow Chef and other things to wait until Monit is actually contactable via the HTTP interface and ready to be used.

:metal: i like it! can you fix the rubocop complaint and then i'll get it merged?


fix(plugins): prevent error when transparent bg is enabled

My proposal to fix #47 with a non- over-engineered solution.
Do not attempt to pass the "NONE" string to shader.replicate_shade() via groups.MenuButtonSelected(). Instead, fall back to a sane default.
Alternatively, a check + fallback could be added inside groups.MenuButtonSelected() directly, but this would prevent callers (e.g. plugin customizations) from picking their own default.

A few screen captures 🖼
Normal buffer

Mason

Lazy (does not use NormalFloat, interestingly)

Light background

@dartey25 this should fix your issue, in case you want to give it a try.

@antoineco Hey! Thank you sm for helping me with this! But I can't seem to make it work. Sorry, I am still very new to neovim and lua.
So I'm trying to pull the fork you did with this config
  {
    "antoineco/ramojus-mellifluous.nvim",
    name = "mellifluous",
    branch = "fix/transparent-bg",
    config = function()
      require("mellifluous").setup({
        transparent_background = {
          enabled = true,
        },
      })
    end,
  },

  {
    "LazyVim/LazyVim",
    opts = {
      colorscheme = "mellifluous",
    },
  },

But still get the same error. I tried the :Lazy sync, and also tried to delete the ~/.local/share/nvim/lazy directory to do a clean install, but no luck yet

@dartey25 Your configuration is correct, the mistake is on my side 🙏
I had a few plugin integrations disabled in the colorscheme while I tested, such as nvim_tree. It turns out that some of these plugin integrations were also failing.
Please check #50 instead, which supersedes this PR.

@antoineco check out #50, my alternative. It covers a wider range of issues, e.g. when I tried a similar fix as yours, I also got similar error for NvimTree, but I see you've just commented about it :D


Preferences option to display displayId or name (if specified)

I've changed it to show Name first, if specified, then displayId.  Might still be good to make a preference.

I've added a preferences setting for whether to always show displayId or always show name if set.


Consider hardlinks in image size.

Fixes #9283. Based on #8984. This patch fixes behavior when image size calculation
didn't consider hardlinks.
Signed-off-by: Dmitry Vorobev<EMAIL_ADDRESS>

/cc @vbatts

I think that if inode number is same, then it's hardlink and it makes sense to exclude it from size. So Nlink check maybe is redundant.

@LK4D4 Have made some fixes according to your comments.

@LK4D4 Updated PR according to your comments.

@LK4D4 Fixed.

LGTM

@LK4D4 I returned IsRegular() check in archive.go:255, because I removed this check from hasHardLinks function.

LGTM
Wish for future feature, I would love to see Size (virtual) and Size (allocated). Since that same syscall.Stat_t can give you stat.Blksize * stat.Blocks. Especially for these block level CoW storage, you could see the potential savings very clearly.

@LK4D4 @vbatts I've removed IsRegular check and changed it with !IsDir in archive.go:255

Nicely. LGTM

@vbatts we can merge then, I think.


Move service description lower

When service is in error state, system will attempt to restart vsftpd before updating /etc/vsftpd.conf.  Chef will forever fail. Updated recipes/_configure.rb to put service description below template calls.  The "notifies" directive still works correctly as it appears the string 'service[vsftpd]' is resolved during the run-phase instead of compile-phase.

Sorry I totally forgot to merge this one! My bad :-/


WARN Unexpected DN

Hi ldap2pg Team
i have configured my ldap i think correct, but when i want to read out my 2 test users from ldap i have this error:
12:37:21 DEBUG  LDAP search done.                                duration=1.859295ms entries=1
12:37:21 DEBUG  Got LDAP entry.                                  dn="cn=test,cn=groups,l=world,o=example,dc=local,dc=domain"
12:37:21 WARN   Unexpected DN.                                   dn="uid=test1,ou=user,l=world,o=example,dc=local,dc=domain" rdn=cn
12:37:21 WARN   Unexpected DN.                                   dn="uid=test2,ou=user,l=world,o=example,dc=local,dc=domain" rdn=cn
I dont see this test users in my postgres instance, so i think it did not make here correct. did you know what is here wrong?
regards
franco

the ldapsearch cmd from debug show me this 2 users correct

test, groups, world, example, local.domain
dn: cn=test,cn=groups,l=world,o=example,dc=local,dc=domain
uniquemember: uid=test1,ou=user,l=world,o=example,dc=local,dc=domain
uniquemember: uid=test2,ou=user,l=world,o=example,dc=local,dc=domain
objectclass: top
objectclass: groupOfUniquenames
cn: test

Hi @pc-dok . Please share the ldap2pg.yml rule.

`version: 6
postgres:
roles_blacklist_query: [postgres, pg_*]
privileges:
Define ro privilege group with read-only grants
ro:

connect
select_on_tables
select_on_sequences
usage_on_schemas
usage_on_types

rules:

description: "Setup static roles and grants."
roles:

names:

PG01_dba
options: NOLOGIN
grant:

privilege: ro
role: PG01_dba

description: "Search LDAP to create readers, writers and owners."
ldapsearch:
base: cn=groups,l=world,o=example,dc=local,dc=domain
filter: "
(|
(cn=PG01_dba)
)
"
role:
name: '{uniquemember.cn}'
options: LOGIN
parent: "{cn}"

`

so i go right in my mind, that when i have 2 members in this PG01_dba ldap group, that ldap2pg will create a role with PG01_dba, and also 2 roles more for my 2 users, or am i wrong?

so i go right in my mind, that when i have 2 members in this PG01_dba ldap group, that ldap2pg will create a role with PG01_dba, and also 2 roles more for my 2 users, or am i wrong?

Hmm, I guess you hit a weird behaviour. ldap2pg does not sub-search cn and tries to read it from Distinguished Name. However, your Distinguished Name has uid.
ldap2pg should trigger a sub-search to get each cn. This can be costy. Can you use {uniquemember.sAMAccountName} ?

version: 6

postgres:
  roles_blacklist_query: [postgres, pg_*]

privileges:
  # Define `ro` privilege group with read-only grants
  ro:
  - __connect__
  - __select_on_tables__
  - __select_on_sequences__
  - __usage_on_schemas__
  - __usage_on_types__

rules:
- description: "Setup static roles and grants."
  roles:
  - names:
    - PG01_dba
    options: NOLOGIN
  grant:
  - privilege: ro
    role: PG01_dba

- description: "Search LDAP to create readers, writers and owners."
  ldapsearch:
    base: cn=groups,l=world,o=example,dc=local,dc=domain
    filter: "
    (|
      (cn=PG01_dba)
    )
    "
  role:
    name: '{uniquemember.uid}'
    options: LOGIN

so it works

for all that is a Oracle Universal Directory - OUD

Ok. I think a better message would help. Something like Missing RDN cn from DN uid=... ?

Fixed in #642


windows: setting logging path in fabio properties

Hi There am I able to set log to a logging path (for example d:\fabio_logs\fabio.log). We have some windows nomad hosts that are running fabio as a windows service.
Reading the fabio.properties, it only offers stdout which is screen in windows (unless I'm mistaken, often am) and not that helpful.

It is possible to redirect standard output to a file on Windows, just as it is on *nix.  Does this get you what you need?  But you are correct, the only logging output currently supported is stdout.

@lcividinwork you're welcome!  And, as always, PR's welcome!


Gateway timeout for exporting demographic data

Describe the bug
I'm logged in as the CiviForm Administrator. I'd like to download an export of all demographic information. I get a "504 gateway timeout" error.
To Reproduce
Steps to reproduce the behavior:

Go to CiviForm.seattle.gov
Click on Log in as Admin
Scroll down to Download Export to CSV
See error

Screenshots
Attached

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Likely similar to #1730, where the data collation is taking too long.
For that I had to optimize the export by removing a redundant lookup done on every row.

@elisekalstad is the action and data you're working with the same as in #1730?

@shanemc-goog - Yeah, it's the exact same issue and action and data and everything. Just figure our data is growing :)

It may be time to make this an asynchronous interaction. There's a couple way that could be presented in the UI, but on the backend this will require us to configure some manner of job runner. Play has a few options for this, there are also open source libraries that could help.

After a quick look, I think we may be able to improve performance enough here to avoid the background job solution for a while longer.

@gwendolyngoetz the PR for reducing N+1 queries is out on staging now. It's possible getting it to production will be an adequate fix for the short term. After it's deployed and @elisekalstad has attempted another export please update here with whether this is still an immediate issue.

@bion I was able to export the demographics data on 2/3 attempts. The first attempt was the same 504 error.

@gwendolyngoetz sounds like the performance improvement may have bought us a little time. Hopefully this will be enough to get the export done for May 1. If not are you able to temporarily increase the timeout?

@bion I should have access, might need to be pointed at the right place though

@gwendolyngoetz this should do it https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers.html#connection-idle-timeout

Not seeing any further errors right now so we will leave the timeout as is unless it becomes a problem.

Getting 504's again. @bion  I raised the timeout as outlined in that above aws link, but that resulted in 502's. There's probably something else that needs longer timeout, but I'm not sure what.

@gwendolyngoetz have you checked the server logs to see if the 502s are coming from the application? If they're not there then it'll be something on the AWS side that needs adjusting.

This has mostly been resolved by the feature to specify a time range for the export. Closing for now.


Enable flox in GroupBy and resample

This PR supersedes https://github.com/pydata/xarray/pull/4540. This is still a very rough draft :)

Closes

[x] Closes #4473
[x] Closes #4498
[x] Closes #659
[x] Closes #2237
[x] xref https://github.com/pangeo-data/pangeo/issues/271

[x] Tests added
[x] Passes pre-commit run --all-files
[x] User visible changes (including notable bug fixes) are documented in whats-new.rst

TODO:

[x] flox vs legacy tests with nans
[x] test passing flox-specific kwargs like engine and method.

We don't have any asv benchmarks for groupby currently. It would be good to add some!

!!!
The only failures are in test_units.py so now I think we can figure out how to implement this cleanly.
FAILED xarray/tests/test_units.py::TestDataArray::test_computation_objects[float64-method_groupby-data]
FAILED xarray/tests/test_units.py::TestDataArray::test_computation_objects[float64-method_groupby_bins-data]
FAILED xarray/tests/test_units.py::TestDataArray::test_computation_objects[int64-method_groupby-data]
FAILED xarray/tests/test_units.py::TestDataArray::test_computation_objects[int64-method_groupby_bins-data]
FAILED xarray/tests/test_units.py::TestDataArray::test_resample[float64] - pi...
FAILED xarray/tests/test_units.py::TestDataArray::test_resample[int64] - pint...
FAILED xarray/tests/test_units.py::TestDataset::test_computation_objects[float64-data-method_groupby_bins]
FAILED xarray/tests/test_units.py::TestDataset::test_computation_objects[int64-data-method_groupby_bins]
FAILED xarray/tests/test_units.py::TestDataset::test_resample[float64-data]
FAILED xarray/tests/test_units.py::TestDataset::test_resample[int64-data] - p...

I like @max-sixty's suggestion of generating the reductions like generate_ops.py. It seems like a good first step would be to refactor the existing reductions in a separate  PR.

Maybe it's also on the pint side? Even if numpy_groupies supports the like argument it will crash because pint doesn't support asanyarray.
import numpy as np
import pint
import dask.array as da

# pint crashes:
np.asanyarray([1, 2], like=pint.Quantity(1, "s"))
Traceback (most recent call last):

  File "<ipython-input-11-f8c20802ca08>", line 1, in <module>
    np.asanyarray([1, 2], like=pint.Quantity(1, "s"))

TypeError: no implementation found for 'numpy.asanyarray' on types that implement __array_function__: [<class 'pint.quantity.Quantity'>]

# dask supports it:
np.asanyarray([1, 2], like=da.array(0))
Out[12]: dask.array<array, shape=(2,), dtype=int32, chunksize=(2,), chunktype=numpy.ndarray>

Maybe it's also on the pint side? Even if numpy_groupies supports the like argument it will crash because pint doesn't support asanyarray.

cc @keewis

Benchmarks are looking good (npg=True means use numpy groupies). Big gains (10-20x) for large number of groups (100), especially with dask.
[  2.78%] ··· groupby.GroupBy.time_agg_large_num_groups                                                                                ok
[  2.78%] ··· ======== ========== =========== ========== ===========
              --                         ndim / npg
              -------- ---------------------------------------------
               method   1 / True   1 / False   2 / True   2 / False
              ======== ========== =========== ========== ===========
                sum     8.38±0ms    101±0ms    9.54±0ms    136±0ms
                mean    7.12±0ms    101±0ms    9.74±0ms    148±0ms
              ======== ========== =========== ========== ===========

[  5.56%] ··· groupby.GroupBy.time_agg_small_num_groups                                                                                ok
[  5.56%] ··· ======== ========== =========== ========== ===========
              --                         ndim / npg
              -------- ---------------------------------------------
               method   1 / True   1 / False   2 / True   2 / False
              ======== ========== =========== ========== ===========
                sum     8.27±0ms    4.55±0ms   9.07±0ms    8.46±0ms
                mean    7.19±0ms    4.50±0ms   9.24±0ms    8.36±0ms
              ======== ========== =========== ========== ===========

[  8.33%] ··· groupby.GroupBy.time_init                                                                                ok
[  8.33%] ··· ====== ==========
               ndim
              ------ ----------
                1     1.72±0ms
                2     4.06±0ms
              ====== ==========

[ 11.11%] ··· groupby.GroupByDask.time_agg_large_num_groups                                                                                ok
[ 11.11%] ··· ======== ========== =========== ========== ===========
              --                         ndim / npg
              -------- ---------------------------------------------
               method   1 / True   1 / False   2 / True   2 / False
              ======== ========== =========== ========== ===========
                sum     8.41±0ms    202±0ms    9.93±0ms    226±0ms
                mean    7.83±0ms    197±0ms    10.7±0ms    213±0ms
              ======== ========== =========== ========== ===========

[ 13.89%] ··· groupby.GroupByDask.time_agg_small_num_groups                                                                                ok
[ 13.89%] ··· ======== ========== =========== ========== ===========
              --                         ndim / npg
              -------- ---------------------------------------------
               method   1 / True   1 / False   2 / True   2 / False
              ======== ========== =========== ========== ===========
                sum     8.41±0ms    8.99±0ms   10.5±0ms    12.5±0ms
                mean    7.98±0ms    8.67±0ms   10.1±0ms    12.2±0ms
              ======== ========== =========== ========== ===========

[ 16.67%] ··· groupby.GroupByDask.time_init                                                                                ok
[ 16.67%] ··· ====== ==========
               ndim
              ------ ----------
                1     1.77±0ms
                2     4.06±0ms
              ====== ==========

[ 36.11%] ··· groupby.Resample.time_agg_large_num_groups                                                                                ok
[ 36.11%] ··· ======== ========== =========== ========== ===========
              --                         ndim / npg
              -------- ---------------------------------------------
               method   1 / True   1 / False   2 / True   2 / False
              ======== ========== =========== ========== ===========
                sum     17.2±0ms    83.3±0ms   17.0±0ms    93.5±0ms
                mean    15.5±0ms    91.0±0ms   17.4±0ms    101±0ms
              ======== ========== =========== ========== ===========

[ 38.89%] ··· groupby.Resample.time_agg_small_num_groups                                                                                ok
[ 38.89%] ··· ======== ========== =========== ========== ===========
              --                         ndim / npg
              -------- ---------------------------------------------
               method   1 / True   1 / False   2 / True   2 / False
              ======== ========== =========== ========== ===========
                sum     16.7±0ms    12.3±0ms   16.7±0ms    13.3±0ms
                mean    15.2±0ms    12.5±0ms   19.3±0ms    13.9±0ms
              ======== ========== =========== ========== ===========

[ 41.67%] ··· groupby.Resample.time_init                                                                                ok
[ 41.67%] ··· ====== ==========
               ndim
              ------ ----------
                1     7.46±0ms
                2     7.26±0ms
              ====== ==========

[ 44.44%] ··· groupby.ResampleDask.time_agg_large_num_groups                                                                                ok
[ 44.44%] ··· ======== ========== =========== ========== ===========
              --                         ndim / npg
              -------- ---------------------------------------------
               method   1 / True   1 / False   2 / True   2 / False
              ======== ========== =========== ========== ===========
                sum     22.3±0ms    561±0ms    28.3±0ms    607±0ms
                mean    22.2±0ms    344±0ms    27.3±0ms    371±0ms
              ======== ========== =========== ========== ===========

[ 47.22%] ··· groupby.ResampleDask.time_agg_small_num_groups                                                                                ok
[ 47.22%] ··· ======== ========== =========== ========== ===========
              --                         ndim / npg
              -------- ---------------------------------------------
               method   1 / True   1 / False   2 / True   2 / False
              ======== ========== =========== ========== ===========
                sum     17.7±0ms    31.2±0ms   20.0±0ms    34.2±0ms
                mean    17.2±0ms    24.4±0ms   19.9±0ms    26.6±0ms
              ======== ========== =========== ========== ===========

[ 50.00%] ··· groupby.ResampleDask.time_init                                                                                ok
[ 50.00%] ··· ====== ==========
               ndim
              ------ ----------
                1     7.43±0ms
                2     6.91±0ms
              ==

OK CI isn't using the numpy_groupies code path for reasons I don't understand. Does anyone see a reason why this might happen?

This builds on #5950 so that should be reviewed and merged first.

@pydata/xarray this is blocked by https://github.com/pydata/xarray/issues/6430 but is ready for review.

There's a bit of mypy errors now. Maybe just removíng _flox_reduce in _reductions.py is enough?

Successfully installed types-PyYAML-6.0.7 types-cryptography-3.3.21 types-paramiko-2.10.0 types-python-dateutil-2.8.14 types-pytz-2021.3.7 types-setuptools-57.4.14
xarray/backends/locks.py:10: error: Cannot assign to a type  [misc]
xarray/backends/locks.py:10: error: Incompatible types in assignment (expression has type "Type[Lock]", variable has type "Type[SerializableLock]")  [assignment]
xarray/backends/locks.py:15: error: Cannot assign to a type  [misc]
xarray/backends/locks.py:15: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Lock]")  [assignment]
xarray/core/types.py:19: error: Cannot assign to a type  [misc]
xarray/core/types.py:19: error: Incompatible types in assignment (expression has type "Type[ndarray[Any, Any]]", variable has type "Type[Array]")  [assignment]
xarray/core/dask_array_compat.py:11: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
xarray/core/duck_array_ops.py:33: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
xarray/core/nanops.py:14: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
xarray/core/_reductions.py:15: error: Skipping analyzing "flox": module is installed, but missing library stubs or py.typed marker  [import]
xarray/core/_reductions.py:15: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
xarray/core/_reductions.py:3194: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:3203: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:3278: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:3287: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:3362: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:3371: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:3462: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:3472: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:3564: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:3574: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:3670: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:3680: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:3793: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:3804: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:3918: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:3929: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:4040: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:4051: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:4162: error: "DatasetResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:4173: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:4269: error: "DatasetResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:5418: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:5426: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:5495: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:5503: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:5572: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:5580: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:5663: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:5672: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:5756: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:5765: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:5853: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:5862: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:5965: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:5975: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:6079: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:6089: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:6190: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:6200: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:6301: error: "DataArrayResampleReductions" has no attribute "_flox_reduce"  [attr-defined]
xarray/core/_reductions.py:6311: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/_reductions.py:6399: error: "DataArrayResampleReductions" has no attribute "reduce"  [attr-defined]
xarray/core/groupby.py:584: error: Skipping analyzing "flox.xarray": module is installed, but missing library stubs or py.typed marker  [import]
xarray/core/dataset.py:113: error: Cannot assign to a type  [misc]
xarray/core/dataset.py:113: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/core/dataset.py:1761: error: Incompatible return value type (got "Union[Tuple[ArrayWriter, AbstractDataStore], bytes, Delayed, None]", expected "Union[bytes, Delayed, None]")  [return-value]
xarray/core/dataarray.py:70: error: Cannot assign to a type  [misc]
xarray/core/dataarray.py:70: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/core/_typed_ops.pyi:24: error: Cannot assign to a type  [misc]
xarray/core/_typed_ops.pyi:24: error: Incompatible types in assignment (expression has type "Type[ndarray[Any, Any]]", variable has type "Type[Array]")  [assignment]
xarray/backends/api.py:38: error: Cannot assign to a type  [misc]
xarray/backends/api.py:38: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/tests/test_testing.py:13: error: Cannot infer type of lambda  [misc]
xarray/tests/test_testing.py:13: error: Incompatible types in assignment (expression has type "Callable[[Any], Any]", variable has type "Callable[[Any, Any, Any, Any, Any, Any, Any, Any, Any], Any]")  [assignment]
Installing missing stub packages:
/usr/share/miniconda/envs/xarray-tests/bin/python -m pip install types-PyYAML types-paramiko types-python-dateutil types-pytz types-setuptools

Thanks @Illviljan I'm having trouble getting the inheritance order right and keeping mypy happy. Help is very welcome!

Ok, this kind of works. But there's a new error because I defined the Data*ResampleBase-classes with a for-loop.

Found 32 errors in 15 files (checked 140 source files)
xarray/backends/locks.py:10: error: Incompatible types in assignment (expression has type "Type[Lock]", variable has type "Type[SerializableLock]")  [assignment]
xarray/backends/locks.py:15: error: Cannot assign to a type  [misc]
xarray/backends/locks.py:15: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Lock]")  [assignment]
xarray/core/types.py:19: error: Cannot assign to a type  [misc]
xarray/core/types.py:19: error: Incompatible types in assignment (expression has type "Type[ndarray[Any, Any]]", variable has type "Type[Array]")  [assignment]
xarray/core/duck_array_ops.py:33: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
xarray/core/nanops.py:14: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
xarray/core/_reductions.py:15: error: Skipping analyzing "flox": module is installed, but missing library stubs or py.typed marker  [import]
xarray/core/_reductions.py:15: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
xarray/core/groupby.py:584: error: Skipping analyzing "flox.xarray": module is installed, but missing library stubs or py.typed marker  [import]
xarray/core/dataset.py:113: error: Cannot assign to a type  [misc]
xarray/core/dataset.py:113: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/core/dataset.py:1761: error: Incompatible return value type (got "Union[Tuple[ArrayWriter, AbstractDataStore], bytes, Delayed, None]", expected "Union[bytes, Delayed, None]")  [return-value]
xarray/core/dataarray.py:70: error: Cannot assign to a type  [misc]
xarray/core/dataarray.py:70: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/core/computation.py:1864: error: Overloaded function implementation cannot satisfy signature 2 due to inconsistencies in how they use type variables  [misc]
xarray/core/computation.py:1864: error: Overloaded function implementation cannot satisfy signature 3 due to inconsistencies in how they use type variables  [misc]
xarray/core/_typed_ops.pyi:24: error: Cannot assign to a type  [misc]
xarray/core/_typed_ops.pyi:24: error: Incompatible types in assignment (expression has type "Type[ndarray[Any, Any]]", variable has type "Type[Array]")  [assignment]
xarray/core/resample.py:14: error: Variable "xarray.core.resample.GroupByBase" is not valid as a type  [valid-type]
xarray/core/resample.py:14: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
xarray/core/resample.py:14: error: Invalid base class "GroupByBase"  [misc]
xarray/core/resample.py:189: error: No overload variant of "__setitem__" of "list" matches argument types "int", "Type[_Resample]"  [call-overload]
xarray/core/resample.py:189: note: Possible overload variants:
xarray/core/resample.py:189: note:     def __setitem__(self, SupportsIndex, None) -> None
xarray/core/resample.py:189: note:     def __setitem__(self, slice, Iterable[None]) -> None
xarray/core/resample.py:194: error: Variable "xarray.core.resample.DataArrayResampleBase" is not valid as a type  [valid-type]
xarray/core/resample.py:194: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
xarray/core/resample.py:194: error: Invalid base class "DataArrayResampleBase"  [misc]
xarray/core/resample.py:285: error: Variable "xarray.core.resample.DatasetResampleBase" is not valid as a type  [valid-type]
xarray/core/resample.py:285: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
xarray/core/resample.py:285: error: Invalid base class "DatasetResampleBase"  [misc]
xarray/backends/api.py:38: error: Cannot assign to a type  [misc]
xarray/backends/api.py:38: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/tests/test_computation.py:2019: error: No overload variant of "polyval" matches argument types "Any", "Any"  [call-overload]
xarray/tests/test_computation.py:2019: note: Possible overload variants:
xarray/tests/test_computation.py:2019: note:     def polyval(coord: DataArray, coeffs: DataArray, degree_dim: Hashable) -> DataArray
xarray/tests/test_computation.py:2019: note:     def [T_Xarray in (DataArray, Dataset)] polyval(coord: T_Xarray, coeffs: Dataset, degree_dim: Hashable) -> Dataset
xarray/tests/test_computation.py:2019: note:     def [T_Xarray in (DataArray, Dataset)] polyval(coord: Dataset, coeffs: T_Xarray, degree_dim: Hashable) -> Dataset
xarray/tests/test_testing.py:13: error: Cannot infer type of lambda  [misc]
xarray/tests/test_testing.py:13: error: Incompatible types in assignment (expression has type "Callable[[Any], Any]", variable has type "Callable[[Any, Any, Any, Any, Any, Any, Any, Any, Any], Any]")  [assignment]

Copy/pasteing the Resample-class removes the rest of the errors. Not especially elegant though, maybe there's a better way?

xarray/core/dask_array_compat.py:8: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
xarray/backends/locks.py:10: error: Cannot assign to a type  [misc]
xarray/backends/locks.py:10: error: Incompatible types in assignment (expression has type "Type[Lock]", variable has type "Type[SerializableLock]")  [assignment]
Installing missing stub packages:
xarray/backends/locks.py:15: error: Cannot assign to a type  [misc]
/usr/share/miniconda/envs/xarray-tests/bin/python -m pip install types-PyYAML types-paramiko types-python-dateutil types-pytz types-setuptools
xarray/backends/locks.py:15: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Lock]")  [assignment]

xarray/core/types.py:19: error: Cannot assign to a type  [misc]

xarray/core/types.py:19: error: Incompatible types in assignment (expression has type "Type[ndarray[Any, Any]]", variable has type "Type[Array]")  [assignment]
Found 25 errors in 14 files (checked 140 source files)
xarray/core/duck_array_ops.py:33: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
xarray/core/nanops.py:14: error: Incompatible types in assignment (expression has type "None", variable has type Module)  [assignment]
xarray/core/_reductions.py:15: error: Skipping analyzing "flox": module is installed, but missing library stubs or py.typed marker  [import]
xarray/core/_reductions.py:15: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
xarray/core/groupby.py:584: error: Skipping analyzing "flox.xarray": module is installed, but missing library stubs or py.typed marker  [import]
xarray/core/dataset.py:113: error: Cannot assign to a type  [misc]
xarray/core/dataset.py:113: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/core/dataset.py:1761: error: Incompatible return value type (got "Union[Tuple[ArrayWriter, AbstractDataStore], bytes, Delayed, None]", expected "Union[bytes, Delayed, None]")  [return-value]
xarray/core/dataarray.py:70: error: Cannot assign to a type  [misc]
xarray/core/dataarray.py:70: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/core/computation.py:1864: error: Overloaded function implementation cannot satisfy signature 2 due to inconsistencies in how they use type variables  [misc]
xarray/core/computation.py:1864: error: Overloaded function implementation cannot satisfy signature 3 due to inconsistencies in how they use type variables  [misc]
xarray/core/_typed_ops.pyi:24: error: Cannot assign to a type  [misc]
xarray/core/_typed_ops.pyi:24: error: Incompatible types in assignment (expression has type "Type[ndarray[Any, Any]]", variable has type "Type[Array]")  [assignment]
xarray/backends/api.py:38: error: Cannot assign to a type  [misc]
xarray/backends/api.py:38: error: Incompatible types in assignment (expression has type "None", variable has type "Type[Delayed]")  [assignment]
xarray/tests/test_computation.py:2019: error: No overload variant of "polyval" matches argument types "Any", "Any"  [call-overload]
xarray/tests/test_computation.py:2019: note: Possible overload variants:
xarray/tests/test_computation.py:2019: note:     def polyval(coord: DataArray, coeffs: DataArray, degree_dim: Hashable) -> DataArray
xarray/tests/test_computation.py:2019: note:     def [T_Xarray in (DataArray, Dataset)] polyval(coord: T_Xarray, coeffs: Dataset, degree_dim: Hashable) -> Dataset
xarray/tests/test_computation.py:2019: note:     def [T_Xarray in (DataArray, Dataset)] polyval(coord: Dataset, coeffs: T_Xarray, degree_dim: Hashable) -> Dataset
xarray/tests/test_testing.py:13: error: Cannot infer type of lambda  [misc]
xarray/tests/test_testing.py:13: error: Incompatible types in assignment (expression has type "Callable[[Any], Any]", variable has type "Callable[[Any, Any, Any, Any, Any, Any, Any, Any, Any], Any]")  [assignment]

Yay, mypy is passing now after flox updates. :)

Hi @dcherian  and @andersy005, thanks a lot for this PR and for flox. I'm currently trying this with our climate indices library (xclim) and it works well! I get a clear improvement in performance,
In xclim's indicators, all groupby operations are time resampling ops and the best performance is achieved with method='blockwise', by far. Is there a plan to let users control the default method through something like xr.set_options ?
Like, use_flox could accept False, True or any method name? (The default still being map-reduce) ? What do you think?

You can pass method as .mean(..., method=...). Clearly this needs docs :)
We could actually consider adding flox_kwargs to the groupby constructor since a method is really only dependent on the distribution of group labels across the chunks. Right now, I'd just like this to get merged :)
For resampling-type, we are using cohorts by default which generalizes to blockwise when applicable but is slower at graph-construction time. Note you can only blockwise if all members of a group are in a single block. So if you are resampling to yearly but a year of data occupies multiple chunks, you want "cohorts", not "blockwise".

Presumably you haven't found any bugs?
Not for now,, but I haven't done much and it wasn't very complex.

You can pass method as .mean(..., method=...)
Yes! I saw that, but I did search a bit. In fact, my idea was triggered partly because I'm lazy. My suggestion was to do something alike what xarray does with keep_attrs : a global/context option that changes the default value of method, so it doesn't need to be explicitly added everywhere.

Note you can only blockwise if all members of a group are in a single block.
Oh! I didn't catch that. My test data was already appropriately chunked.

Right now, I'd just like this to get merged :)
Haha, I understand. My idea, if pertinent, could be done in another PR.

a global/context option that changes the default value of method

Unfortunately the optimal method depends on distribution of group labels across chunks, so a  global option doesn't make sense. It would make sense to create a method="auto" and use that but it doesn't exist yet ("cohorts" is closest)

👍  this looks great to me!

Congratulations @dcherian & @Illviljan !!

and @andersy005 !


Create designs in Figma for word entry UI

Create a UI for adding words and their translations to the app

Figma


chore: use php-cs-fixer 3.49

php-cs-fixer 3.49 (or some recent minor release) wants to use the ? (nullable) syntax on optional parameters.
That syntax has been supported for quite a while since PHP https://www.php.net/manual/en/migration71.new-features.php
In the cases here, as well as the default value of the parameter being null, the ? explicitly allows the caller to pass the value null if they want.
This seems reasonable, I don't see how it can break any existing usage.

Has a minor complaint about project code coverage - strange report, the codecov/patch reports 100% of diff hit.

Bumped to use cs-fixer 3.51


Random Helper

This PR implements a random helper and provides an interface for upper-bounded randoms, lower- and upper-bounded randoms, and the option to configure decimal precision.
I filled out the README table with a bit more explanation (which, I'll admit, does widen the Markdown a bit. I'd be up for writing a set of full-text descriptions as we go on).
I also added some random examples to the dummy app demo and gave its results a slight color flourish. One thing to note, though, I'm not quite familiar with using flexi, so I broke from that a bit in the template. @rwwagner90, if you had a suggestion for "flexi way" of marking this up, I can definitely update and rebase.

Thanks for the PR @BrianSipple!
It seems like tests are failing for ember-release only, on Travis, but I'm not sure why. Perhaps the random is too random and thus randomly fails tests? 😛
As for flexi markup, I think you should just be able to replace your section and its classes with a simple <centered> tag. Give that a try and let me know if it achieves the desired results.

Travis seems to be okay now, but Circle is having some problems with npm install. Could it be that we need to clear caches on each build?


JSX types for SVG elements (SVGElementTags) wrongly use runtime DOM element types instead of attributes

Describe the bug
As part of an SVG element I was building with solid, I wanted to use an <feDropShadow> element like this:
<feDropShadow in='SourceGraphic' dx='0.2' dy='0.4' stdDeviation='0.2' />

only to find out that TypeScript was complaining about dx and dy having wrong types, expecting an object of the shape SVGAnimatedNumber.
And that's because feDropShadow is defined as Partial<SVGFEDropShadowElement> in interface SVGElementTags, in file solid-js/types/jsx.d.ts
The issue here is that SVGFEDropShadowElement, defined in typescript/lib/lib.dom.d.ts, represents the runtime instance of the created element, not the input attributes type, which should define the inputs dx/dy I mentioned as numbers/strings (for an example, here's how it's defined in React: @types/react/index.d.ts).
Your Example Website or App
https://stackblitz.com/edit/solidjs-templates-xd2mqb?file=src%2FApp.tsx
Steps to Reproduce the Bug or Issue
Just write a solid component with JSX in TypeScript and try defining dx or dy with numbers or strings on a <feDropShadow> element: the tooling (IDE, compiler, etc.) will report an error.
Expected behavior
The type definition should accept numbers/strings and not an SVGAnimatedNumber object. But more importantly, all JSX SVG element types should be defined according to their input attributes, not to their corresponding runtime instance objects.
Screenshots or Videos

Platform

solid version: 1.7.12

Additional context
No response

That was so fast ❤
By checking how this was resolved, I actually noticed two things:

I hadn't even paid attention to the fact that the other elements were properly typed, and that I had just stumbled upon one of the rare occurrences not yet completed - I feel a bit sorry for "explaining" how the typing should be done instead
I know now that the typings (and more) are developed in a separate package and repository

Sincere thanks for all the excellent work

Yeah no problem. The separate repo trips people up all the time and I was honestly surprised that slipped by, probably part of a PR I didn't check closely enough.


feat(vitest): allow overiding package installer with public API

Description
Closes #4864
Proposed API:

env VITEST_SKIP_INSTALL_CHECKS that disables all install checks - Vitest assumes all dependencies are installed
VitestPackageInstaller class with ensureInstalled method (exported via vitest/node) - other implementations can override it if needed and provide a new class to vitestOptions when starting Vitest:

await startVitest('test', [], {}, {}, {
  packageInstaller: new CustomPackageInstaller()
})

cc /@Jinjiang

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[ ] It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
[ ] Ideally, include a test that fails without this PR but passes with it.
[ ] Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

[ ] Run the tests with pnpm test:ci.

Documentation

[ ] If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

[ ] Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@sheremet-va it looks great! May I confirm what would happen if I return false in the ensureInstalled() method?

Depends on the caller. In most situations, it will exit


Start a CPU rasterizer backend for piston-graphics

Useful for off-screen rendering, testing and painting software.

There is already a library for this: https://crates.io/crates/graphics_buffer/


Missing info in doc : vpn.conf

I had to rename my xxxxx.ovpn file to vpn.conf for the vpn to work, but it is not mentioned anywhere in the README (I had to look at the vpn container logs to understand that).

This might be related to your .ovpn config file, it should be the only file in the folder packages/vpn
Could it be related to ?

https://github.com/dperson/openvpn-client/issues/209
https://github.com/dperson/openvpn-client/issues/254

Yes, my credentials for my vpn are stored in a credentials.conf file which I put in the same directory, so probably coming from here, thanks!
Anyway renaming my .ovpn file to vpn.conf resolved the issue without having to move my credentials.conf file.


Crawl queue update

Got rid of the need to truncate the Crawl Queue on every URL detect, which was blowing up custom URLs to crawl.

varchar(2083) is too big to be UNIQUE which has a max length of 3072 and can't be a primary key. So we'd need something that is shorter to match against. Could implement a hashed url which would be a static length and make a good UNIQUE candidate, but would have to add hashing to the function (not actually that hard). Let me see if I can get a proof of concept going for that.

This works as it should, not sure it's ultimately what you want here, but it does the job and you can seed the table with other entries and not loose them with the truncate.
do need to truncate, deactivate, update, activate to make this work with the new table layout if you already have this.

Thanks @sgerlach! Looks great! I'll see how I go when I get back to V7 and this area.
Just working last main piece of V6 work before that should be ready for release: https://github.com/WP2Static/static-html-output-plugin/pull/87 was going well tonight, until toothache about debilitating me now :(


Refactor core

See Issue #541
This needs to be approved by @migueldeicaza .

I'm closing this so I can re-do it in a way that tracks changes (I didn't use git mv).


Record any timings that have been captured for all completion states …

…(timed out, failed, etc) rather than only when a pipeline completes successfully

This PR was successfully deployed to production by CircleCI build 93 at 2016-12-15 10:21 GMT.


Add deliverables

Fixes https://github.com/HyphaApp/hypha/issues/2383

Codecov Report

Merging #2556 (682a2ba) into main (38f040d) will decrease coverage by 0.07%.
The diff coverage is 72.85%.

@@            Coverage Diff             @@
##             main    #2556      +/-   ##
==========================================
- Coverage   81.18%   81.10%   -0.08%
==========================================
  Files         637      640       +3
  Lines       23208    23318     +110
==========================================
+ Hits        18841    18913      +72
- Misses       4367     4405      +38

Impacted Files
Coverage Δ

hypha/apply/projects/models/__init__.py
100.00% <ø> (ø)

hypha/apply/projects/urls.py
100.00% <ø> (ø)

hypha/apply/api/v1/projects/views.py
40.00% <40.00%> (ø)

hypha/apply/api/v1/projects/serializers.py
70.00% <70.00%> (ø)

hypha/apply/api/v1/mixin.py
83.33% <71.42%> (-16.67%)
:arrow_down:

hypha/apply/projects/models/payment.py
90.58% <86.66%> (-0.50%)
:arrow_down:

hypha/apply/projects/models/project.py
94.16% <88.88%> (-0.23%)
:arrow_down:

hypha/apply/api/v1/urls.py
100.00% <100.00%> (ø)

...apply/projects/migrations/0039_add_deliverables.py
100.00% <100.00%> (ø)

hypha/apply/projects/tests/test_views.py
100.00% <100.00%> (ø)

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38f040d...682a2ba. Read the comment docs.

@frjo, have fixed the reviewed changes, thanks!


Apps Using Significant Data

This is something from my to-do list that I may as well document here. It'd be nice to have a section for Apps Using Significant Data, exactly like how the Battery menu has a section for Apps Using Significant Energy:

Actually, I might just add a sort option at some point.


Percentage string converter

Add a converter that converts a string which stands for a percentage number ending with %
to a validate double number.

Hi Chen,
This seems extremely specific. Should we talk about the correct way of doing this kind of transformations?


List of "gotcha"s

When calling swal as a response to a click (or similar) event, the popup can automatically close itself, and that can be terribly annoying if you don't know the reason. As it stands, the reason is that mobile devices (most of all), submit more than one event when the user touches the screen. The first event would trigger the button and show the popup, but the second event would then trigger the popup to close.
The solution in this case is to call preventDefault before showing the popup:
function onClick(event) {
    event.preventDefault(); // without this, there's a chance the swal popup closes itself

    swal("It works!");
}

Perhaps it would be useful for have a list of these kinds of behaviours present in this repository. Especially in this case the sweetalert2 module appears broken, while it is really the browser that is being a bit weird to preserve backward compatibility with older websites.

platform/browser?

I can't reproduce this issue (https://jsfiddle.net/ad3quksn/84/show/)
In future for bug reports, please include a jsfiddle in order for us to be able to reproduce the issue.

JSFiddle template: https://jsfiddle.net/limon/ad3quksn/

See Contributing.md

This wasn't intended as a bug report, as it's not a bug in this module. It's simply quirky browser behaviour for many mobile devices. I'm quite sure there must be more of these strange quirks, so it would be nice to start a little list somewhere.
In my case I was using Chrome on an Android 6 device, but that really doesn't matter.
@limonte


Response of "aut account verify-signature --help"

In the response of "aut account verify-signature --help" we show an extre Grave Accent " ` "   after    " Verify that the signature in SIGNATURE_FILE "
Exemple :

command:
$ aut account verify-signature --help

Response Usage:
aut account verify-signature [OPTIONS] MESSAGE SIGNATURE_FILE
Verify that the signature in SIGNATURE_FILE` is valid for the message
MESSAGE, signed by the owner of the FROM address. Signature must be
contained in a file.

Done in v0.3.0


ARTEMIS-2052 Fixing initial credit negotiation

@cshannon your last negotiation commit broke the CI. this should fix it. Can you look at it?

It looks like one of tests I added in ConsumerWindowSizeTest broker....testConsumerWindowSizeAddressSettingsDifferentAddressAndQueueName

The issue is here: https://github.com/apache/activemq-artemis/blob/106e71bdf4f1cf7ddf0dfc37ce6bab76d5c00dad/tests/integration-tests/src/test/java/org/apache/activemq/artemis/tests/integration/client/ConsumerWindowSizeTest.java#L1457
That line should now change to: assertTrue(Wait.waitFor(() -> cons.getAvailableCredits().get() == consumer.getClientWindowSize() * 2, 5000, 500));
And that will fix it

@clebertsuconic - besides the test fix it looks good to me, thanks for fixing it


[Profile] az account get-access-token: Add --client-id argument

Related command
az account get-access-token
Description
Requires https://github.com/Azure/azure-cli/pull/30300
This PR enables getting an access token for another first-party application that can do single sign-on with Azure CLI. For example, to get an access token for Azure PowerShell:
az account get-access-token --client-id 1950a258-227b-4e31-a9cf-717495945fc2

Testing Guide
# Sign in as Azure CLI
az login

# Get an access token as Azure PowerShell
az account get-access-token --client-id 1950a258-227b-4e31-a9cf-717495945fc2

Thank you for your contribution! We will review the pull request and get back to you soon.


feat: Improved folder scanning performance

PR for new semantic release.

:tada: This PR is included in version 3.7.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


CreateEffect doesn't react when object in store changes a parameter type from array to object and vice-versa

Describe the bug
When creating a store and subscribing via an effect to any change in the store, a simple change of a top-level value from array to object or vice-versa does not trigger the effect.
Your Example Website or App
none
Steps to Reproduce the Bug or Issue
import { createEffect } from 'solid-js';
import {  createStore } from 'solid-js/store';

const [get, set] = createStore({ key: [] });

createEffect(() => {
    if ({ ...get }) {
      console.log('Something changed in get');
      console.log(JSON.stringify(get));
    }
  });

setInterval(() => {
    set('key', { time: Date.now() });
  }, 1000);

This is true for any depth the property I want to change from array to object is in. The crucial part is that here:
const [get, set] = createStore({ key: [] });

The property key is an array.
On the other hand setting it to an object initially will trigger the effect though.
const [get, set] = createStore({ key: {} });

Additional proof:
Arbitrary depth also does not trigger the effect:
import { createEffect } from 'solid-js';
import {  createStore } from 'solid-js/store';

const [get, set] = createStore({ key1: { key2: { key3: { key4: [] } } } });

createEffect(() => {
    if ({ ...get }) {
      console.log('Something changed in get');
      console.log(JSON.stringify(get));
    }
  });

setInterval(() => {
    set('key1', 'key2', 'key3', 'key4', { time: Date.now() });
  }, 1000);

Expected behavior
I expect the effect to trigger every second.
Screenshots or Videos
No response
Platform

OS: Window
Version: latest

Additional context
The effect is not triggered by the simple { ...get } destructuring I do in the effect.
But imo it should trigger it, unless I did not read the docs correctly or overlooked some hidden "caveats" section with the reactivity system.
Please confirm wether I am wrong or not on this one!
Edit:
I also subscribed to the value directly without destructuring and it still does not trigger the effect unless the intital type for key is object and not array when overwriting it with an object:

import { createEffect } from 'solid-js';
import {  createStore } from 'solid-js/store';

const [get, set] = createStore({ key: [] });

createEffect(() => {
    if (get.key !== undefined) {
      console.log('Something changed in get');
      console.log(JSON.stringify(get));
    }
  });

setInterval(() => {
    set('key', { time: Date.now() });
  }, 1000);

The issue is that the array is not getting changed by the date, but rather a property "time" is getting added and then updated in the array, as top level objects gets merged with the existing ones.
If you update the store like this it will trigger the update, as the array gets actually replaced:
//     v - updated property
//     v       v - updated property
//     v       v      v - new object
set('key', { time: Date.now() })

//     v - updated property
//     v       v - new object
set({ key: { time: Date.now() } })

https://playground.solidjs.com/anonymous/14d88fac-018d-44fc-b61b-375a3e2dafef
it may still be considered a bug if you expect the object to not be merged if the exiting object is an array
Although this will work fine, so the issue is just that JSON.stringify() doesn't iterate over non-number keys on arrays, so the new property gets ignored.
const [get, set] = createStore({ key: [] })

set("key", "time", Date.now())

https://playground.solidjs.com/anonymous/28c4f1e4-1ccf-4a5a-b4f4-85f6422fffd9

@thetarnav  This is just broken imo. Your first playground example is fine. You are replacing the store with a totally new object so I expect a change/it working like expected. But the second you presented shows even more that ther eis something broken because now your store is not reactive anymore because the get or get.key cannot be used in any form anymore and be reactive. get.key is for ever now an empty array seemingly but get.key.time is still retrievung the time property from that supposedly empty array.
I think this is a huge caveat with the reactivity system

So I understand. Are you expecting it to be an object at the end?
If you expect it to be an object then this is a matter of not using the right syntax. Objects merge in setStore so it would continue to be an array. In @thetarnav's example he's not actually replacing the store with a new object. He's merging "key" field into the top-level store which replaces the array with the object. This might not seem obvious but we mention it in the docs, and its legacy goes back to setState in React class components where it merges on keys rather than replaces the object. We kept this consistent throughout because it reduces the amount of code on multifield updates without resorting to cloning objects (and less granular updates)
You can see in this example otherKey never goes away: https://playground.solidjs.com/anonymous/edcf09cd-9d4c-4d39-aff4-0a02b9306be4
What is happening in your example is more like:
const store = { key: [] }
store.key.time = Date.now();
JSON.stringify(store);

If you run this code you will see what I mean. It isn't that anything loses reactivity. But that JSON.stringify just shows the empty array because it is still an empty array that just happens to have a property on it that JSON.stringify doesn't look at.
Where as in the example I shared above(and in @thetarnav first example) it is more like:
const store = { key: [], otherKey: "hi" }
store.key = { time: Date.now() }
JSON.stringify(store);

I guess the question is, if we detect someone setting a key on an array should we assume it can't be merged and should just replace with a new object? My gut is probably no because you could want to listen to a property on said array. I think if we changed it to behave that way it would be unexpected to those who know the API merges. That being said this is a prime example for why the produce interface seems preferable as the default in future versions of Solid Stores.

I know JS is weird with how arrays are handeled compared to other languages as they are some type of objects. Well let's call this an edge case then and if you run into it, it is your problem because your design choices might be questionable. You just have to keep in mind then if you decide to store data as an array in the store you cannot get rid of it being an arrays and same applies to objects unless you overwrite with a new reference in such a way that it is not merged.
@ryansolid I think a mutable approach to stores should be the way to go in the future too if it doesn't sacrifice too much on the performance side.

You just have to keep in mind then if you decide to store data as an array in the store you cannot get rid of it being an arrays and same applies to objects unless you overwrite with a new reference in such a way that it is not merged.

It's not so much an edge case as an API that has a defined behavior that some might not expect. Everything you wish to accomplish is expressible with the correct syntax. It isn't obvious but once you know it isn't an issue and it is consistent. Only top objects merge and this makes sense in that top level it has to merge unless it diffs as stores are a reference you can't replace.

@ryansolid I think a mutable approach to stores should be the way to go in the future too if it doesn't sacrifice too much on the performance side.

It isn't so much performance as it can be dangerous to pass around mutable reactive objects. We've lived in an immutable world so long with React and VDOMs it sometimes isn't as obvious, but back before then passable mutability was a surefire way to cause all sorts of data flow messes. I see the benefit in providing a mutable interface but it will still be guarded so that the ability to mutate isn't implicit from just passing the store down.


fix(frontend): upstream option for anyof array

[x] fix: upstream option for anyof array
[x] fix: #265
[x] fix: anyOf(string, null) is rendering same options as anyOf(string, null, int, float, date, etc).

anyOf(string, null) is rendering same options as anyOf(string, null, int, float, date, etc).
We should render only the allowed types.
For example, in email sender piece the arg_value of the InnerArgModel is typed as Optional[str] and the upstream options is allowing we select all types from Log piece.

@nathan-vm
Now is good but I think we should validate fields with "format" as well for generating upstream options.
The way we have we are accepting date, datetime, time, enum in string fields.
Strings without any format are pure strings
Strings with date must connect only with strings with date, the same for time, datetime, etc.


/customers/?lookupParam=username returns all customers

curl 'http://<IP_ADDRESS>:9000/2/customers/?lookupParam=username' -H 'authorization: Token d54f9200b680ff11eb1ffcb01a99bde2' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: nb' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.82 Electron/1.3.4 Safari/537.36' -H 'content-type: application/json' -H 'Accept: /' -H 'If-None-Match: W/"1a9-kj3uaKZ9JCpzeoUonj3p9A"' -H 'Connection: keep-alive' -H 'x-abacash-application-version: 0.0.1' --compressed

Since you dont specify a parameter to lookup the express routes the request to get list route. Do you feel this is an undesired behaviour?
Would you rather get a an error telling you to specify a parameter?

larsen is throwing out systems altogether


Fix improper CommonJS import from web3-utils

When running a jest test that ultimately imports this, I get an error like:

SyntaxError: The requested module 'web3-utils' does not provide an export named 'sha3'

This seems to be because Node 20 / cjs-module-lexer 1.3.1 chokes on the module.exports = {...} object of web3-utils and ends up not including sha3 as a distinct import. The solution is to import the default export and access its .sha3 field normally. Long-term it might be better to upgrade web3-utils.
Discovered while working on https://github.com/PaimaStudios/paima-game-templates/pull/72.

I feel like I remember there was a reason we did this where not doing it like this broke something else
Our goal though is to remove everything related to the web3 library though. It's an absolute garbage library and the entire industry is moving away from it

Since it seems like this has some risk of mysteriously breaking stuff and I only ran into the problem in Jest tests or when running tsc output directly, which is a corner case, I'll close this in favor of someday replacing web3 entirely.