pre-commit: fixing import bug for built-in hooks

In a previous change, I prevented PYTHONPATH from propatating into nix devshells.  This fixed some problems where pre-commit's dependenceis were being used instead of the user's dependencies.
But pre-commit bundles three pre-commit hooks internally (as opposed to "normal" hooks, which are references to external repo's).  By removing PYTHONPATH, I prevented these internal hooks from being correctly imported.
This change fixes what I broke by adding PYTHONPATH to the wrapper which launches pre-commit so that those built-in hooks can be called directly.
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Where I broke this: https://github.com/NixOS/nixpkgs/pull/235123
A previous attempt to fix it: https://github.com/NixOS/nixpkgs/pull/241698
Fixes: https://github.com/NixOS/nixpkgs/issues/238345

Pre-commit has a "language" called system (docs) which lets you bypass pre-commit's encapsulation and just run commands in your "system".
By setting the pythonpath in the wrapper, we've fixed what we set out to fix.  But we're clobbering pre-commit's idea of what's available in "system".  In my case, I've got python311 in the calling shell, but my system hooks are using python310 because that's the version that pre-commit is packaged with.
Perhaps the wrapper needs to either:

merge the calling environment with the packaged values instead of clobbering the former with the latter
make note of what's in the env before clobbering it so that system hooks can un-clobber

I'll explore this...

Ok, I fixed it by patching pre-commit to only expose the calling environment's PATH and PYTHONPATH to the hook runner if they're written in the system language.  This broke the internal hooks (which are also system), so I created a new language (meta) for these.

meta -> sees PATH and PYTHONPATH as prepared by nix
system -> sees the calling environment's PATH and PYTHONPATH

meta hooks are hard-coded into pre-commit.  So far as I know, nobody is extending them.  So this distinction need not be thought about by users (:crossed_fingers:).

@austinbutler last time I touched this, I merged the fix too early.  Then you created https://github.com/NixOS/nixpkgs/issues/238345 which is how I learned that I broke it (thanks for catching me).
Would it be convenient for you to try out the fixed version (in my fork) and give me a 👍 so that I can merge with confidence?
Otherwise I'll play around with it a bit more in the coming days and merge once I feel I've banged on it sufficiently.

pandas has a nontrivial .pre-commit-config.yaml.  I've been testing this by adding the following flake to that repo and running its pre-commit hooks.
{
  inputs = {
    flake-utils.url = "github:numtide/flake-utils";

    # switch the comment below to activate the other codebase
    nixpkgs.url = "github:MatrixManAtYrService/nixpkgs?ref=fix-pre-commit";
    # nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
        };
      in
      {

        devShells.default = pkgs.mkShell {
          packages = [
            pkgs.pre-commit
            pkgs.nodejs
          ];
        };
      });
}

Ruff is still a problem, but it's not a problem for this PR. As for everything else, this PR fixes a wide swath of hooks, all of which previously failed with errors like:
Unwanted patterns.......................................................................................Failed
- hook id: unwanted-patterns
- exit code: 1

/nix/store/pkj7cgmz66assy7l18zc7j992npb41nx-python3-3.10.12/bin/python3.10: Error while finding module specification for 'pre_commit.languages.pygrep' (ModuleNotFoundError: No module named 'pre_commit')

Also, it introduces no new failures.
It seems pretty clear that it's much better than it was before.  Let's merge it.

Thanks to @anund for pointing me at the wrapper script as a place to address this kind of problem. I've had to go a bit further than you recommended but thanks to that help I think I'm on the right path here.  If you have any thoughts about this approach, they'd be valued.

@MatrixManAtYrService I think you may be able to just define the python version your dev shell uses rather than patching pre-commit. This comes from thinking about what the 'system' is for your example. if pre-commit is using the wrong python version and you're setting that version in nix develop the question that came to mind was "how do you tell pre-commit how to use a different python?". First I look at override, and overrideAttrs which cover swapping things around at a slight cost on not getting upstream caching for free anymore.
To reproduce I had to adjust somethings in the example flake.nix you provided. I need a version of nixpkgs with the bug and to use overridePythonAttrs on pre-commit to add in just the makeWrapperArgs change.
#flake.nix
{
  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:nixos/nixpkgs?rev=a4832e490d62bdb1e60cab751a9ae38b4431934e";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
      in rec
      {
        devShells.default = pkgs.mkShell {
          packages = [
            pkgs.python311
            (pkgs.pre-commit.overrideAttrs (previousAttrs: {
              makeWrapperArgs = ''--set PYTHONPATH $PYTHONPATH'';
            }))
          ];
        };
      });
}

$nix develop and pre-commit install && pre-commit run --all --verbose
This gets us to the mismatch behaviour if we create .pre-commit-config.yaml the way you helpfully define above. Note: there is a overridePythonAttrs to interact with the buildPythonPackage call you may need to use in some other cases.
Updating the python version is ~easy since pre-commit takes python3Packages as an input. (python3Packages has the reference to python buildPythonPackage depends on)
#flake.nix
{
  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:nixos/nixpkgs?rev=a4832e490d62bdb1e60cab751a9ae38b4431934e";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };
      in rec
      {
        devShells.default = pkgs.mkShell {
          packages = [
            pkgs.python311
            # example getting the parens right, you should probably define pre-commit above in the let
            ((pkgs.pre-commit.override{ python3Packages = pkgs.python311Packages; }).overrideAttrs (previousAttrs: {
              makeWrapperArgs = ''--set PYTHONPATH $PYTHONPATH'';
            }))
          ];
        };
      });
}

Note you'll probably use python311Packages.<package-name> to keep any nix supplied python deps in line with your python3 version. (propogatedBuildInputs would probably bring in a valid python version for other python libs the way you had to work around for pre-commit)

Hmm, looking a bit closer at the suggested fix the way the python path is built is a bit wrong. It captures the pythonPath generated during build that includes more than just a reference to python. A path like the following might be closer to what is needed.
--set PYTHONPATH ${pkgs.python311Packages.python}/${pkgs.python311Packages.python.sitePackages}`

I finally got a chance to meditate on @anund's comments.  For the particular repo I'm most worried about (work stuff), this seems pretty close:
        devShells.default = pkgs.mkShell {
          packages = [
            pkgs.poetry
            pkgs.python311
            # pkgs.pre-commit
            (pkgs.pre-commit.overrideAttrs (previousAttrs: {
              makeWrapperArgs = ''
                --set PYTHONPATH $PYTHONPATH
                --suffix PYTHONPATH : ${pkgs.python311Packages.ruamel-yaml}/lib/python3.11/site-packages
                --suffix PYTHONPATH : .
              '';
            }))

The above overrides won't work for everyone.  Your hooks may need different paths, but the --set line ought to get most people pretty far.
I'm thinking about just closing this PR and writing a blog post about how to make pre-commit work inside of nix.  I'll think about that.

@anund's approach almost solves the problem. The last hurdle for me was that the propagated build inputs needed to be injected to the environment:
pre-commit.overrideAttrs (prev:
    let depsPath = python3Packages.makePythonPath prev.propagatedBuildInputs;
    in {
      makeWrapperArgs = [
        "--prefix PYTHONPATH : ${depsPath}"
        "--prefix PYTHONPATH : $out/${python3Packages.python.sitePackages}"
      ];
    }
  )

I'm not sure whether a difference between python3Packages.python and the python version used by the pre-commit derivation might occur and pose a problem, though.

Thanks for the fix! I've written a test for a different issue which might've been caused by the same underlying problem. Feel free to cherry-pick it.

Result of nixpkgs-review pr 251418 run on x86_64-linux 1

4 packages failed to build:

python311Packages.asteroid-filterbanks
python311Packages.asteroid-filterbanks.dist
python311Packages.pyannote-audio
python311Packages.pyannote-audio.dist

2 packages built:

pre-commit
pre-commit.dist

I didn't know about that tool, that's pretty cool. I'll investigate how those packages are using pre-commit so I can avoid breaking them.

Just ran into this while working on a flake. I used the override from https://github.com/NixOS/nixpkgs/pull/251418#issuecomment-1780442006 for now and it works, but think it would be nice when this gets in rather than having to manually update every flake that uses pre-commit.
I've never used nixpkgs-review before, but just gave it a run on x86_64-linux as well and didn't get any of the build errors mentioned above (maybe I did something wrong):
nix-shell -p nixpkgs-review.out --run 'nixpkgs-review pr 251418'
this path will be fetched (0.06 MiB download, 0.32 MiB unpacked):
  /nix/store/rx6laqr63fhb15sysy8z4magbijb8g0i-nixpkgs-review-2.10.4
copying path '/nix/store/rx6laqr63fhb15sysy8z4magbijb8g0i-nixpkgs-review-2.10.4' from 'https://cache.nixos.org'...
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/251418/head:refs/nixpkgs-review/1
remote: Enumerating objects: 183036, done.
remote: Counting objects: 100% (29281/29281), done.
remote: Compressing objects: 100% (1251/1251), done.
remote: Total 183036 (delta 28476), reused 28580 (delta 27952), pack-reused 153755
Receiving objects: 100% (183036/183036), 109.76 MiB | 6.52 MiB/s, done.
Resolving deltas: 100% (108115/108115), completed with 7618 local objects.
From ssh://github.com/NixOS/nixpkgs
 * [new branch]                master                -> refs/nixpkgs-review/0
 * [new ref]                   refs/pull/251418/head -> refs/nixpkgs-review/1
$ git worktree add /home/aa/.cache/nixpkgs-review/pr-251418/nixpkgs 971ea0a28ad94b23d221c8377120340a303fc736
Preparing worktree (detached HEAD 971ea0a28ad9)
Updating files: 100% (41726/41726), done.
HEAD is now at 971ea0a28ad9 Merge pull request #320833 from matteo-pacini/easytag-resurrection
$ git merge --no-commit --no-ff ed76e4ddcfde2f8b2891455e7e61b71cfc6da502
Auto-merging pkgs/tools/misc/pre-commit/default.nix
Automatic merge went well; stopped before committing as requested
$ nix build --file /nix/store/rx6laqr63fhb15sysy8z4magbijb8g0i-nixpkgs-review-2.10.4/lib/python3.11/site-packages/nixpkgs_review/nix/review-shell.nix --nix-path nixpkgs=/home/aa/.cache/nixpkgs-review/pr-251418/nixpkgs nixpkg
s-overlays=/run/user/1000/tmpbsohbw02 --extra-experimental-features nix-command no-url-literals --no-link --keep-going --no-allow-import-from-derivation --option build-use-sandbox relaxed --argstr system x86_64-linux --argst
r nixpkgs-path /home/aa/.cache/nixpkgs-review/pr-251418/nixpkgs --argstr nixpkgs-config-path /run/user/1000/tmpufvqw8fw.nix --argstr attrs-path /home/aa/.cache/nixpkgs-review/pr-251418/attrs.nix

Link to currently reviewing PR:
https://github.com/NixOS/nixpkgs/pull/251418

6 packages built:
pre-commit pre-commit.dist python311Packages.asteroid-filterbanks python311Packages.asteroid-filterbanks.dist python311Packages.pyannote-audio python311Packages.pyannote-audio.dist

$ /nix/store/j7rp0y3ii1w3dlbflbxlv4g7hbaaz3bs-nix-2.18.2/bin/nix-shell --argstr system x86_64-linux --argstr nixpkgs-path /home/aa/.cache/nixpkgs-review/pr-251418/nixpkgs --argstr nixpkgs-config-path /run/user/1000/tmpufvqw8
fw.nix --argstr attrs-path /home/aa/.cache/nixpkgs-review/pr-251418/attrs.nix --nix-path nixpkgs=/home/aa/.cache/nixpkgs-review/pr-251418/nixpkgs nixpkgs-overlays=/run/user/1000/tmpbsohbw02 /nix/store/rx6laqr63fhb15sysy8z4ma
gbijb8g0i-nixpkgs-review-2.10.4/lib/python3.11/site-packages/nixpkgs_review/nix/review-shell.nix

@imincik, I wonder if something has changed, because like @fidgetingbits, I also find that nixpkgs-review pr 251418 succeeds.
I spent a bit of time with asteroid-filterbanks and had a hard time getting the pre-commit hooks to succeed with or without this change (see :point_up: for details about the failure).
I also tested this change with pyannote-audio.  It fails with the same error in all three cases:

pre-commit installed via pip
pre-commit installed via nix (without this change)
pre-commit installed via nix (with this change)

pyannote/audio/core/io.py:37:1: F401 'numpy as np' imported but unused

I'm not sure if these are cases where pre-commit is failing to be as deterministic as it aims to, or whether they're cases where maintainers have pushed commits without first running their pre-commit hooks, but as far as I can see, there's no indication that this change makes things any worse in those repos.

I'm getting errors from nixpkgs-review pr 251418 after rebasing onto master.

Rebased & fixed for pre-commit v4


VNet create/update support for IPAM pool association (2024-05-01)

Preconditions

[x] No need to upgrade Python SDK or the Python SDK is ready.

Related command
az network vnet create
az network vnet update
Resource Provider
Microsoft.Network
Description of Feature or Work Requested
Feature request for the vnet create/update commands to introduce a new parameter: --ipam-pool-prefix-allocations. This enhancement aims to facilitate the association/disassociation of a VNET with an IP Address Management (IPAM) pool, thereby streamlining the allocation of address prefixes.
This feature support is crucial for IPAM service going GA, which was targeted for mid-November.

https://learn.microsoft.com/en-us/cli/azure/network/vnet?view=azure-cli-latest#az-network-vnet-create
https://learn.microsoft.com/en-us/cli/azure/network/vnet?view=azure-cli-latest#az-network-vnet-update

New Parameter details:

Optional Parameter
Enables customers to specify an IPAM pool for allocating address prefixes to resources within the VNET, instead of defining an address prefix during VNET creation/update.
If this parameter is utilized, the addressPrefixes array must explicitly be empty. Otherwise, a default address prefix will be used, causing an error.

Good example:

Bad example:

Minimum API Version Required
2024-05-01
Swagger PR link / SDK link
swagger:
https://github.com/Azure/azure-rest-api-specs/pull/30560/files#diff-3865305b0c4c82aae9addc443cc6c5001ea7a833c4332d15f6f7dfe646e300b0
example PR of changes needed in CLI repo (may not be everything was just testing locally):
https://github.com/Azure/azure-cli/pull/30152
Request Example
https://github.com/Azure/azure-rest-api-specs/blob/kamboj-prjwl/release-network-Microsoft.Network-2024-05-01/specification/network/resource-manager/Microsoft.Network/stable/2024-05-01/examples/VirtualNetworkCreateWithIpamPool.json
Target Date
2024-11-15
PM Contact
LIJAY
Engineer Contact
DANLOPE
Additional context
No response

Thank you for opening this issue, we will look into it.


NodeJS without $persist_dir

How about this? It's an alternative NodeJS manifest using persistence but without requiring the $persist_dir variable in env_add_path or post_install.
This way, we can keep the complexities of persisted directories hidden from manifest authors and users (reducing the learning curve).
This change would also make it unnecessary for the workaround to check for $ variables in prepare_env_path.
From a user's point of view, they would just need to know that the current version of NodeJS is installed at ~/scoop/apps/nodejs/current, any path or other environment variables will reference this too, and the bin directory inside that current directory will be persisted across updates.
Perhaps this would also help us avoid having to add complicated source and target definitions in persist to move files, as you suggested might be necessary in future. I would much prefer to keep things simple, if possible.
Ref: lukesampson/scoop#1372.

This breaks npm updating itself!
npm can be run because the nodejs directory is in the path too. But because the prefix is changed npm does not find itself as a module and update it. Currently npm version 4.1.2 is bundled with nodejs but the latest version is 4.5.
We would need to move the npm stuff inside the bin directory to make that work. Therefore my move proposal.
Adding a post_install script like 7bfeb2aef8920016efee7f22493f3444afdb51b4 would fix it … but, eeeeeh :-)

How are you updating NPM? Without 7bfeb2aef8920016efee7f22493f3444afdb51b4, I am able to run npm install -g npm and it works (although it only installs version 4.4). The version in ~/scoop/apps/nodejs/current/bin takes precedence over the version in ~/scoop/apps/nodejs/current, due to the order Scoop adds directories to the path.
Installing NodeJS from their official MSI installer seems to have the same behavior—i.e. running npm update -g npm does nothing, but npm install -g npm updates to version 4.4.

Interesting that the official installer has the same behaviour.
What should we do?

Inherit the behaviour of the official installer and keep an old npm version
Keep the scoop (stable) behaviour with 7bfeb2aef8920016efee7f22493f3444afdb51b4 and have the latest (pre-release) version
Instead of 7bfeb2aef8920016efee7f22493f3444afdb51b4 run npm install -g npm inside post_install to receive the latest stable npm version and be able to keep it updated with npm update -g

Personally I would prefer to have the latest (stable) version. But that could be achieved by running npm install -g npm manually once.
Version 4.4.4 is the current stable version, 4.5 is still pre-release. I am not sure why we get it after an update.
Couldn't it be easy once? It always has to be complicated. 😆

Haha! Yes NodeJS is probably not the ideal test-case because it's so complicated, but I'm excited about this persistence feature overall. It looks like it will work, and will be a big improvement for Scoop, both in terms of "just working" for users, and making it easy for manifest authors to persist data.
I prefer option one, because:

it keeps the manifest simple and easy to understand
it's a quicker default install with nothing else to download
it matches the official install (the perceived behavior and limitations), and is probably perfect for people who just want it to work
people who do know about and want a later version of npm can opt-in by manually running the command as you said. Maybe they could even have upgraded npm before updating NodeJS.
future official releases might come with a stable NPM (so other workarounds might be unnecessary)

Ok, I am ok with that.
I reverted the post_install script to match that behaviour, I also removed the npm update -g command.
It was introduced by 1896adb587acb13bc9840ae18bf0cbdc31208641 but does not have an effect on the installed npm version anymore.


chore: implement polling toolchain task events instead of pushing

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#61
#60  👈
#59
#58
#57
#56
#55
#54
#53
#52
#51
#50
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @NathanFlurry and the rest of your teammates on  Graphite


Microsoft provides an identity name and family name that the program won't accept

Microsoft provides an identity name and family name starting with numbers. When attempting to build with supplied packageIdentity and packageName, i get the following error:
MakeAppx : error: Error info: error C00CE169: App manifest validation error: The app manifest must be valid as per schema: Line 26, Column 18, Reason: '238....' violates pattern constraint of '([A-Za-z][A-Za-z0-9])(.[A-Za-z][A-Za-z0-9])*'.
The attribute 'Id' with value '238...' failed to parse.
Microsoft doesn't allow me to upload it to their store without this being fulfilled.

Looks like only identityName needs to be filled out and not the family ID. You should make this more clear in the documentation.

Duplicate of https://github.com/felixrieseberg/electron-windows-store/issues/82

Should be fixed in https://github.com/felixrieseberg/electron-windows-store/pull/131; in the mean time the correct (but confusing) way to use this package is like so:
const convertToWindowsStore = require('electron-windows-store');
convertToWindowsStore({
    identityName: '12345MyCompany.Ghost',  // This is actually the package name!
    packageName: 'Ghost',  // This is actually the application id!!
    // ...
});

I've since come to the conclusion that the right way to use electron-windows-store is to write your own AppxManifest.xml, and pass this in, like so
const convertToWindowsStore = require('electron-windows-store');
convertToWindowsStore({
    manifest: '.\\AppXManifest.xml',  // For many reasons, we write should own
    packageName: 'Ghost',  // Still needed for filename 'Ghost.appx'
    // ...
});


Fix deleting full audio file in chaptered mode

Quotes are required for rm to properly parse the params. Otherwise spaces separate the file path into several params and the script dies.

Thanks!


Add key range check for _all_docs

Overview

Using key and start/end_key in a different order will produce different responses when querying _all_docs.
To reduce confusion, add a key range check for _all_docs.

If direction=fwd, start_key > end_key throws an error
If direction=rev, start_key < end_key throws an error
Otherwise, return relevant responses

Previous behavior:
$ curl -XPUT $DB/db
$ curl -XPOST $DB/db/_bulk_docs -H 'Content-Type: application/json' -d '{"docs": [{"_id": "a"},{"_id": "b"},{"_id": "c"}]}'

$ curl $DB/db/_all_docs'?key="c"&endkey="a"'
{"total_rows":3,"offset":2,"rows":[ ]}

$ curl $DB/db/_all_docs'?key="c"&endkey="a"&descending=true'
{"total_rows":3,"offset":0,"rows":[
  {"id":"c","key":"c","value":{"rev":"1-967a00dff5e02add41819138abb3284d"}},
  {"id":"b","key":"b","value":{"rev":"1-967a00dff5e02add41819138abb3284d"}},
  {"id":"a","key":"a","value":{"rev":"1-967a00dff5e02add41819138abb3284d"}}
]}

After the change:
$ curl $DB/db/_all_docs'?key="c"&endkey="a"'
{"error":"query_parse_error","reason":"No rows can match your key range, reverse your start_key and end_key or set descending=true"}

$ curl $DB/db/_all_docs'?key="c"&endkey="a"&descending=true'
{"total_rows":3,"offset":0,"rows":[
  {"id":"c","key":"c","value":{"rev":"1-967a00dff5e02add41819138abb3284d"}},
  {"id":"b","key":"b","value":{"rev":"1-967a00dff5e02add41819138abb3284d"}},
  {"id":"a","key":"a","value":{"rev":"1-967a00dff5e02add41819138abb3284d"}}
]}

Testing recommendations
make eunit apps=chttpd,couch_mrview

Related Issues or Pull Requests
https://github.com/apache/couchdb/issues/3977

Checklist

[ ] Code is written and works correctly
[x] Changes are covered by tests
[ ] Any new configurable parameters are documented in rel/overlay/etc/default.ini
[ ] Documentation changes were made in the src/docs folder
[ ] Documentation changes were backported (separated PR) to affected branches

I like the idea of a flag but wouldn't we need a roadmap for the full set of changes first? (in a python 2/3 way)

the flag would be a stopgap until we have proper versioning, it’d be a toggle between version 0 and 1, and later we allow versions 2, 3, 4 etc. once we know what these look like. — I wouldn’t mind parking this PR until we have a proper system set up and skip the binary flag for now, but if we would want the PR to land quickly, that’d be the fasted way there, but yeah, we probably want a complete scrub before we introduce even the toggle.


Missing asin, acos, atan and one form of pow

Hi @sibaku!
I've been this library in a personal project. Thanks for building it! I've ended up adding stuff to it, like...

HessNum3 a_asin(in HessNum3 a) which is like asin(float a)
HessNum3 a_acos(in HessNum3 a) which is like acos(float a)
HessNum3 a_atan(in HessNum3 a) which is like atan(float a)
apow(in HessNum3 a, in HessNum3 b)

I'd be happy to open a PR that adds these. But I figured I'd ask first to see if you're open to it!
I've implemented these only for HessNum3. But I would implement them on all data types for consistency.
Let me know what you think!

Hi,
glad someone else has a use for it!
That would be great. A more complete coverage of the basics is always good. So feel free to add those functions!


Added docs upload to S3

This will not work until we cut a new release, but it should successfully place a docs.tar.gz file next to the released code.

Is there any pr on docs-generator side to go along with this one?

No, not currently; there should not be one until docs get published once.


version '150' is not supported in macOS

in macos:
ERROR: 0:1: '' :  version '150' is not supported
ERROR: 0:1: '' : syntax error: #version

What example did you tried? Try gles3 and following glfw initialigation:
    glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3);
    glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 3);
    glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE);
    glfwWindowHint(GLFW_OPENGL_FORWARD_COMPAT, GL_TRUE);

thanks man! it's work!


DS_Store fix in find_classes

small "bug" or annoyance in the find_classes function - it picks up '.DS_Store' as a class. simple fix to check if class is actually a directory .. you can see this is done in the keras sampling code https://github.com/fchollet/keras/blob/master/keras/preprocessing/image.py#L778
Example Usage (pretend you have mnist images saved in a main directory w/ sub-directories for each class:
from torchvision.datasets import ImageFolder
folder = ImageFolder(root='/users/ncullen/desktop/data/mnist/train')
print(folder.classes) # previously: ['.DS_Store', '0', '1', ...]

thanks!


Switch to a more reliable data provider

something like https://exchangerate.host/ or other services from https://github.com/public-apis/public-apis#currency-exchange

Hey. Can you please help? Whenever I try to convert one currency to another, I get this error: 'Failed to get the exchange rate.'


allow a specific raster file to be specified if two downloaded

This is something that should be addressed by software using geoapis and not within geoapis

See above


Testing content

#53

[x] Test basic button injection
[x] Create fully-functioning stubs

[x] chrome.extension.storage.sync
[x] MutationObserver

[x] Attempt to test with mocked window.location
[x] Test option loading behavior
[x] Consider adding more granular unit tests per class/file/method/function
[x] Consider using dom-testing-library for better assertions and consistency with #87
[ ] Complete test coverage

Can we fix the merge conflicts and merge this? If everything is set up, I don't think we need to wait until we have complete coverage before merging.

Friendly ping.

I'm going to see if I can swap in sinon-chrome similarly to #87 before I merge this

Are you still working on this? If not, I think it would be best to close the PR for now so others can feel free to work on it (would love to land this PR if you're still working on it, though!).


[iOS] Invalid JSON error

Overview (Required)

Invalid JSON at 180746: Unexpected end in string was shown on starting

Steps To Reproduce

launch app

Expected behavior
show no error
Screenshots

Env:

Device: iPhone SE
OS: iOS12.1.2
App Version or the branch : first released version in App store

Additional context
Any other notes

It looks like fixed with this pr finally
https://github.com/Kotlin/kotlinx-io/pull/36


L515 Model's Depth Sensor issue (invalid value in rs2_open_multiple)

librealsense
2.44.0 RELEASE

OS
Windows

Name
Intel RealSense L515

Serial Number
f0350937

Firmware Version
<IP_ADDRESS>

Camera Locked
YES

Usb Type Descriptor
3.2

Product Line
L500

Asic Serial Number
0003b52476ff

Firmware Update Id
0003b52476ff

When I start streaming the depth sensor I get this error.
Invalid Value in rs2_open_multiple(sensor:000002843434C550,
profiles:00000284333B48D8, count:2):

@Hwoongg
Does this happen all the time?  Are you using any other USB3 connector other than the cable that came with the camera?  Can you attempt to change orientation of the USBC connection point and flip it and try?  Are you able to get RGB stream as well as IMU stream?  If this cannot be resolved via FW update, then it might be a hardware issue where you need to RMA.
You can also submit a ticket here:  https://support.intelrealsense.com/hc/en-us/requests/new

@Hwoongg Do you need further help? If we don't hear from you in 7 days, this issue will be closed.

I tried changing the USBC connection direction but got the same result. RGB data can be obtained, but depth shooting is not activated. I bought another one of the same model and connected it to it, and it works fine. It doesn't seem to be a problem with the PC or the cable. It looks like the hardware needs repair.
ㅇ
ㅇ
-----Original Message-----
From: @.>
To: @.>;
Cc: @.>; @.>;
Sent: 2021-06-16 (수) 01:38:37 (GMT+09:00)
Subject: Re: [IntelRealSense/librealsense] L515 Model's Depth Sensor issue (invalid value in rs2_open_multiple) (#9042)
@Hwoongg
Does this happen all the time? Are you using any other USB3 connector other than the cable that came with the camera? Can you attempt to change orientation of the USBC connection point and flip it and try? Are you able to get RGB stream as well as IMU stream? If this cannot be resolved via FW update, then it might be a hardware issue where you need to RMA.
You can also submit a ticket here: https://support.intelrealsense.com/hc/en-us/requests/new
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

@Hwoongg Please do the RMA.


[SMALLFIX] Correct typo in ExceptionMessage for Unexpected (Unexpecetd) Journal Entry

Minor, found while doing [TACHYON-1186].

Can one of the admins verify this patch?

Jenkins, test this please.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/6544/
Test PASSed.

Thanks!


Documentation problem

The documentation website (https://kotlin.github.io/dataframe/overview.html) still shows version 0.8 and I think it should be updated.
I tried to do this myself (seems related to https://github.com/Kotlin/dataframe/blob/master/docs/StardustDocs/project.ihp) but I couldn't figure out how to generate the documentation. Running ./gradlew korro gives an error:
Could not apply requested plugin [id: 'io.github.devcrocod.korro', version: '0.1.1-dev-29'] as it does not provide a plugin with id 'io.github.devcrocod.korro'. This is caused by an incorrect plugin implementation. Please contact the plugin author(s).
> Plugin with id 'io.github.devcrocod.korro' not found.

Same here. I tried googling stardust and stardustdocs with no result.
And I had to comment out korro in my local build because gradle could not find it.
Always wanted to ask about it on slack or so some day.

Issue with korro is weirdly related to Gradle JVM. Build doesn't work with JDK 17 very well, so lowering Gradle JVM to 11 will make it work.
I think we need to just update gradle wrapper to newer versions to resolve it. Can you please try to update Gradle in gradle-wrapper.properties to newer versions and see if it resolves the issue for JDK 17?
As for documentation versions, i would have to ask my team :D I don't know how to update it. Now docs are built from md files in StartdustDocs/topics via our CI/CD. Korro is an utility to embed code snipets from tests into *.md files

Ok, apparently updating version in that ihp file is how it should be done https://github.com/Kotlin/dataframe/commit/4ff5d54f1bf30ea7011df22720710ec41b7aee81. I'll have to deploy docs from CI after that

Updating gradle wrapper version does not help, and downgrading to JDK11 fixes the korro problem.
And I have created a PR to update the version number.

@koperagen gentle reminder

to deploy docs from CI

This issue is closed, but the documentation website still shows version 0.8
Related #227


Flat lib

Merge fragment_mz_df and fragment_intensity_df into fragment_df in FlatSpecLib by using flatten_fragments(). The new fragment_df looks like:

mz
intensity
type
loss_type
charge
number
position

98.060040
0.4
98
0
1
1
0

49.533658
0.7
98
0
2
1
0

769.265157
0.6
121
0
1
6
0

385.136217
0.8
121
0
2
6
0

671.288262
0.1
121
98
1
6
0

...
...
...
...
...
...
...

152.584411
0.0
121
0
2
2
11

1572.620589
0.6
98
0
1
13
12

786.813933
0.9
98
0
2
13
12

175.118952
0.3
121
0
1
1
12

88.063114
0.3
121
0
2
1

where each column refers to:

mz:        float, fragment mz value
intensity: float32, fragment intensity value
type:      np.int8, ASCII code of the ion type (97=a, 98=b, 99=c, 120=x, 121=y, 122=z), or more ion types in the future. See https://en.wikipedia.org/wiki/ASCII for more ASCII information
number:    uint32, fragment series number
position:  uint32, fragment position in sequence (from left to right, starts with 0)
charge:    int8, fragment charge
loss_type: int16, fragment loss type, 0=noloss, 17=NH3, 18=H2O, 98=HPO4 (phos), ...

The frag_start_idx and frag_end_idx in precursor_df will be re-indexed as well.

At the moment the module structure looks like this:
spectral_library
   decoy_library
      DecoyLib
   flat_library
      FlatSpecLib
   library_base
      SpecLibBase

Furthermore there's
protein
   fasta
       FastaLib

I think it would be best to align the naming between the different modules and classes and remove redundancy. Personally, If I see the same naming scheme, I assume they are related or belong into the same category. If I see the naming scheme varies I expect there must be a reason.
spectral_library
   decoy
      SpecLibDecoy
   flat
      SpecLibFlat
   base
      SpecLibBase

We can also go for BaseLib, FlatLib, DecoyLib but this would require refactoring the SpecLibBase Use across all repositories.

Other than this I think we should be good for merging! I will start incorporating the new FlatSpecLib class with alphaDIA

At the moment the module structure looks like this:
spectral_library
   decoy_library
      DecoyLib
   flat_library
      FlatSpecLib
   library_base
      SpecLibBase

Furthermore there's
protein
   fasta
       FastaLib

I think it would be best to align the naming between the different modules and classes and remove redundancy. Personally, If I see the same naming scheme, I assume they are related or belong into the same category. If I see the naming scheme varies I expect there must be a reason.
spectral_library
   decoy
      SpecLibDecoy
   flat
      SpecLibFlat
   base
      SpecLibBase

We can also go for BaseLib, FlatLib, DecoyLib but this would require refactoring the SpecLibBase Use across all repositories.

It makes sense to rename them now, before more repos use it

No, let's do it while we switch to Sphinx, as we are going to add more docs


hasItem potential bug

I am writing integration tests with Arquillian. I have junit 4.12 on my project. hamcrest-core 1.3 is brought in through junit and does not appear anywhere else in my pom.
I am attempting to find an object created with Java in a list that is the result of a database call from an EJB. The test passes when I use:
Assert.assertTrue("indicator2 should have been found", indicators.indexOf(indicator2) >= 0);
but fails when I try:
assertThat(indicators, hasItem(indicator2));
I can see in the debugger that the object is in fact in the list, but when I put a breakpoint in equals of my object, many of the entries do not enter the breakpoint which is the cause of the failure.

        Indicator indicator2 = new Indicator();
        indicator2.setId(2);
        indicator2.setIndicatortype(indicatorType3);
        indicator2.setOrganization(org);
        indicator2.setSummary("<IP_ADDRESS>");
        indicator2.setDescription("This should be an address");
        indicator2.setRating(new BigDecimal("4.00"));

        List<Indicator> indicators = indService.getIndicatorsByOrganization(org, 0, 1000, getAdminUser());

        // This passes
        Assert.assertTrue("indicator2 should have been found", indicators.indexOf(indicator2) >= 0);

        // This does not
        assertThat(indicators, hasItem(indicator2));

Actually, the bug could be in Assert.assertThat

It's possible that the problem is that the matcher calls actual.equals(expected), which is the opposite of indexOf(). This is probably wrong but quite a dangerous change. Not sure what to do.


Make the type of AbstractNode.parent concrete to avoid runtime type casts

Let AbstractNode take the concrete node type as a type argument such that parent, adoptChild, and dropChild could take concrete node types without runtime type casts.

See also https://github.com/flutter/flutter/pull/103832

Thanks for the reminder! Yes, AbstractNode isn't giving us much, or anything, really. Instead of making it more complicated with generics we should figure out how to remove it completely as was attempted in https://github.com/flutter/flutter/pull/103832.

Closing. @goderbauer is working on removing AbstractNode altogether.


Distance calculating app.

Description
distance calculating Django app built with the help of

Geopy
folium
crispy forms

Fixes #351
Have you read the Contributing Guidelines on Pull Requests?

[x] Yes
[ ] No

Type of change

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have created a helpful and easy to understand README.md
[x] My documentation follows Template for README.md
[x] My changes generate no new warnings
[x] I have added tests/screenshots(if any) that prove my fix is effective or that my feature works.

@avinashkranjan please review my PR.


Remove space reserved for scrollbar in database cell markdown

Without the change:

Notice the green 'padding' space meaning that markdown text is not properly centered.
With the change:

Notice that markdown text is properly centered, including the 'Select' labels' text.

thanks @Lordfirespeed for your contribution.
Since Obsidian add to its roadmap databases I am not maintaining the project as the last year. I am working into another plugin but we can create a minor version with some maintaining changes like this one


Set up dev configuration for remote execution/MQTT-related development

Fixes #951.
View commit history for more details.

coverage: 95.309% (-0.04%) from 95.345% when pulling 2a96298832e59a3ec4bd223d93d383238dae32db on RichDom2185:23-summer/set-up-mqtt-local-env into 470d220e2937d0b73a95a59505e5a8d2b2fb61e4 on source-academy:master.

LGTM. Will be great if the -0.04% is addressed. Otherwise, feel free to merge too.

Seems like the coverage decrease is from a file I didn't touch for some reason (https://coveralls.io/builds/60861743/source?filename=lib%2Fcadet%2Faccounts%2Fnotifications.ex#L39). Either way, the notifications will be updated soon, so I'll just merge it.
Thanks!


beagle: implement header tables for beaglebone/beagleboard board family

It will be useful coupled with am335x support (issue #199) and PRU-ICSS support (issue #200).
https://beagleboard.org/

For the raspberry pi you set up the pin layout with bcm283x.GPIO.
For the Beaglebone (Black) the analogue would be the am335x?

Yes, there's scaffolding at http://periph.io/x/periph/experimental/host/am335x and http://periph.io/x/periph/experimental/host/beagle.
In practice a full am33x implementation is not initially needed, only detecting the presence. But even so, I think there's beaglebones on different processors (?) so detecting the board is what is important here, and registering the right header.

I only used the Beaglebone Black so far. Detection is done rudimentary by checking sysfs files?
How do you plan to support pin configuration (as we discussed on slack)? We could try to mimic the config-pin tool which I think just interfaces sysfs to find the current pin configuration, however this would make things more complicated. Alternatively we could just allow the user to overwrite specific pins?

Detection should be done via /proc/device-tree.
Let's aim to have the header definition first, and add magicness later.

Done (enough for now)


Functions too tighly coupled to Activities.

Can we please pass a Context as a parameter, instead of passing Activity, only in the end to call Activity#getAppplicationContext().  This does not go well with ViewModels, for example. We should be able to use it, anywhere else, apart from Activities.

I can't do this because of: https://github.com/marchinram/RxGallery/blob/master/library/src/main/java/com/marchinram/rxgallery/RxGallery.java#L192
I need to start the invisible activity that manages the gallery functionality

You don't an Activity  instance to start another Activity. The Context has Context#startActivity(intent);.  @

Ooops of course, I can fix this up later today of if you want sooner make a pr

Fixed in 0.6.5 commit ebd129d0b90def708d3787ec41ffc3c4bff263c4

Thank you so much @marchinram


[Feature discussion] Loadbalancer support to route traffic directly to Pods instead of NodePort.

Loadbalancer is able to route traffic directly to Pods instead of NodePort for some cloud platforms with cni supported.
such as:

GKE container-native LB: https://cloud.google.com/kubernetes-engine/docs/concepts/container-native-load-balancing
EKS NLB IP mode: https://github.com/aws/containers-roadmap/issues/981

So should we add a new loop like syncEndpoint into loadbalancer controller to update loadbalacer with pods ips
func (s *Controller) Run(ctx context.Context, workers int) {
	for i := 0; i < workers; i++ {
		go wait.UntilWithContext(ctx, s.worker, time.Second)
	}

	go s.nodeSyncLoop(ctx, workers)

        // Add syncEndpoint code here
}

AWS at least seems pretty clear they will deprecate the service controller for their own AWS Load Balancer Controller.


Multiple controllers popping up

I'm not sure where to track this problem example because there are multiple components involved:

I'm on a raspberry pi
I added $HOME/Documents/steamcontroller/scripts/sc-xbox.py start to my .bashrc so the controller loads on startup.
I'm running emulationstation (via retropie)
emulationstation is running an SNES emulator
The SNES emulator says the xbox controller is connected to port #0

Then after a while of playing, it says something connected to port #1 and then disconnects. The same starts happening with port #2. In order to fix the problem, I've gotta run sc-xbox.py stop and then sc-xbox.py start. I checked lsusb and nothing new shows up, so I don't know why it starts recognizing multiple controllers plugged in. Any ideas?

The same happens in the case where #4 above is a genesis emulator. I imagine it'll be true in others too.

What I've noticed is that there are multiple controllers popping up in evtest:
$ evtest
No device specified, trying to scan all of /dev/input/event*
Not running as root, no devices may be available.
Available devices:
/dev/input/event1:        Mini Keyboard
/dev/input/event2:        Mini Keyboard
/dev/input/event3:      Microsoft X-Box 360 pad
/dev/input/event4:      Steam Controller Keyboard
/dev/input/event5:      Steam Controller Mouse
/dev/input/event8:      Microsoft X-Box 360 pad
/dev/input/event9:      Steam Controller Keyboard
/dev/input/event10:     Steam Controller Mouse

but only 1 usb connection:
lsusb
Bus 001 Device 007: ID 1997:2433
Bus 001 Device 006: ID 0a5c:21e8 Broadcom Corp. BCM20702A0 Bluetooth 4.0
Bus 001 Device 005: ID 28de:1142
Bus 001 Device 004: ID 148f:5370 Ralink Technology, Corp. RT5370 Wireless Adapter
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

I ran it through sc-xbox.py debug, and figured out that, it's frequently ending the debug process. When I restart sc-xbox.py debug it reconnects to the right controller port, but when the daemon is running, it assigns to another port.

wrapping sc-xbox.py debug in a bash while loop seems to alleviate some of my pain. The controller still disconnects and reconnects frequently, but at least it reassigns to the right port now.

Hi,
Are you using it with the wireless dongle or from USB port ? do you have
enough power to this USB port ?
What kind of hardware are you using (Pi A, A+, B, B+, 2, ...) ?
What is your power supply ?
I never saw this issue with a PC, I'll try on my Pi B+ and Pi 2 if i have
some free time.
Beside I'll try to make might driver more robust to disconnections.
Regards,
On Sun, Jan 24, 2016 at 3:41 PM, John Jelinek IV<EMAIL_ADDRESS>wrote:

wrapping sc-xbox.py debug in a bash while loop seems to alleviate some of
my pain. The controller still disconnects and reconnects frequently, but at
least it reassigns to the right port now.
—
Reply to this email directly or view it on GitHub
https://github.com/ynsta/steamcontroller/issues/19#issuecomment-174304143
.

I'm mostly using the wireless dongle, but the problem persists when wired and and when wireless. I think I have enough power to the USB port, how can I check? I'm on a Raspberry Pi 2 Model B CanaKit Starter. It's got a 2.5A power supply adapter.

@ynsta: were you able to repro?

Not yet to much work beside this project. Maybe this week end.
On Thu, Jan 28, 2016 at 7:19 AM, John Jelinek IV<EMAIL_ADDRESS>wrote:

@ynsta https://github.com/ynsta: were you able to repro?
—
Reply to this email directly or view it on GitHub
https://github.com/ynsta/steamcontroller/issues/19#issuecomment-176006492
.

very strange -- so when the controller is plugged in, my loops doesn't automatically work: while true; do sc-xbox.py debug; done -- it still disconnects, but I have to ctrl+C to manually stop sc-xbox.py debug so it'll reinitialize. I'm surprised the controller is disconnecting at all while plugged in. Any thoughts or scenarios I should test?

Do you have any traces from sc-xbox.py debug ?
On Sat, Feb 6, 2016 at 4:25 PM, John Jelinek IV<EMAIL_ADDRESS>wrote:

very strange -- so when the controller is plugged in, my loops doesn't
automatically work: while true; do sc-xbox.py debug; done -- it still
disconnects, but I have to ctrl+C to manually stop sc-xbox.py debug so
it'll reinitialize. I'm surprised the controller is disconnecting at all
while plugged in. Any thoughts or scenarios I should test?
—
Reply to this email directly or view it on GitHub
https://github.com/ynsta/steamcontroller/issues/19#issuecomment-180789435
.

it doesn't say anything, so I guess no traces. Are there any flags I should set?

I've learned that my wired cable is kind of defective. It sometimes turns the controller on and off. This accents the issue where it detects multiple xbox 360 controllers. sc-xbox.py debug outputs nothing, unless I hit CTRL+C. Is there any way I can debug this further?


Proposal: explicitly declare dependencies per service.

The proposal is to create Env objects within the server packages.  These objects will contain everything a specific pachyderm service needs to run.  There are a few reasons to do this, with the main goal being better testing.

It's pretty common to lose track of what depends on what.  Eventually we need to know in order to build a feature, and so someone does an inventory of what a service needs, posts it in slack once, and then it is forgotten.  With this model, that knowledge is kept up to date and easily query-able.
This is a contract between the service developers and the test/CI engineers.  The problem of achieving fast tests is roughly equivalent to creating Envs quickly.
We can use the compiler to improve the system architecture.  The question "How much would have to change, for us to get rid of x's dependency on y?" Can be answered precisely, by just removing y from the service x's Env.

This P.R. is not ready to merge.  If the proposal is accepted, the final PR should contain the service Envs for all the services.

This is ready for a review now.  I have addressed some of the higher level feedback offline, but I'm just going to summarize here.
RE: Dependency Injection
I think it is generally a net-negative to buy into a dependency injection framework.  It limits the control we have when creating components and the structure of the application is too important to be obfuscated.
DI seems to make this assumption that there is one way to create something and it is burdensome to do it repeatedly.  There will usually be at least 2 ways to create something.  At least one way in the tests, and one way from the production configuration.  Getting a cheap instantiation of a dependency for use in tests is something we address with the test constructors (of the form NewTest___(t testing.TB)).  That code will generally not be repeated.  Someone will figure out a way to make a DB suitable for testing, or an Auth service suitable for testing and export those constructors for other implementations to consume in their own tests.
Then there is the issue of assuming that each dependency is a singleton.  Maybe this is addressed by the frameworks, and I just didn't look hard enough.  I want to be able to create multiple object stores, multiple DBs, multiple PFSs, multiple pachds, that don't conflict with each other. And I don't want to get multiple references to the same one. We actually have 2 DB connections in the production application.  One constructor for the DB type isn't going to work. It might even be necessary to have multiple ways to create an object in tests.  I could see us having a mostly in memory pachd in the near future, with tests against it using the pach client.  That would be constructed differently than a pach client pointing to a full deployment.
The code that actually wires together the application is probably among the most important for developers to understand in order to reason about the application as a whole.  Anything more than a simple flow of objects from a component to its sub-components is needless complexity in a critical area.  Copying fields from a big struct of dependencies to a smaller one is boring; boring is good.  I don't want to wade through someone else's resume-builder to figure out where a dependency comes from.
RE: Circular Dependencies
While putting together this P.R, I didn't really see any place where a dependency cycle is absolutely necessary.  I'm not trying to change the architecture with this P.R. just make it explicit, so I didn't remove those cycles.  I generally tried to order setup topologically, and have one "reverse" edge for the cycles. Those are represented by a closure func() X vs just an instance of X.  This is what we did for everything with serviceenv.  Now we just do it where we have to.
There are some things that are implemented in a circular way, which are not circular in nature.  The auth interceptor is a good example.  The auth interceptor is needed by the grpc server, which is seemingly needed to create an auth server.  However, this is not the case.  The auth server implemenation can be created without a gRPC server, then passed to the auth interceptor, which is used to create the gRPC server.  Then the auth server is registered with the gRPC server.  That can be tackled in another P.R.
Lastly, this refactor may have made architectural issues more obvious, but they were there before, and fixing them should be out of the scope of this PR.


Lightning network support

Implement support for the bitcoin lightning network

Hello! Yesterday I have implemented that, check out docs at sdk.bitcartcc.com
I will update docs with some examples soon, but you can use api documentation too:
sdk.bitcartcc.com/en/latest/api.html

Also take a look at MrNaif2018/bitcart#51 for some instructions how to start daemons after new update

Did it help you @tiedtoastar?

Closing this for now, will add more docs soon, also check out new usage example in examples/ folder of the repo.


Wrong version number in docs/pom.xml

There is a wrong version number in the docs/pom.xml, so it's not possible to build the documentation package on the develop branch.

fixed with #12


HV-1312 In XML, containerElementType should be renamed container-element-type

https://hibernate.atlassian.net/browse/HV-1312

We need to wait for the metadata API to be updated. Otherwise CI will be broken.

@gsmet can this be merged now (using 2.0.0.Beta1 as API version)?

Closing, will create a grouped PR.


magnetometer's move to PX4Magnetometer and cleanup

I didn't particularly want to do this now as we're closing in on a release, but I found a number of inconsistencies and errors (eg bmm150 completely missing calibration). Many of these drivers are quite similar as they were created by duplicating an existing driver, so this was actually pretty fast and easy to do.

Saves about 2.6 kB of flash on fmu-v2 and > 9 kB on fmu-v4/v5, etc.

LIS3MDL temperature looks bogus.

Quickly verified hmc5883, ist8310, lsm303d, lis3mdl.

pixhawk4 v5 f-450
Modes Tested
Position Mode: Good.
Altitude Mode: Good.
Stabilized Mode: Good.
Mission Plan Mode (Automated): Good.
RTL: Good.
- Procedure
Arm and Take off in position mode, after flying for approximately one minute, switched to altitude then stabilized mode proceed to switch to mission plan mode then make sure that vehicle follows all waypoints as shown in QGC, once completed all waypoin activate RTL
Notes:
No issues noted, good flight in general.
Logs:
https://review.px4.io/plot_app?log=b6c9e488-ab13-4a01-b09b-54968b6b0062
https://review.px4.io/plot_app?log=f25e9a1c-092f-4211-80e0-af9de92b2fc4

Tested on NXP FMUK66 V3
Modes Tested

Position Mode: Good.
Altitude Mode: Good.
Stabilized Mode: Good.
Mission Plan Mode (Automated): Good.
RTL (Return To Land): Good.

Procedure
Arm and Take off in position mode, after flying for approximately one minute, switched to altitude then stabilized mode proceed to switch to mission plan mode then make sure that vehicle follows all waypoints as shown in QGC, once completed all waypoint then trigger RTL.
Notes
No issues noted, good flight in general.
Logs

https://review.px4.io/plot_app?log=234626e0-8e65-450e-ad2e-b2ab5d93867e
https://review.px4.io/plot_app?log=c69cd3df-0328-42a1-89ab-189219caa288

Tested on PixRacer V4
- Modes Tested
Position Mode: Good.
Altitude Mode: Good.
Stabilized Mode: Good.
Mission Plan Mode (Automated): Good.
RTL (Return To Land): Good.
- Procedure
Arm and Take off in position mode, after flying for approximately one minute, switched to altitude then stabilized mode proceed to switch to mission plan mode then make sure that vehicle follows all waypoints as shown in QGC,  once completed all waypoint then swith RTL..
- Notes
In altitude mode, the vehicle did not respect the height that continued to increase in pitch forward
Log:
https://review.px4.io/plot_app?log=e6f911c2-f5a2-48bc-ab05-a2033626add9
https://review.px4.io/plot_app?log=ac5ee748-7309-4945-99b1-f5967b3e853d

pixhawk4 mini v5 f-500
Modes Tested
Position Mode: Good.
Altitude Mode: Good.
Stabilized Mode: Good.
Mission Plan Mode (Automated): Good.
RTL: Good.

Procedure
Arm and Take off in position mode, after flying for approximately one minute, switched to altitude then stabilized mode proceed to switch to mission plan mode then make sure that vehicle follows all waypoints as shown in QGC, once completed all waypoin activate RTL

Notes:
No issues noted, good flight in general.
Logs:
https://review.px4.io/plot_app?log=f97f9b2d-0d08-4d54-9326-6bdc27fdb10c
https://review.px4.io/plot_app?log=7e10a7d8-5513-4f27-b534-a5d699c378ed


support multiple kubeconfig files

Expected Behavior

When using the Dapr cli, I expect it to use the kubeconfigs I already use for everything else.

The `kubectl` documentation suggests using [multiple files to keep your config clean](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/). This means that for each use of the dapr cli, not just `dapr init --kubernetes`, we need to have the `KUBECONFIG` env var specified to one file. This causes additional grief if you have your `current-context` defined outside of that one file, again for better organization.

Behaviour like that of using `kubectl config view` would be more expected.

instead of parsing the (expectedly one and only one) config file
https://github.com/dapr/cli/blob/64c303d8a633423306611b85e597a13cfb9015ac/pkg/kubernetes/client.go#L69

Actual Behavior

➜  ~ ls ~/.kube
config-context           config-microk8s
➜  ~ dapr list -k
❌  multiple kubeconfigs in KUBECONFIG environment variable - /Users/###/.kube/config-context:/Users/###/.kube/config-microk8s
➜  ~ KUBECONFIG=~/.kube/config-microk8s dapr list --kubernetes
❌  invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

Similar but closed issue: #228 , handled by making error message more helpful
Steps to Reproduce the Problem

Split your `~/.kube/` config files into two or more files.

for example:
➜  ~ cd ~/.kube
➜  .kube cat config-context
apiVersion: v1
clusters: null
contexts: null
current-context: microk8s
kind: Config
preferences: {}
users: null
➜  .kube cat config-microk8s
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ####==
    server: https://#.#.#.#:#
  name: microk8s-cluster
contexts:
- context:
    cluster: microk8s-cluster
    user: admin
  name: microk8s
kind: Config
preferences: {}
users:
- name: admin
  user:
    token: ###

Release Note

RELEASE NOTE: add support for multiple kubeconfig files

@qayshp Which version of the CLI are you using?

@qayshp Any updates on this?

@qayshp Any updates on this? What version of CLI and Dapr are you using?

@qayshp Any updates on this? What version of CLI and Dapr are you using?

latest.
Facing the same issue you cannot have multiple kubeconfigs in ENV var. It's really useful when you have to manage multiple clusters via kubectx.

@qqbuby  please comment here with /assign to be assigned to this issue :)
Thanks!

/assign


Bot won't send messages

Hey man, first of all, thank you for making this selfbot! That being said, I want to say I followed the steps correctly because when I launch the js file with "Node bot.js" on the Node js cmp, the bot comes online. However, whenever I type "/spam", nothing happens. I tried debugging the bot.js file by loading the file through Node.js and it says that "Identifier Client has already been declared." I'm not entirely sure what to do.

Did you set the bot to listen for the command from your user account? If the bot is running then that's the main reason I can think of for why it doesn't respond to your command.
Did you modify the file at all to perhaps declare Client more than once?

Hey man thank you for the reply. I actually put in an actual bot token instead of a user token in the config so that's why it didn't work.
It runs well except for when it stops and produces this error:
Error: read ECONNRESET
at TLSWrap.onStreamRead (node:internal/stream_base_commons:220:20)
I have a stable internet connection so I'm not sure how to deal with this. Any ideas?

It is possible that Discord has since changed their software to be better at detecting self bots and prevent this from working, I haven't had a need to use this software in a while so I am unsure at the moment. This response is typed from my phone, so I apologize in advance if the formatting looks strange.
I have several questions that may help me to better solve this problem:

How long after running the code do you begin to experience that error?
Is the error fatal, meaning does it kill the process?
What do you have the time parameters set to?
Have you changed the random message to anything other than a set string?
Are there any other configuration changes that you have made, other than what I have asked about above?

It happens at random, but always at least after an hour or so
Yes, it does kill the process. It prints a lot of other things as well but I felt like it might be overwhelming to paste those here
I haven't set the time parameters
I only changed the message the string sends, keeping your format with the count
I have not

I'm going to close this for now due to inactivity, please reach out if there is anything else that you need help with!


Add --egress-selector-mode flag while starting k3d

Change Overview
Kanister CI tests were failing very frequently with below error while execing a command into a pod
error dialing backend: EOF

We are not able to reproduce this, and did look like a networking issue to me, because this was failing even before initiating the pod deletion. Which means, it's not like the pod is not in the correct state and that is the reason we see this failure.
By looking further I got to this issue https://github.com/k3s-io/k3s/issues/5633 which seems to be very closely related to our failure. This commit incorporates the suggested change in our code.
Pull request type
Please check the type of change your PR introduces:

[ ] :construction: Work in Progress
[ ] :rainbow: Refactoring (no functional changes, no api changes)
[ ] :hamster: Trivial/Minor
[x] :bug: Bugfix
[ ] :sunflower: Feature
[ ] :world_map: Documentation
[ ] :robot: Test

Issues

fixes #issue-number

Test Plan

[ ] :muscle: Manual
[ ] :zap: Unit test
[ ] :green_heart: E2E

Moved to kind


Return empty array if input is empty array

https://github.com/selrond/knit/blob/a3bd9a3904694a71a219e8f24dd65271e2d3cd3d/src/index.js#L7-L9
Idk about other use cases, but to me, it is more useful to have an empty array returned compared to having an error thrown.
The Array.prototype.join() method also doesn't throw an error, it returns an empty string if the array is empty:
[].join(' + ')
''

@ChocolateLoverRaj yep, you’re right. Wanna open a PR?

@selrond I will

@ChocolateLoverRaj out of curiosity—do you use this package? If yes, what kind of project do you use it on?

@ChocolateLoverRaj out of curiosity—do you use this package? If yes, what kind of project do you use it on?

I am using this package here: https://github.com/ChocolateLoverRaj/ezc/blob/777fd637bba3d711b50a72ee8d24b80ca0294858/lib/core/unparseNode/unparseFunction.ts#L60-L77
I haven't added documentation to ezc about what it is yet but basically how I'm using this package is to turn AST nodes into an array of tokens in a programming language that I'm making.

nice! How did you find it?

nice! How did you find it?

I was searching for a package that could insert a value between array elements on https://npms.io


spack install takes an infinite time when installing cardioid

environment: CentOS7+gcc 8.3.0+cmake 3.21.1+cuda 11.1.105+perl 5.16.3
I was trying to install Cardioid following the instruction in README.
After running spack add cardioid+cuda and spack install, the process after ==> cardioid: Executing phase: 'cmake' is printed on my screen seems to take an infinite time to complete (I waited for at least 20 min yet nothing proceeded).
The details is as follows:
$ spack install
==> Installing environment /home/myenv
[+] /home/cmake-3.21.1 (external cmake-3.21.1-kgvtacydi7vchye4nll5bq6xatv3kzal)
[+] /usr/local/cuda (external cuda-11.1.105-szapcmcc4oxqlsjoqlmbq7b4tr6vhhs7)
[+] /usr (external perl-5.16.3-lcr6qhivgtjs74edphkzngm7ofvmpvv2)
[+] /home/openmpi (external openmpi-3.1.5-hlafh73mwzxiolbptzntznccqefxzvxt)
==> Installing openblas-0.3.17-33sulnfbl6uv34rg4htzxyou7p6enf2f
==> No binary for openblas-0.3.17-33sulnfbl6uv34rg4htzxyou7p6enf2f found: installing from source
==> Using cached archive: /home/spack/spack-develop/var/spack/cache/_source-cache/archive/df/df2934fa33d04fd84d839ca698280df55c690c86a5a1133b3f7266fce1de279f.tar.gz
==> No patches needed for openblas
==> openblas: Executing phase: 'edit'
==> openblas: Executing phase: 'build'
==> openblas: Executing phase: 'install'
==> openblas: Successfully installed openblas-0.3.17-33sulnfbl6uv34rg4htzxyou7p6enf2f
  Fetch: 0.13s.  Build: 1m 33.61s.  Total: 1m 33.74s.
[+] /home/spack/spack-develop/opt/spack/linux-centos7-skylake_avx512/gcc-8.3.0/openblas-0.3.17-33sulnfbl6uv34rg4htzxyou7p6enf2f
==> Installing cardioid-develop-lei4go23dpiitmnde5zkcmaaf5umwdgi
==> No binary for cardioid-develop-lei4go23dpiitmnde5zkcmaaf5umwdgi found: installing from source
==> No patches needed for cardioid
==> cardioid: Executing phase: 'cmake'

Same problem occurs when trying the following way to install/build:

make
modified the option in Cmakecache to enable cuda

See #9, this should be able to fix the issue


a level up rep system

probably an insane ask, but would be cool if we could rank / level up using rep and it displays our lvl.
Also would love to eventually see a way to upgrade gear for better stats at the garage. No doubt none of this would be possible but still hope for it

This'd require a way to track players beyond just username, so it's most likely out of scope. Sorry! We don't plan on adding an account system.


ABI for syscall and call instructions

This PR updates the ABI of the assembly procedures, making their usage with call and syscall safe.
Corresponding issue: #685

During the refactoring I also updated the doc comments, making them consistent and uniform. I think this is a good opportunity to create some kind of format guidebook (or stylebook) to have some rules and templates of how doc comments (and internal procedure comments) should be formatted. It is still a matter of discussion, but here is what I used during this refactoring.
Format guidebook
A book defining the format for documentation comments and regular comments for masm procedures.
General
Entire procedure doc comment should be created using the #! pair of symbols as the commenting sign.
Doc comment for a procedure should have these blocks:

Procedure description.
Inputs and outputs.
Description of the values used in the "Inputs and outputs" block (optional).
Panic block (optional).
// Annotation hint (optional and will became redundant after implementation of the procedure annotations)

Each block should be separated from the others with a blank line.
Example:
#! Sets the code of the account the transaction is being executed against.
#!
#! Inputs:  [CODE_COMMITMENT]
#! Outputs: []
#!
#! Where:
#! - CODE_COMMITMENT is the hash of the code to set.
#!
#! Panics if:
#! - this procedure is executed on the account which type differs from the `basic mutable`.
#!
#! Annotation hint: is used only with `exec`

Procedure description
Contains the general information about the purpose of the procedure and the way it works. May contain any other valuable information.
If some list is used for description, it should be formatted like so:

The description of the list should not have a blank like between it and the list.
Description should have a colon at the end.
Depending on what kind of sentences form a list, they should start with a capital letter and end with a dot, or start with a lowercase letter and without dot at the end.
List should use a - symbol in case of unordered list or arabic numerals for ordered ones (for example, for the description of the execution steps).
Nested list should follow the same format.

Some data could be formatted as a subparagraph, in that case a blank line should be used to separate them (note: not sure about that, wouldn't it be confusing since we are using a blank line to separate different blocks?)
Example:
#! Transaction kernel program.
#!
#! This is the entry point of the transaction kernel, the program will perform the following
#! operations:
#! 1. Run the prologue to prepare the transaction's root context.
#! 2. Run all the notes' scripts.
#! 3. Run the transaction script.
#! 4. Run the epilogue to compute and validate the final state.
#!
#! See `prologue::prepare_transaction` for additional details on the VM's initial state, including
#! the advice provider.

Inputs and outputs
Each variable could represent a single value or a sequence of four values (a Word). Variable representing a single value should be written in lowercase, and a variable for the word should be written in uppercase.
Example:
#! Inputs: [single_value, SOME_WORD]

Inputs
Inputs block could contain three containers: operand stack, advice stack and advice map.
Description of the each container should be offseted with two spaces relative to the start of the Inputs word.
Each name of the container should be separated from its value by the colon (e.g. Operand stack: [value_1]).
Operand stack and advice stack should be presented as an array containing some data.
// Not sure about the next section, I have two different options
// Option 1:
If the input line exceeds 100 elements it should be broken, and the end of the line should be moved to the new line with an offset such that the first character of the first element on the second line should be directly above the first character of the first element on the first line (see the value of the FOREIGN_ACCOUNT_ID in the example in Formats section).
// Option 2:
If the input line exceeds 100 elements, an array should be formatted in a column, forming a Word on each line. Each word should be offseted with two spaces relative to the name of the container.
Example:
#! Inputs:
#!   Operand stack: []
#!   Advice stack: [
#!     account_id, 0, 0, account_nonce,
#!     ACCOUNT_VAULT_ROOT,
#!     ACCOUNT_STORAGE_COMMITMENT,
#!     ACCOUNT_CODE_COMMITMENT
#!   ]

// end of the option 2
To show that some internal value array could have dynamic length, additional brackets should be used (see the [VALUE_B] in the advice stack in the example in Formats section). (note: not sure that this is a good idea)
Advice map should be presented as a sequence of the key-value pairs in the curly brackets. Opening bracket should stay on the same line, and the closing bracket should be placed on the next line after the last key-value pair with the same offset as the Advice map phrase.
Each pair should start at the new line with additional two spaces offset relative to the start of the Advice map phrase. Pairs should be separated with comma. The same formatting rules as to the operand and advice stacks should be applied for the each key-value pair .
Outputs
Outputs should show the final state of each container, used in the inputs, except for the advice map: almost always the final state of the advice map is unimportant (since it is always the same as at the beginning of the procedure execution).
Formats
Full version
In case the values are provided not only through the operand stack, but also through any other container, the full version if the inputs should be used.
Notice that operand stack should be presented in any case, even if it is empty. Other containers should be presented only if they have some values used in the describing function.
Example:
#! Inputs:
#!   Operand stack: []
#!   Advice stack: [VALUE_A, [VALUE_B]]
#!   Advice map: {
#!     FOREIGN_ACCOUNT_ID: [[foreign_account_id, 0, 0, account_nonce], VAULT_ROOT, STORAGE_ROOT,
#!                          CODE_ROOT],
#!     STORAGE_ROOT: [[STORAGE_SLOT_DATA]],
#!     CODE_ROOT: [num_procs, [ACCOUNT_PROCEDURE_DATA]]
#!   }
#! Outputs:
#!   Operand stack: [value]
#!   Advice stack: []

Short version
In case the values are provided only through the operand stack, a short version of the inputs and outputs should be used. In that case only Inputs and Outputs containers are used, representing the values on the operand stack.
Input values array should be offseted by one space to be inline with the output values array (see the example).
Example:
#! Inputs:  [single_value, WORD_1]
#! Outputs: [WORD_2]

Description of the used values
If some value was used in the inputs and outputs block (and its meaning is not obvious) this value should be described.
Values description block should start with Where word with a colon at the end.
Definitions should be represented as an unordered list constructed with - symbols, without any space offset.
Each definition should start with the name of the variable followed by the is/are the phrase (note: not sure about that), after which the definition should be placed. At the end of each definition should be a dot.
Example:
#! Where:
#! - tag is the tag to be included in the note.
#! - aux is the auxiliary metadata to be included in the note.
#! - note_type is the storage type of the note
#! - execution_hint is the note's execution hint
#! - RECIPIENT is the recipient of the note.
#! - note_idx is the index of the crated note.

Panic block
If the describing procedure could potentially panic, a panic block should be specified.
Panic block should start with Panics if phrase with a colon at the end.
Panic cases should be represented as an unordered list constructed with - symbols, without any space offset. Definitions should start with lowercase letter, except for the cases which form the nested list (see example). Each case should end with a dot.
Example:
#! Panics if:
#! - the transaction is not being executed against a faucet.
#! - the invocation of this procedure does not originate from the native account.
#! - the asset being burned is not associated with the faucet the transaction is being executed
#!   against.
#! - the asset is not well formed.
#! - For fungible faucets:
#!   - the amount being burned is greater than the total input to the transaction.
#! - For non-fungible faucets:
#!   - the non-fungible asset being burned does not exist or was not provided as input to the
#!     transaction via a note or the accounts vault.

Annotation hint
A temporary comment showing how the procedure is used. It will help to implement the procedure annotations in future.
The hint could show how this procedures is invoked:

only with exec
only with call/syscall
with both exec and call
is not used anywhere

Regarding the procedures ABI the only thing left is to handle the auth_tx_rpo_falcon512 and end_foreign_context procedures. The point why I left them is because they don't require anything be on the stack and return nothing, so each call looks like so:
# pad the stack before call
padw padw padw padw

call.<...>

# clean the stack
dropw dropw dropw dropw

This looks quite strange to me, but that's how the call should be handled.
So I'm not sure does it worth it to apply ABI rules for these procedures, since we will have a lot of these redundant pads and drops.

@PhilippGackstatter Thank you for the detailed review!

Description should have a colon at the end.

Did you mean "period" instead of "colon"?

Sorry, I wasn't specific enough, I phrased it in a confusing way. By the "colon at the end" I meant only the list description, not the overall procedure description. I'll update this line.
Note section sounds good to me, but I'm not sure what is better in terms of standardization: if I remember correctly, we don't use a dedicated note section in miden-vm, just a new paragraph.

You mostly replaced [...] with pad or nothing, and I think that's fine. There are still some occurrences of this string in the codebase, and we should probably also update those. If I understood correctly, what was previously [x, y, ...] is now [x, y] and as mentioned for the executable procedures which truncate the stack, those should use pad. Which is all to ask that we no longer need ... anywhere, right?

Right. Previously we used the ... to show that there could be other elements on the stack, so we should not modify (or do it carefully) the rest of the stack (it was mostly referred to the call instructions). Now for the procedures which are executed this is true by default, although these procedures should not modify the rest of the stack, and for the call instructions we always should explicitly pad the stack before the call, so we show this with the pad(x), which should guarantee that we don't have anything valuable deeper on the stack.

Examples are easier to read and extrapolate from than a textual description <...> Having fairly exhaustive examples in the guidelines that cover many possible cases would be great,<...> and I would go further in that direction.

I agree, it's much easier for me to look at the example than to read the definition. We should definitely improve the current version, for now it is just a raw and relatively poorly structured set of my thoughts.

Not a review yet, but a couple of comments:

Regarding the procedures ABI the only thing left is to handle the auth_tx_rpo_falcon512 and end_foreign_context procedures. The point why I left them is because they don't require anything be on the stack and return nothing, so each call looks like so:
# pad the stack before call
padw padw padw padw

call.<...>

# clean the stack
dropw dropw dropw dropw

This looks quite strange to me, but that's how the call should be handled. So I'm not sure does it worth it to apply ABI rules for these procedures, since we will have a lot of these redundant pads and drops.

As discussed offline, I would go for consistency here even if it ends up costing extra 32 cycles.

You mostly replaced [...] with pad or nothing, and I think that's fine. There are still some occurrences of this string in the codebase, and we should probably also update those. If I understood correctly, what was previously [x, y, ...] is now [x, y] and as mentioned for the executable procedures which truncate the stack, those should use pad. Which is all to ask that we no longer need ... anywhere, right?

I would make a distinction here as follows:
[a, b, c] - means that c is at the bottom of the stack (i.e., there are no elements after it).
[a, b, c, ...] - means that there may be other values on the stack that a procedure doesn't care about
                 and does not modify.

So, for something like truncate_stack, we'd use:
#! Inputs:  [pad(16)]
#! Outputs: [OUTPUT_NOTES_COMMITMENT, FINAL_ACCOUNT_HASH, tx_expiration_block_num, pad(7)]

because we expect that the depth of the stack is exactly 16 when the procedure is entered.

cc @igamigo - changes in this PR may have small effects on the client (e.g., in how we call note scripts). Could you check if everything still works fine there?

@Fumuran - could you extract the Format Guidebook from https://github.com/0xPolygonMiden/miden-base/pull/971#issuecomment-2476856816 and add it as a stand alone Markdown file in miden-lib crate?


Autorun of the program does not work

Autorun does not work. And the autorun setting itself is constantly being reset. For example, if you go to the "Settings" menu and then return to the Home menu, the "Autostart" setting is reset. In the previous version 2.0.2, autorun works.

Oops, the rewritten version (0.3.0) still needs to be improved. (Actually, the rewritten version has not been tested much.)
Thanks for taking the trouble to try it. I'll try to fix it a bit. Please wait.

There are no more problems with autorun in the updated version. Autorun is working.

It's wonderful. Now I'll close.


Align with breaking change in workflow-task-status API (fractal-server 2.0.0a10)

fractal-server 2.0.0a10 (to be released in a few minutes) includes a small API breaking change.
The old endpoint
GET /api/v2/project/{project_id}/dataset/{dataset_id}/status/

becomes
GET /api/v2/project/{project_id}/status/?dataset_it={dataset_id}&workflow_id={workflow_id}

Note that we changed the internal logic (ref https://github.com/fractal-analytics-platform/fractal-server/pull/1416), and we slightly restricted its response contents: now the response only includes workflow tasks of a specific workflow, rather than all the ones that ever ran for a given dataset. In fractal-web this should introduce no problem, since the endpoint is called from a single-workflow page and the workflow ID is known.

Another (very minor) change is that the status property of the response body goes from being "object or null" to being "object" - ref https://github.com/fractal-analytics-platform/fractal-server/pull/1416/commits/63c66781faa8858d1395d0108c54ef45677ecd22.

2.0.0a10 is now available


dev cannot be merged into master

Required for https://github.com/HIP-infrastructure/app-in-browser/pull/106

Should be good now , did a little trimming

Thanks.


Fixed the order of coordinates in predictions.csv for WSI inference

Fixes #492
Proposed Changes

we are returning valid coordiantes in terms of [width, height], so the processing should be in terms of [x,y]

Checklist

[x] I have read the CONTRIBUTING guide
[x] My PR is based from the current GaNDLF master
[x] Non-breaking change (does not break existing functionality): provide as many details as possible for any breaking change
[x] Function/class source code documentation added/updated
[x] Code has been blacked for style consistency
[x] If applicable, version information has been updated in GANDLF/version.py
[x] If adding a git submodule, add to list of exceptions for black styling in pyproject.toml file
[x] Usage documentation has been updated, if appropriate
[x] History has been updated, if appropriate
[x] Tests added or modified to cover the changes; if coverage is reduced, please give explanation
[x] If customized dependency installation is required (i.e., a separate pip install step is needed for PR to be functional), please ensure it is reflected in all the files that control the CI, namely: python-test.yml, and all docker files [1,2,3,4]

Looks good to me.


Change the title of rkt page to say rkt, not rat.

Pretty sure there is no such thing as rat yet.
cc @dchen1107 @vishh

LGTM :rat:

LGTM

LGTM


macOS build instructions

I'm going to merge this and then make revisions to several files.

Okay, I've refactored macOS support to conform to my vision for the build files generator module of utilikit. I did a simple test and the code seems to work. If there are any other changes you want to make, let me know.


Receiver and sender nodes agree on proof courier address protocol

As a first step in sending an asset, the receiver node generates a tap address. The address contains a proof courier service address (which is used in transferring proofs related to the send procedure).
The proof courier address includes a schema/protocol specifier. We must assume that a particular version of the tap daemon can only support a subset of all possible proof courier address protocols. (Later tap daemons could support protocols that earlier daemons do not.)
We need to design a means of selecting/negotiating a viable proof courier protocol based on the capabilities of the receiver and sender nodes.

We need to design a means of selecting/negotiating a viable proof courier protocol based on the capabilities of the receiver and sender nodes.

I don't think the sender+receiver would have an interactive session to communicate that. This is why the potential defaults (and also the eventual sync feature of cross multiverse instances) are important.

I agree that we probably don't need to handle that. If the receiver is on an old version, then the sender should be fine. If the sender is on an old version and doesn't understand the address protocol, then I think an error message along the lines of "unknown proof courier protocol  in address, please update your tapd" should be fine.

If the sender is on an old version and doesn't understand the address protocol, then I think an error message along the lines of "unknown proof courier protocol in address, please update your tapd" should be fine.

I'll update this issue's title to reflect that this is the work to be done.

Fixed by https://github.com/lightninglabs/taproot-assets/pull/497


Compared with Notepad++ and Sublime

Some of the basic functions of the missing.
For example:
Block edit, mouse drag and drop selection
Temporary storage is not lost, notepad++ temporary open tab, if you do not save close notepad++, tab content will not be emptied
Multiple line to line pattern
These basic features, atom have to rely on the installation of third party plug-ins.

Atom is called "a hackable text editor" because users can configure- and extend it to their needs. It provides a lot of basic functions and I do not agree that your listed features are all basic.

mouse drag and drop selection

There are already packages like simple-drag-drop-text to add this feature.
This can already be enabled in the settings too.

Temporary storage is not lost, notepad++ temporary open tab, if you do not save close notepad++, tab content will not be emptied

Anyway, you should create a ticket per missing feature and wait for a Atom team member to reply because I don't know if there are plans to implement such features into the core.

Thanks for reaching out!
Because we treat our issues list as the Atom team's backlog, we close issues that are questions since they don't represent a task needing to be completed. For most questions about Atom there are a lot of options, check out:

The Atom Flight Manual
The Atom FAQ on Discuss
Discuss, the official Atom and Electron message board
Join the Atom Slack team

On Discuss and in the Atom Slack team, there are a bunch of helpful community members that should be willing to point you in the right direction.


Interpreter: fix fiber's resumable property

The resumable property of fiber contexts was wrong in interpreted code: it was never reset to 0.
This patch lets the interpreted fiber read the resumable property of the interpreter's fiber (the real one) though a new primitive. The interpreter is now responsible from reporting the value properly. This avoids having to manipulate resumable in the interpreted code, and allows to act on the actual state.
Alternative: the interpreter could set the resumable properties of the fibers in the interpreter_swapcontext primitive, it would be fake but probably not critical (the interpreter's scheduler would still act on the real property), but we don't have references back into the interpreted fibers (we could add it when interpreted) + I don't know how the interpreter updates values into an interpreted type (I could learn).

@beta-ziliani I just added a test. It fails with the interpreter of crystal 1.11 but succeeds otherwise (this patch, or when compiled).


Makefile: Fix installation and stripping

Found a few problems with the Makefile while reviewing it for Void Linux

Separate stripping into another step of the package
Conditionalize stripping the binary, distros don't like when packages do
it because they want the debug symbols included and do their own
stripping after extracting them
use -Dm755 to create the directory (-D) and set the correct executable
mode
respect PREFIX which can be set to /usr /usr/local and even /
respect DESTDIR which is set by distros when they are packaging

Thanks; I had initially just added a simple Makefile for my personal purposes. Never expected a lot of people to use (much less package) this.


Codecov fix

Codecov Report

Merging #37 (843ec81) into main (249678f) will decrease coverage by 31.34%.
The diff coverage is 17.40%.

@@             Coverage Diff             @@
##             main      #37       +/-   ##
===========================================
- Coverage   66.25%   34.91%   -31.35%
===========================================
  Files         306      140      -166
  Lines       46663    21087    -25576
===========================================
- Hits        30918     7363    -23555
+ Misses      13343    13081      -262
+ Partials     2402      643     -1759

Flag
Coverage Δ

integration-tests
34.91% <17.40%> (-0.17%)
:arrow_down:

kind-e2e-tests
?

unit-tests
?


Unexpected input 'PULL_REQUEST_AUTO_MERGE_METHOD',

PULL_REQUEST_AUTO_MERGE_METHOD is a Unexpected input but is in documentation
Warning: Unexpected input(s) 'PULL_REQUEST_AUTO_MERGE_METHOD', valid inputs are ['GITHUB_TOKEN', 'FROM_BRANCH', 'TO_BRANCH', 'PULL_REQUEST_TITLE', 'PULL_REQUEST_BODY', 'PULL_REQUEST_IS_DRAFT', 'CONTENT_COMPARISON', 'REVIEWERS', 'TEAM_REVIEWERS']

I'm getting the same thing

PULL_REQUEST_AUTO_MERGE_METHOD was added in #76, maybe a new version needs to be released

@TreTuna can you release a new version please?

+1 getting this warning


Bitmoji issue with 1.47.0

With a 1.47.0 release we are getting weird blinking of the Bitmoji in the game same as in this project: https://playcanvas.com/editor/scene/1023597
Steps to Reproduce

Open: https://playcanvas.com/editor/scene/1023597
Hit Play from Editor

And after a few reloads, I see flicker again.

we're on it, thanks!

Looks like it is fixed. we are not getting this bug anymore.


Adding USEEXT to pkg.info does not trigger pkg rebuild

Specifically, I was adding "Geant4" to the dgpriv OSGExamples/pkg.info, and it diid not rebuild.

Adding USEPKG Utils on the other hand, DOES trigger a rebuild of the package.


Improve gulp build commands

I have some tests using htmlproofer and it's cool, but my gulp workflow is kind of broken. Produces things similar to #9 picture when changing a file. Like if the order of the executed commands was not right, something like that.

[x] make gulp build work
[ ] Fix reload with browsersync that seems to run before sass and coffee are ready
[ ] Have the tests running from gulp test

3 years ago, gulp used to be a thing 😅

3 years ago, gulp used to be a thing 😅


Streamline injection of WM monitoring metrics to MONIT infrastructure

Impact of the new feature
With increased number of information we would like to monitor I suggest to streamline the procedure how we inject monitoring metrics into MONIT infrastructure.
Is your feature request related to a problem? Please describe.
Currently, we inject FWJR nested (JSON wise) documents into WMArchive data-service. First, this nested data-format is very difficult to handle from monitoring point of view as it requires very complex queries and CMS Monitoring group advised to flat out our data. Second, we get request to add additional metrics, see #11538 , and provided aggregated metrics. Most likely, in a future our monitoring metrics may change based on evolution of the system and use-cases. Therefore, it is desired to review current procedure and provide a flexible way to accommodate current and future use-cases.
Describe the solution you'd like
While working on issue #11538 I made a proposal to streamline injection of monitoring metrics as following:

use WMArchive service (or more general CMSAMQProxy) to inject monitoring metrics into MONIT, and convert it to a proxy service

it will be available as normal CMSWEB service and be open to clients via proper cert or token authentication
it will be using CERN AMQ brokers to inject data into MONIT infrastructure
it should support data injection to different MONI topics

current data-format of FWJR documents we inject to WMArchvie has nested dict structure and it should be flatten out to improve CMS Monitoring needs, e.g. queries, dashboards, etc.

during transition from nested to flatten data-format we should preserve current WMArchive data-format to ensure backward compatibility with existing dashboards
we create flat out FWJR data for monitoring purposes and send it over to separate topic
- for that we will need to request a new MONIT topic
- we can flat out data either within WMCore code-base or as part of WMArchive (or CMSAMQProxy) service

we will add new layer to WMArchive (or CMSAMQProxy) to aggregate information, e.g. take metrics A and B and create another one via specific math operation/function. This aggregated data should flow to independent MONIT topic

for that we will need to request and use separate MONIT topic
I suggest to provide aggregated records via JSON data-format, e.g.

[...
    {"metric": "aggregated-metric-X", "function": "sum", "metrics": ["metric1', "metric2"]}
...]

The format of this input for aggregation should be properly determined though.
-All new metrics will be shipped to new proxy service in a raw (flat) form
Describe alternatives you've considered
We may continue using nested FWJR data-format for WMArchive but it leads to potential problems:

current data may become very large due to inclusion of LFNs, lumis and MONIT infrastructure imposes a limit of 20MB (or at this order) for document size. We already strip off LFNs and lumis from the document but adding more and more metrics to it can hit this limit again
adding new metrics, e.g. aggregated one, may require code changes and it is better to delegate this task to the service instead of patching WM components.
moreover, we should support streaming data into different end-points (topics) to scale better the system and currently WMArchive service does not support this

Additional context
This issue can be converted into Meta-Issue with the following sub-tasks:

[ ] Converge WMArchvie as a part of CMSAMQProxy service
[ ] Adjust CMSAMQProxy service to support multiple MONIT end-points (topics)
[ ] Flat-out WMArchive data either in WMArchvie or WMCore sides
[ ] Add aggregation layer to CMSAMQProxy
[ ] Integration tasks with CMS Monitoring group to create new topics for multiple data-streams
[ ] Convert existing WMArchive dashboard from using nested (dict) data-format to new flat data structure

FYI: @amaltaro , @todor-ivanov , @klannon , @leggerf, @brij01
I suggest to include this (meta) issue to next phase of planning for WMCore activities. Since it is x-issue between multiple groups we should properly identify scope, responsibilities, and R&D tasks within each group. For example, we should identify which group is responsible for support, maintenance and development of WMArchive/CMSAMQProxy service and plan accordingly.

@vkuznet thank you for creating this issue and pointing out to potential sub-tasks.

Converge WMArchvie as a part of CMSAMQProxy service

I have a question on this sub-task though. Do you mean merging the WMArchive capabilities in CMSAMQProxy and then deprecating the WMArchive service in favor of CMSAMQProxy? Or is this line just about the line of support for both WMArchive and CMSAMQProxy?
Other details can be discussed when we get to the point of writing those tickets down.
I would like to suggest to rename this ticket though to something like: "Streamline monitoring of CMSSW FJR metrics within MONIT infrastructure" or something like that. I understand that CMSAMQProxy will be generic, but all the developments listed here are initially targetting a better propagation and monitoring of the CMSSW metrics.

I think we should merge WMArchive into CMSAMQProxy and make the later as generic service independent from data, clients, etc.


How to organize yourself to get things done

  How to organize yourself to get things done

  Questions and comments for
https://www.ramonmorcillo.com/how-to-organize-yourself-to-get-things-done/

Very nice article sir


Istio e2e, Use BeforeAll and AfterAll for VMI used as a HTTP Server

What this PR does / why we need it:
Ginkgo v2 allows using BeforeAll and AfterAll to safe time
of setup stage by creating VMs once for multiple specs.

In this commit, Server VMI used in Outbound traffic tests
is created only once for all specs.
All the specs use the same Server VMI, so it's a perfect candidate
to create it only once.

BeforeAll can only be used in Ordered containers, which is why the
Outbount Traffic container (describe block) is decorated with Ordered
decorator.

BeforeEach is decorated with OncePerOrdered decorator, so that
in Ordered containers, it's executed only once before the first spec.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Release note:

NONE

How much do we gain by introducing this?

This specific change safes fairly little at this point. The server VMI is created once and re-used, but the client VMIs are still created beforeEach spec.
More restructuring is needed to take bigger advantage.
There is still potential to safe a couple of VMI creations.


Modify auth

Use OAuth2 as local auth. This means no local users. This has some advantages.

Password recovery is possible.
We need the oauth token in any case.

Look into how to use a dynamic address for the OAuth2 provider so that users can connect to third party servers if they wish.

Done.


Updating Read Me for mbircone

Following updates have been made to allow for error free installation of mbircone

Under "Install Demo Requirements section" changed requirement.txt to requirements_demo.txt
Under "Installation Section" included a point to install GCC per the instructions found on SVMBIR page

Surojit, if I understand correctly, your only changes in this PR are to the README.md file. So you are just making fixes to the instructions.
Is that correct?
In the future, you should list Diyu as a checker too.


963 fix the previews

Context
it bork
Changes proposed in this pull request
Remove code rot, enforce rot code
Guidance to review
Visit
'/view_components'

 Fix the previews


Fix links of examples and add link of GCE PD examples

Fix links of examples and add link of GCE PD examples.

/lgtm