Enable test suite for MIR optimizations again

miri's test suite used to also run with enabled MIR optimizations, but does not any more because of too many bugs.  We should get it enabled again, eventually.

[ ] For run-pass, see https://github.com/rust-lang/miri/pull/926#issuecomment-525162906
[ ] For compile-fail, the current blocker is https://github.com/rust-lang/rust/issues/50411.

The run-pass test suite works with optimizations now, but the compile-fail test suite still explodes with higher optimization levels.

Had to disable optimizations for run-pass again, likely caused by https://github.com/rust-lang/rust/pull/63580.

So, I'm trying to test some local changes to rustc I have and I'm getting build errors when I run ./miri test:
error[E0463]: can't find crate for `core`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `unicode-width`.
warning: build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `term`.
warning: build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `proc_macro`.

To learn more, run the command again with --verbose.
error: `"/home/wesley/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo" "build" "--release" "--manifest-path" "/tmp/xargo.uuhYMKXriUiG/Cargo.toml" "--target" "x86_64-unknown-linux-gnu" "-p" "test"` failed with exit code: Some(101)
note: run with `RUST_BACKTRACE=1` for a backtrace
fatal error: Failed to run xargo

I've built my local rustc with ./x.py build --stage 2, I've added it to rustup as a custom toolchain, and I've run rustup override in the miri repo. Any ideas what might be causing that?

That is strange. I regularly build Miri against a local rustc. Usually I do ./x.py build src/rustc, maybe that makes the difference? The directory you added as custom toolchain is .../build/x86_64-unknown-linux-gnu/stage2, right? Which OS/platform are you on?
In which part of the Miri build process does the error occur? Is it while building Miri, or while building the custom libstd? Please post a complete build log, starting with your ./miri invocation.

Ok, so here's what I just tried:

In ~/code/rust/rust run ./x.py build src/rustc
In ~/code/rust/miri run rustup override set stage2 which links to ~/code/rust/rust/build/x86_64-unknown-linux-gnu/stage2
In ~/code/rust/miri run ./miri test

Which does this:

wesley@endurance:~/code/rust/miri> ./miri test
   Compiling autocfg v0.1.6
   Compiling proc-macro2 v0.4.30
   Compiling unicode-xid v0.1.0
   Compiling libc v0.2.62
   Compiling cc v1.0.41
   Compiling syn v0.15.44
   Compiling proc-macro2 v1.0.1
   Compiling unicode-xid v0.2.0
   Compiling failure_derive v0.1.5
   Compiling syn v1.0.5
   Compiling rustc-demangle v0.1.16
   Compiling memchr v2.2.1
   Compiling bitflags v1.1.0
   Compiling cfg-if v0.1.9
   Compiling serde v1.0.99
   Compiling ryu v1.0.0
   Compiling log v0.4.8
   Compiling rand_core v0.4.2
   Compiling lazy_static v1.4.0
   Compiling getrandom v0.1.11
   Compiling byteorder v1.3.2
   Compiling itoa v0.4.4
   Compiling regex-syntax v0.6.11
   Compiling quick-error v1.2.2
   Compiling semver-parser v0.7.0
   Compiling termcolor v1.0.5
   Compiling hex v0.3.2
   Compiling rustc-workspace-hack v1.0.0
   Compiling shell-escape v0.1.4
   Compiling thread_local v0.3.6
   Compiling rand_core v0.3.1
   Compiling rand_jitter v0.1.4
   Compiling humantime v1.2.0
   Compiling rand_hc v0.1.0
   Compiling rand_xorshift v0.1.1
   Compiling rand_isaac v0.1.1
   Compiling num-traits v0.2.8
   Compiling num-integer v0.1.41
   Compiling rand_chacha v0.1.1
   Compiling rand_pcg v0.1.2
   Compiling rand v0.6.5
   Compiling aho-corasick v0.7.6
   Compiling quote v0.6.13
   Compiling quote v1.0.2
   Compiling time v0.1.42
   Compiling dirs-sys v0.3.4
   Compiling atty v0.2.13
   Compiling rand_os v0.1.3
   Compiling directories v2.0.2
   Compiling regex v1.2.1
   Compiling chrono v0.4.7
   Compiling backtrace-sys v0.1.31
   Compiling env_logger v0.6.2
   Compiling vergen v3.0.4
   Compiling synstructure v0.10.2
   Compiling backtrace v0.3.35
   Compiling serde_derive v1.0.99
   Compiling failure v0.1.5
   Compiling miri v0.1.0 (/home/wesley/code/rust/miri)
   Compiling semver v0.9.0
   Compiling serde_json v1.0.40
   Compiling rustc_version v0.2.3
   Compiling cargo_metadata v0.8.2
    Finished release [optimized] target(s) in 1m 14s
    Finished release [optimized] target(s) in 0.02s
     Running `target/release/cargo-miri miri setup`
    Updating crates.io index
   Compiling compiler_builtins v0.1.19
   Compiling core v0.0.0 (/home/wesley/code/rust/rust/src/libcore)
   Compiling cc v1.0.41
   Compiling libc v0.2.62
   Compiling build_helper v0.1.0 (/home/wesley/code/rust/rust/src/build_helper)
   Compiling cmake v0.1.42
   Compiling unwind v0.0.0 (/home/wesley/code/rust/rust/src/libunwind)
   Compiling rustc_tsan v0.0.0 (/home/wesley/code/rust/rust/src/librustc_tsan)
   Compiling rustc_asan v0.0.0 (/home/wesley/code/rust/rust/src/librustc_asan)
   Compiling rustc_lsan v0.0.0 (/home/wesley/code/rust/rust/src/librustc_lsan)
   Compiling rustc_msan v0.0.0 (/home/wesley/code/rust/rust/src/librustc_msan)
   Compiling std v0.0.0 (/home/wesley/code/rust/rust/src/libstd)
   Compiling rustc-std-workspace-core v1.0.0 (/home/wesley/code/rust/rust/src/tools/rustc-std-workspace-core)
   Compiling cfg-if v0.1.8
   Compiling alloc v0.0.0 (/home/wesley/code/rust/rust/src/liballoc)
   Compiling panic_abort v0.0.0 (/home/wesley/code/rust/rust/src/libpanic_abort)
   Compiling rustc-std-workspace-alloc v1.0.0 (/home/wesley/code/rust/rust/src/tools/rustc-std-workspace-alloc)
   Compiling panic_unwind v0.0.0 (/home/wesley/code/rust/rust/src/libpanic_unwind)
   Compiling hashbrown v0.5.0
    Finished release [optimized] target(s) in 28.99s
   Compiling unicode-width v0.1.6
   Compiling term v0.0.0 (/home/wesley/code/rust/rust/src/libterm)
   Compiling proc_macro v0.0.0 (/home/wesley/code/rust/rust/src/libproc_macro)
error[E0463]: can't find crate for `core`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `unicode-width`.
warning: build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `proc_macro`.
warning: build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `term`.

To learn more, run the command again with --verbose.
error: `"/home/wesley/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo" "build" "--release" "--manifest-path" "/tmp/xargo.kHu0e3jhZph2/Cargo.toml" "--target" "x86_64-unknown-linux-gnu" "-p" "test"` failed with exit code: Some(101)
note: run with `RUST_BACKTRACE=1` for a backtrace
fatal error: Failed to run xargo
wesley@endurance:~/code/rust/miri>

That looks like your xargo is outdated. What does xargo --version say? Are you sure you are using the latest master of Miri?

Are you using the latest master of miri?

Yes, I just rebased my changes on master (3a5c44c2d83b04651a2e04178087e23e3db6fccc).

That looks like your xargo is outdated.

I don't remember ever installing xargo so I assume ./miri did that?
> xargo --version
xargo 0.3.16
cargo 1.38.0-nightly (677a180f4 2019-07-08)

This is very strange. If I use the latest nightly (2019-08-31), ./miri test works fine. Furthermore, if I re-enable the hashmap tests, they also pass under the latest nightly. So I guess this is fixed?
I'll submit a PR to re-enable the tests and verify it works on the CI.

./miri test does not use the bad opt-level any more. I didn't want to hold up Miri on this issue so I reduced the level we are using for testing.
Try this:
RUST_BACKTRACE=1 ./miri run-debug tests/run-pass/hashmap.rs -Zmir-opt-level=2

Thanks, that repro's on nightly.
I tried running various parts of the ./miri script by hand and I noticed that if I run cargo run --release --bin cargo-miri -- miri setup on my stage2 toolchain, I get this:
    Finished release [optimized] target(s) in 0.07s
     Running `target/release/cargo-miri miri setup`
Installing rust-src component: `rustup component add rust-src`
error: toolchain 'stage2' does not support components
info: caused by: invalid toolchain name: 'stage2'
fatal error: Failed to install rust-src component

Should it be trying to install the rust-src component? Do I need to do something special to get that component to point to my local repo?

You need to set XARGO_RUST_SRC to the src dir of your Rust toolchain. Components don't work with linked toolchains.
Also, the HashMap test indeed takes a somewhat different code path on macOS due to https://github.com/rust-lang/miri/issues/686, which might explain the platform differences.

Hmm... the error looks the same as before even with XARGO_RUST_SRC set:
    Finished release [optimized] target(s) in 28.55s
   Compiling unicode-width v0.1.6
   Compiling proc_macro v0.0.0 (/home/wesley/code/rust/rust/src/libproc_macro)
   Compiling term v0.0.0 (/home/wesley/code/rust/rust/src/libterm)
error[E0463]: can't find crate for `std`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `term`.
warning: build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `std`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `proc_macro`.
warning: build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `core`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `unicode-width`.

To learn more, run the command again with --verbose.
error: `"/home/wesley/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo" "build" "--release" "--manifest-path" "/tmp/xargo.mEQSIPsXjGY4/Cargo.toml" "--target" "x86_64-unknown-linux-gnu" "-p" "test"` failed with exit code: Some(101)
note: run with `RUST_BACKTRACE=1` for a backtrace
fatal error: Failed to run xargo

What exactly did you set XARGO_RUST_SRC to? It should end in src -- it must be the dir containing libstd etc

XARGO_RUST_SRC=~/code/rust/rust/src
wesley@endurance:~/code/rust/miri> ls ~/code/rust/rust/src
bootstrap     liballoc        libpanic_unwind       librustc_ast_borrowck     librustc_errors       librustc_llvm      librustc_passes         librustc_traits  libsyntax      llvm-emscripten  stage0.txt
build_helper  libarena        libproc_macro         librustc_codegen_llvm     librustc_fs_util      librustc_lsan      librustc_plugin         librustc_tsan    libsyntax_ext  llvm-project     stdarch
ci            libcore         libprofiler_builtins  librustc_codegen_ssa      librustc_incremental  librustc_macros    librustc_privacy        librustc_typeck  libsyntax_pos  README.md        test
doc           libfmt_macros   librustc              librustc_codegen_utils    librustc_interface    librustc_metadata  librustc_resolve        librustdoc       libterm        rtstartup        tools
etc           libgraphviz     librustc_apfloat      librustc_data_structures  librustc_lexer        librustc_mir       librustc_save_analysis  libserialize     libtest        rustc
grammar       libpanic_abort  librustc_asan         librustc_driver           librustc_lint         librustc_msan      librustc_target         libstd           libunwind      rustllvm

Very strange.
Depending in your shell, I think ~ might not work when setting env vars. Try giving a fully explicit absolute path?

Very strange.
Depending in your shell, I think ~ might not work when setting env vars. Try giving a fully explicit absolute path?

It doesn't seem to make a difference.
Yeah, I'm kind of stumped. I've tried it on two different computers and two different OS's and I can't seem to get it to work on either. I don't think I have anything especially strange installed or configured on either.

If I use a nightly, then miri compiles fine

And the test suite also passes?

cargo 1.38.0-nightly (677a180f4 2019-07-08)

This looks like a fairly old nightly version of Rust.
But I guess you are using the latest rustc and Miri master for testing the self-compiled builds?

if I use my local copy (with up to date HEAD and xargo), then I get the above errors.

What's your rustc HEAD? Did you try using the latest rustc master for that (so that it is a version I can also try and reproduce this with)?
For the record, here's the not-commented-out part of my config.toml for rustc:
[llvm]
assertions = true
ninja = true
[build]
submodules = false
[rust]
debug-assertions = true
debuginfo-level = 1
incremental = true
deny-warnings = false

Note that you need to turn optimized run pass tests back on in order to see the failure. I just tried it and it still fails on nightly

And the test suite also passes?

With a clean checkout of miri? Yes. If I re-enable the MIR optimizations then the hashmap test fails.

This looks like a fairly old nightly version of Rust.
But I guess you are using the latest rustc and Miri master for testing the self-compiled builds?

I have a checkout of rustc from just after rust-lang/rust#63580 was merged. I'm going to pull and rebase my change right now. I have a build started on top of fdaf594bab31eec75fb6d582cd33e5a5b43de7f4 which is the current HEAD on master of rust-lang/rust. My checkout of miri is at 3a5c44c2d83b04651a2e04178087e23e3db6fccc which is the current master.

The issue here is @wesleywiser can't even run the test suite as xargo fails.

I have a checkout of rustc from just after rust-lang/rust#63580 was merged.

Ah, that might be it. We had to update Miri when https://github.com/rust-lang/rust/pull/63637 landed and that means rustc prior to that don't work any more.

In ~/code/rust/rust I have fdaf594bab31eec75fb6d582cd33e5a5b43de7f4 checked out with a change I'm testing on top of that.

I ran ./x.py build

That completed successfully.
In ~/code/rust/miri I have 3a5c44c2d83b04651a2e04178087e23e3db6fccc checked out with a change to run the tests with optimization on top of that.

I ran rustup override set stage2.
I ran cargo clean.
I ran XARGO_RUST_SRC=/home/wesley/code/rust/rust/src ./miri test

That worked!

So the issue was almost certainly my outdated rustc checkout. Sorry for wasting your time!

My rustc test build to reproduce is just done. I guess I don't need that any more then. ;)
Good to hear it works now. :)

XARGO_RUST_SRC=/home/wesley/code/rust/rust/src ./miri test

In theory, even just ./miri test should work. Could you try that?

In theory, even just ./miri test should work. Could you try that? (Do rm ~/.cache/miri/ -rf to make sure it actually builds the thing and does not use a cached version.)

Yep, that works great!
Perhaps I should add an entry to the "Common Problems" section of the Readme with this info?
Also, I just opened rust-lang/rust#64100 which will fix the regression. I believe I need to wait for that PR to land and then open a PR here with a rustup right?

Perhaps I should add an entry to the "Common Problems" section of the Readme with this info?

This concrete issue is unlikely to still come up in a month or two because old compilers will get used less and less.
Do you have a good idea about where to put the general advice of "use the latest rustc" (or the one in rust-version)? We already say something along those lines but obviously not strongly enough. ;)

I was going to add something like:
error[E0463]: can't find crate for `std`
This may indicate that your local checkout of rust is out of date. Try to update both rust and miri to the latest version with git pull.

This is unlikely to be the error you get with other Miri/rustc version mismatches. And IIRC you also get that error when using a bogus XARGO_RUST_SRC. So I don't think that specifically would still be useful advice in, say, 2 months.

Ah, ok. Feel free to disregard then. :) Given how much back and forth it took to figure that out, I was hoping a warning might prevent this kind of thing in the future.

Final bullet fixed by https://github.com/rust-lang/miri/pull/1133


Question about implementation

I found your library from a link you posted at mudblazor's repo. It is very useful, thanks for making it.
I have a small question. What is the purpose of this:
https://github.com/janeczkoqba/StepperOnMud/blob/4246c477447629717513f311af6d8293278c8dea/StepperOnMud/Shared/Stepper/MyMudStepper.razor#L33
Thanks @janeczkoqba!

Hey, thanks.
It's a quick fix for a problem with updating the MudTabPanel state from a child component. This is how it works without it:

And with it:

I'm not an expert in Blazor and I didn't investigate this problem. For my purpose it was a good enough solution.

Hmh, I assumed it was a problem with my implementation and I didn't check if it is a MudBlazor problem. That's a good idea to check and let them know. I will try to do that (after some investigation), but I don't mind if you will do it :)
Thanks


Audit scanner errors out instead of skipping * policies

The example is safelabels policy - if I leave * defaults in rules audit scanner jobs fail with error.
As a side-effect cluster has many Error pods from failed cronjobs which are not cleaned automatically.
apiVersion: policies.kubewarden.io.v1
kind: ClusterAdmissionPolicy
metadata:
  name: starlabels
  namespace: ''
  annotations:
    artifacthub/pkg: safe-labels/safe-labels/0.1.14
spec:
  backgroundAudit: true
  policyServer: default
  module: ghcr.io/kubewarden/policies/safe-labels:v0.1.14
  rules:
    - apiGroups:
        - '*'
      apiVersions:
        - '*'
      resources:
        - '*'
      operations:
        - CREATE
        - UPDATE
  mutating: false
  namespaceSelector:
    matchExpressions:
      - key: kubernetes.io/metadata.name
        operator: NotIn
        values:
          - calico-system
          - cattle-alerting
          - cattle-fleet-local-system
          - cattle-fleet-system
          - cattle-global-data
          - cattle-global-nt
          - cattle-impersonation-system
          - cattle-istio
          - cattle-logging
          - cattle-pipeline
          - cattle-prometheus
          - cattle-system
          - cert-manager
          - ingress-nginx
          - kube-node-lease
          - kube-public
          - kube-system
          - rancher-operator-system
          - security-scan
          - tigera-operator
    matchLabels: {}
  settings:
    description: null
    constrained_labels: {}
    denied_labels:
      - qqq
    mandatory_labels:
      - aaa
  contextAwareResources: []
  mode: monitor

~ k logs -n cattle-kubewarden-system audit-scanner-28522836-ggpkr
{"level":"info","time":"2024-03-25T12:36:00Z","message":"clusterwide resources scan started"}
{"level":"fatal","error":"no matches for */*, Resource=*","time":"2024-03-25T12:36:00Z","message":"Error on cmd.Execute()"}

I'm seeing this exact behavior as well:
{"level":"info","time":"2024-03-25T12:00:20Z","message":"clusterwide resources scan started"}
{"level":"fatal","error":"no matches for */*, Resource=*","time":"2024-03-25T12:00:20Z","message":"Error on cmd.Execute()"}


Add network interface ordering test

Make sure the interfaces are ordered according IPs in the CNI result.

This change is

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


the plugin is creating node_modules folder

Hi, why the plugin is creating a node_modules ? if this is python plugin not nodejs???

ah, oka.
With "sls plugin install -n serverless-python-requirements" it add me the node_modules folder
To avoid that folder I manage to install the plugin and then add:
custom:
pythonRequirements:
dockerizePip: non-linux
fileName: requirements.txt


Bitscope tools don't seem to work

Describe the bug
None of the Bitscope tools seem to work for me.
https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/applications/science/electronics/bitscope/packages.nix
Steps To Reproduce
Steps to reproduce the behavior:

Run nix-shell -p bitscope.meter
Try to run bitscope.meter (according to https://search.nixos.org/packages)
See that it wasn't found.
Try to run bitscope-meter-2.0.FK22G
Get error message bwrap: Can't chdir to /path/you/was/in: No such file or directory

Expected behavior
The Bitscope Meter UI opens
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
I'm on today's NixOS Unstable
Notify maintainers
@vidbina, @bjornfor
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.38, NixOS, 23.11 (Tapir), 23.11.20230725.ef99fa5`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.15.1`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Hah, thank you! Running it inside a nixpkgs folder really does work, even with nix-shell. Have you tried running it outside a nixpkgs folder?

Weird. It works for me no matter how I run it:
$ cd $HOME
$ /nix/store/qznfhzfbdvw73wly3n4x0xxz1bhf64zv-bitscope-meter-2.0.FK22G/bin/bitscope-meter-2.0.FK22G

$ NIXPKGS_ALLOW_UNFREE=1 NIX_PATH=nixpkgs=$HOME/nixpkgs nix-shell -p bitscope.meter --run bitscope-meter-2.0.FK22G

$ cd nixpkgs
$ NIXPKGS_ALLOW_UNFREE=1 nix-build -A bitscope.meter && ./result/bin/bitscope-meter-2.0.FK22G

I'm on NixOS 23.05 while running the app from nixpkgs-unstable / NixOS 23.11.

That's so weird, I also can use other paths, but not /etc/nixos (the path where I checked out my https://github.com/pbek/nixcfg) nor any sub-paths of that path.
Then I'll get e.g. a:
bwrap: Can't chdir to /etc/nixos: No such file or directory

Ok, that I can reproduce!
Looks like a general issue with buildFHSEnvBubblewrap. Maybe "nixos" should be added here, since it's a common path to have?

Maybe "nixos" should be added here, since it's a common path to have?

I wonder why /etc/PATH entries are listed individually instead of mapping /etc/ itself into the user env.

Ok, that I can reproduce!

Great, thank you!

Looks like a general issue with buildFHSEnvBubblewrap. Maybe "nixos" should be added here, since it's a common path to have?

Is that something I can add to my config, or does that need discussion with the Nix community?
Do you think I should close this issue?

Looks like a general issue with buildFHSEnvBubblewrap. Maybe "nixos" should be added here, since it's a common path to have?

Is that something I can also add to my config, or does that need discussion with the Nix community?

There's probably a way to fixup locally, but I think it's best to patch nixpkgs (and possibly apply the patch locally if you don't want to wait for it to arrive in the channel.)

Do you think I should close this issue?

I don't have strong feelings about it. Technically, you could create a new and more generic issue about buildFHSEnvBubblewrap breaking when run from /etc/nixos, and link this one to it.

Thanks for jumping on this @bjornfor 🙏🏿
@pbek I've cooked these up in what feels like forever ago, haven't actively fiddled with the scopes in a bit to notice and must admit I don't understand buildFHS* well enough, so I'm happy to learn from @bjornfor here.
Not an answer but I've always build stuff somewhere in my home directory ~, perhaps that approach kind of allows you to get by this issue for now (provided, I understood the issue correctly).
TL;DR: Even my nixos-config resides somewhere in ~/src. In my non-Flake days I would provide the args -I nixos="$(MY_NIXPKGS_PATH)/nixos" -I nixpkgs="$(MY_NIXPKGS_PATH)" to nixos-rebuild switch in order to not have to cd. With flakes, I haven't had to think about this at all.
Hope you're set up to probe boards soon-ish.

@bjornfor, thank you for your explanation. I'm not sure if I know enough of if to attempt to open a generic issue where I would explain the issue.
@vidbina, yes, on newer installation I've my nixcfg also in my home directory, it was a coincidence that I was in that directory while testing the tools...

Seems to work again...


POST Object ist generated as Data

Consider this OpenAPI spec.
openapi: 3.0.1
info:
  title: test
  description: ""
  contact:
    name: API Support
    url:
    email:
  version: "1.0"
servers:
- url: https://localhost:8080/
- url: http://localhost:8080/
paths:
  /marketplace/create-payment-intent:
    post:
      tags:
      - Marketplace
      summary: Create Payment Intent
      requestBody:
        description: MerchantItem Ids
        content:
          '*/*':
            schema:
              type: array
              items:
                type: integer
        required: true
      responses:
        200:
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/PaymentIntent'
      security:
      - Authorization: []
      x-codegen-request-body-name: body
components:
  schemas:
    PaymentIntent:
      required:
      - clientSecret
      type: object
      properties:
        clientSecret:
          type: string
  securitySchemes:
    Authorization:
      type: apiKey
      name: Authorization
      in: header

Generating the API with Create API created following signature.
extension Paths {
    public static var createPaymentIntent: CreatePaymentIntent {
        CreatePaymentIntent(path: "/marketplace/create-payment-intent")
    }

    public struct CreatePaymentIntent {
        /// Path: `/marketplace/create-payment-intent`
        public let path: String

        /// Create Payment Intent
        public func post(_ body: Data) -> Request<PaymentIntent> {
            .post(path, body: body)
        }
    }
}

You can see that signature wants a Data object instead of [Int].

hey,
if you set the content type of your request to 'application/json': then it works:

      requestBody:
        description: MerchantItem Ids
        content:
          'application/json':
            schema:
              type: array
              items:
                type: integer
        required: true

Result:
import Foundation
import Get

public extension Paths {
    static var createPaymentIntent: CreatePaymentIntent {
        CreatePaymentIntent(path: "/marketplace/create-payment-intent")
    }

    struct CreatePaymentIntent {
        /// Path: `/marketplace/create-payment-intent`
        public let path: String

        public func post(_ body: [Int]) -> Request<PaymentIntent> {
            .post(path, body: body)
        }
    }
}

Thanks it worked! Btw long time no see @JanC :-)


firewall_rules_edit: Improve help wording

The original one had too much word repetition, so I think I have improved the text.
Backport to RELENG_2_3 if accepted.
Thanks!

I committed a version with different text yet. This one still had some wording that wasn't reading quite right.


Color Distortion after Chrome OS Update

$sudo enter-chroot -n xenial croutonversion
Entering /mnt/stateful_partition/crouton/chroots/xenial...
crouton: version 1-20180718130616~master:5d4c3014
release: xenial
architecture: amd64
xmethod: xorg
targets: gnome-desktop
host: version 10895.56.0 (Official Build) stable-channel eve
kernel: Linux localhost 4.4.141-14563-g78fd9f781aae #1 SMP PREEMPT Thu Sep 13 00:00:24 PDT 2018 x86_64 x86_64 x86_64 GNU/Linux
freon: yes

Please describe your issue:
I'm using a Pixelbook with a single xenial chroot installed with the gnome desktop.  I recently updated to Chrome OS Version 69.0.3497.95 (Official Build) (64-bit) and this issue presented itself immediate after the update.  I have updated my chroot (using the instructions here).  There were no errors in the update (as far as I could tell) but it did not resolve the issue.
If I launch the gnome desktop for my chroot (sudo startgnome) and then return to Chrome OS (using either the keyboard shortcut or by logging out) the colors are distorted.  It is almost like the colors are inverted but they are also very blurry and the display will flicker back and forth between nearly normal colors and some different distorted colors.  If I return to the chroot everything looks fine on the gnome desktop but if I then return to Chrome OS things are still distorted.  Restarting my laptop returns everything to normal but the problem returns when I launch the gnome desktop.  I don't have any problems if I enter the chroot with sudo enter-chroot.
If I take a screenshot while the screen is distorted, the image does not capture the distortion.  That is, after I restart, the screenshot that I took of the distortion displays as a normal image with normal colors.  Here is a link to a photograph of the problem: https://i.imgur.com/1t7DF9m.jpg  If it helps, I could post a video of myself reproducing it as there is some flickering of the colors that isn't captured in a photo.
If known, describe the steps to reproduce the issue:
From a fresh restart I do the following:

Launch my chroot from a shell with sudo startgnome.
Return to the Chrome OS desktop either by logging out of the gnome desktop or by using ctrl+alt+shift+back.

At this point, the problem presents itself.  The colors are warped and the text is distorted and unreadable.

It has been recommended to me that I disable the night light flag in chrome://flags.  I did try that but unfortunately the problem persists.

I do have a workaround.  When the problem occurs, anything that refreshes the screen seems to clear it up the distortion momentarily (though it comes back again when you switch to gnome again).  I have found four ways (so far) to do that:

Switch to tablet mode and back.
Change the resolution.
Rotate the screen four times (ctrl+shift+refresh).
Toggle night light mode on or off (with the flag enabled).

@danraies,
I'm not sure crouton is the culprit here, when it next happens you might want to file a report using the key-combo alt+shift+i or file a crbug here.
And perhaps close this issue if you agree.
-DennisLfromGA

i have the same issue and ive tried all the suggested solutions and none of them have worked

I have the same problem
The solution is enable night light (the real one) and turn back off.

Issue persists across multiple (all?) Chromebooks until this day, on Chrome OS 74.
Any news on a potential fix?


Data transforms error handling

Hi, I'm having some trouble handling errors caused on data transform functions.
json2csv: 5.0.7
Node v16.13.2
When you run this code
const { AsyncParser } = require('json2csv');
const { Readable } = require('stream');

const mockData = [
  {
    id: 1,
    name: 'John',
    surname: 'Doe',
  },
  {
    id: 2,
    name: 'Jane',
    surname: 'Doe',
  }
];

const mockDataStream = Readable.from(mockData);

const dataTransform = (data) => {
  if (data.name === 'Jane') throw new Error('Unhandled error');
  return data;
};

const asyncParser = new AsyncParser({ transforms: [dataTransform] }, { objectMode: true });

try {
  const parsingProcessor = asyncParser.fromInput(mockDataStream).toOutput(process.stdout);
  parsingProcessor.promise(false).catch(err => {
    console.log('Something happened');
  });
} catch (error) {
  console.log('Hello?');
}

You cannot handle the exception, it'll be an unhandledException one.

Only approach that works is to put a try catch inside the data transform function like this.
const dataTransform = (data) => {
  try {
    if (data.name === 'Jane') throw new Error('Unhandled error');
    return data;
  } catch (error) {
    return undefined;
  }
};

I think this could be solved if you directly use JSON2CSVTransform class and modifies the pushLine method with a try catch like this. But I think this won't solve the problem for the classes use it internally.
  pushLine(data) {
    try {
      const processedData = this.preprocessRow(data);

      if (!this._hasWritten) {
        this.opts.fields = this.opts.fields || this.preprocessFieldsInfo(Object.keys(processedData[0]));
        this.pushHeader();
      }

      processedData.forEach(row => {
        const line = this.processRow(row, this.opts);
        if (line === undefined) return;
        this.emit('line', line);
        this.push(this._hasWritten ? this.opts.eol + line : line);
        this._hasWritten = true;
      });
    } catch (error) {
      this.emit('error', error);
    }
  }

This does not happen in v6 which was just published as a separate package: https://www.npmjs.com/package/@json2csv/node
Although, the API has changed to make the async api more similar than the other.
Also, when piping to an output stream the promise method has been removed since it's trivial to wait for the end event and node nowadays  even provides the pipeline method for that.
In v6 you can do:
const { pipeline } = require('node:stream/promises');

// ...

try {
  pipeline(asyncParser.parse(mockDataStream), process.stdout).catch(err => {
    console.log('Something happened');
  });
} catch (error) {
  console.log('Hello?');
}

or using the transform
const json2csvTransform = new Transform({ transforms: [dataTransform] }, { objectMode: true });

try {
  pipeline(mockDataStream, json2csvTransform, process.stdout).catch(err => {
    console.log('Something happened');
  });
} catch (error) {
  console.log('Hello?');
}

I haven't tested the code. It's just a quick guidelines.
If you have more issues with it I can look deeper into it.

Thank you @juanjoDiaz I didn't know that package existed, is this repo "deprecated" then?
I think I'll just migrate all the code from v5 to the new v6.

Yup. I'd consider it deprecated.
And I think that you'd do well moving to v6 🙂


Application crashes when attempting to show folder properties in system partition

Describe the bug
Files crashes when trying to get folder property in system partition.
To Reproduce
Steps to reproduce the behavior:

Go to application
Click open system partition
Try to get any folder properties
Application crashes inmediately.

Expected behavior
To show the property of the selected folder.
Desktop (please complete the following information):

OS Version: [e.g. Windows 10 20H2 19042.685 Windows Feature Experience Pack 120.2212.551.0]
App version: [v<IP_ADDRESS>]

Log file
debug.txt

Duplicate of #2537.


corpus ribes scorer throws division by zero error

Division by zero error is constantly thrown for corpus_ribes() function.
To replicate:
Take the example from the docs (https://www.nltk.org/api/nltk.translate.html) and shorten some
of the input sentences. In this case I only shortened hyp2 and hyp2a.
from nltk.translate.ribes_score import corpus_ribes

hyp1 = ['It', 'is', 'a', 'guide', 'to', 'action', 'which', 'ensures', 'that', 'the', 'military', 'always', 'obeys', 'the', 'commands', 'of', 'the', 'party']
ref1a = ['It', 'is', 'a', 'guide', 'to', 'action', 'that',  'ensures', 'that', 'the', 'military', 'will', 'forever',  'heed', 'Party', 'commands']
ref1b = ['It', 'is', 'the', 'guiding', 'principle', 'which', 'guarantees', 'the', 'military', 'forces', 'always', 'being', 'under', 'the', 'command', 'of', 'the', 'Party']
ref1c = ['It', 'is', 'the', 'practical', 'guide', 'for', 'the',  'army', 'always', 'to', 'heed', 'the', 'directions',  'of', 'the', 'party']

hyp2 = ['he', 'read', 'the']
ref2a = ['he', 'was', 'interested', 'in', 'world', 'history', 'because', 'he']

list_of_references = [[ref1a, ref1b, ref1c], [ref2a]]
hypotheses = [hyp1, hyp2]
corpus_ribes(list_of_references, hypotheses)

Output:
--------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-175-b02599af69e7> in <module>
      1 list_of_references = [[ref1a, ref1b, ref1c], [ref2a]]
      2 hypotheses = [hyp1, hyp2]
----> 3 corpus_ribes(list_of_references, hypotheses)

/anaconda3/envs/torch/lib/python3.7/site-packages/nltk/translate/ribes_score.py in corpus_ribes(list_of_references, hypotheses, alpha, beta)
    117     # Iterate through each hypothesis and their corresponding references.
    118     for references, hypothesis in zip(list_of_references, hypotheses):
--> 119         corpus_best_ribes += sentence_ribes(references, hypothesis, alpha, beta)
    120     return corpus_best_ribes / len(hypotheses)
    121

/anaconda3/envs/torch/lib/python3.7/site-packages/nltk/translate/ribes_score.py in sentence_ribes(references, hypothesis, alpha, beta)
     53         # Collects the *worder* from the ranked correlation alignments.
     54         worder = word_rank_alignment(reference, hypothesis)
---> 55         nkt = kendall_tau(worder)
     56
     57         # Calculates the brevity penalty

/anaconda3/envs/torch/lib/python3.7/site-packages/nltk/translate/ribes_score.py in kendall_tau(worder, normalize)
    288     num_possible_pairs = choose(worder_len, 2)
    289     # Kendall's Tau computation.
--> 290     tau = 2 * num_increasing_pairs / num_possible_pairs - 1
    291     if normalize:  # If normalized, the tau output falls between 0.0 to 1.0
    292         return (tau + 1) / 2

ZeroDivisionError: division by zero

Hi @tibnb545, thank you for your report.
I believe this is a duplicate of #2433.

`# -- coding: utf-8 --
Natural Language Toolkit: RIBES Score

Copyright (C) 2001-2020 NLTK Project
Contributors: Katsuhito Sudoh, Liling Tan, Kasramvd, J.F.Sebastian
Mark Byers, ekhumoro, P. Ortiz
URL: http://nltk.org/
For license information, see LICENSE.TXT
""" RIBES score implementation """
from itertools import islice
import math
from nltk.util import ngrams, choose
def sentence_ribes(references, hypothesis, alpha=0.25, beta=0.10):
"""
The RIBES (Rank-based Intuitive Bilingual Evaluation Score) from
Hideki Isozaki, Tsutomu Hirao, Kevin Duh, Katsuhito Sudoh and
Hajime Tsukada. 2010. "Automatic Evaluation of Translation Quality for
Distant Language Pairs". In Proceedings of EMNLP.
http://www.aclweb.org/anthology/D/D10/D10-1092.pdf
The generic RIBES scores used in shared task, e.g. Workshop for
Asian Translation (WAT) uses the following RIBES calculations:
RIBES = kendall_tau * (alphap1) * (betabp)
Please note that this re-implementation differs from the official
RIBES implementation and though it emulates the results as describe
in the original paper, there are further optimization implemented
in the official RIBES script.
Users are encouraged to use the official RIBES script instead of this
implementation when evaluating your machine translation system. Refer
to http://www.kecl.ntt.co.jp/icl/lirg/ribes/ for the official script.
:param references: a list of reference sentences
:type reference: list(list(str))
:param hypothesis: a hypothesis sentence
:type hypothesis: list(str)
:param alpha: hyperparameter used as a prior for the unigram precision.
:type alpha: float
:param beta: hyperparameter used as a prior for the brevity penalty.
:type beta: float
:return: The best ribes score from one of the references.
:rtype: float
"""
best_ribes = -1.0
# Calculates RIBES for each reference and returns the best score.
    # Collects the *worder* from the ranked correlation alignments.

worder = word_rank_alignment(str(reference), str(hypothesis))
nkt = kendall_tau(worder)

    # Calculates the brevity penalty
bp = min(1.0, math.exp(1.0 - len(reference) / len(hypothesis)))

    # Calculates the unigram precision, *p1*
p1 = len(worder) / len(hypothesis)

_ribes = nkt * (p1 ** alpha) * (bp ** beta)

if _ribes > best_ribes:
    best_ribes = _ribes

return best_ribes

def corpus_ribes(list_of_references, hypotheses, alpha=0.25, beta=0.10):
"""
This function "calculates RIBES for a system output (hypothesis) with
multiple references, and returns "best" score among multi-references and
individual scores. The scores are corpus-wise, i.e., averaged by the number
of sentences." (c.f. RIBES version 1.03.1 code).
Different from BLEU's micro-average precision, RIBES calculates the
macro-average precision by averaging the best RIBES score for each pair of
hypothesis and its corresponding references
>>> hyp1 = ['It', 'is', 'a', 'guide', 'to', 'action', 'which',
...         'ensures', 'that', 'the', 'military', 'always',
...         'obeys', 'the', 'commands', 'of', 'the', 'party']
>>> ref1a = ['It', 'is', 'a', 'guide', 'to', 'action', 'that',
...          'ensures', 'that', 'the', 'military', 'will', 'forever',
...          'heed', 'Party', 'commands']
>>> ref1b = ['It', 'is', 'the', 'guiding', 'principle', 'which',
...          'guarantees', 'the', 'military', 'forces', 'always',
...          'being', 'under', 'the', 'command', 'of', 'the', 'Party']
>>> ref1c = ['It', 'is', 'the', 'practical', 'guide', 'for', 'the',
...          'army', 'always', 'to', 'heed', 'the', 'directions',
...          'of', 'the', 'party']
>>> hyp2 = ['he', 'read', 'the', 'book', 'because', 'he', 'was',
...         'interested', 'in', 'world', 'history']
>>> ref2a = ['he', 'was', 'interested', 'in', 'world', 'history',
...          'because', 'he', 'read', 'the', 'book']
>>> list_of_references = [[ref1a, ref1b, ref1c], [ref2a]]
>>> hypotheses = [hyp1, hyp2]
>>> round(corpus_ribes(list_of_references, hypotheses),4)
0.3597
:param references: a corpus of lists of reference sentences, w.r.t. hypotheses
:type references: list(list(list(str)))
:param hypotheses: a list of hypothesis sentences
:type hypotheses: list(list(str))
:param alpha: hyperparameter used as a prior for the unigram precision.
:type alpha: float
:param beta: hyperparameter used as a prior for the brevity penalty.
:type beta: float
:return: The best ribes score from one of the references.
:rtype: float
"""
c=0
for i in range(len(list_of_references)):
for j in range(len(list_of_references[i])):
print(type(list_of_references[i][j].split()))
c+=sentence_ribes(list_of_references[i][j], hypotheses[i])
print(c/len(hypotheses))
def position_of_ngram(ngram, sentence):
"""
This function returns the position of the first instance of the ngram
appearing in a sentence.
Note that one could also use string as follows but the code is a little
convoluted with type casting back and forth:
char_pos = ' '.join(sent)[:' '.join(sent).index(' '.join(ngram))]
word_pos = char_pos.count(' ')
Another way to conceive this is:
return next(i for i, ng in enumerate(ngrams(sentence, len(ngram)))
if ng == ngram)
:param ngram: The ngram that needs to be searched
:type ngram: tuple
:param sentence: The list of tokens to search from.
:type sentence: list(str)
"""
# Iterates through the ngrams in sentence.
for i, sublist in enumerate(ngrams(sentence, len(ngram))):
# Returns the index of the word when ngram matches.
if ngram == sublist:
return i
def word_rank_alignment(reference, hypothesis, character_based=False):
"""
This is the word rank alignment algorithm described in the paper to produce
the worder list, i.e. a list of word indices of the hypothesis word orders
w.r.t. the list of reference words.
Below is (H0, R0) example from the Isozaki et al. 2010 paper,
note the examples are indexed from 1 but the results here are indexed from 0:
>>> ref = str('he was interested in world history because he '
... 'read the book').split()
>>> hyp = str('he read the book because he was interested in world '
... 'history').split()
>>> word_rank_alignment(ref, hyp)
[7, 8, 9, 10, 6, 0, 1, 2, 3, 4, 5]
The (H1, R1) example from the paper, note the 0th index:
>>> ref = 'John hit Bob yesterday'.split()
>>> hyp = 'Bob hit John yesterday'.split()
>>> word_rank_alignment(ref, hyp)
[2, 1, 0, 3]
Here is the (H2, R2) example from the paper, note the 0th index here too:
>>> ref = 'the boy read the book'.split()
>>> hyp = 'the book was read by the boy'.split()
>>> word_rank_alignment(ref, hyp)
[3, 4, 2, 0, 1]
:param reference: a reference sentence
:type reference: list(str)
:param hypothesis: a hypothesis sentence
:type hypothesis: list(str)
"""
worder=[]
hypothesis=hypothesis.split()
reference=reference.split()
hyp_len = len(hypothesis)
# Stores a list of possible ngrams from the reference sentence.
# This is used for matching context window later in the algorithm.
ref_ngrams = []
hyp_ngrams = []
for n in range(1, len(reference) + 1):
for ng in ngrams(reference, n):
ref_ngrams.append(ng)
for ng in ngrams(hypothesis, n):
hyp_ngrams.append(ng)
for i, h_word in enumerate(hypothesis):
    # If word is not in the reference, continue.
    if h_word not in reference:
        continue
    # If we can determine one-to-one word correspondence for unigrams that
    # only appear once in both the reference and hypothesis.
    elif hypothesis.count(h_word) == reference.count(h_word) == 1:
        worder.append(reference.index(h_word))

    else:
        max_window_size = max(i, hyp_len - i + 1)
        for window in range(1, max_window_size):
            if i + window < hyp_len:  # If searching the right context is possible.
                # Retrieve the right context window.
                right_context_ngram = tuple(islice(hypothesis, i, i + window + 1))
                num_times_in_ref = ref_ngrams.count(right_context_ngram)
                num_times_in_hyp = hyp_ngrams.count(right_context_ngram)
                # If ngram appears only once in both ref and hyp.
                if num_times_in_ref == num_times_in_hyp == 1:
                    # Find the position of ngram that matched the reference.
                    pos = position_of_ngram(right_context_ngram, reference)
                    worder.append(pos)  # Add the positions of the ngram.
                    break
            if window <= i:  # If searching the left context is possible.
                # Retrieve the left context window.
                left_context_ngram = tuple(islice(hypothesis, i - window, i + 1))
                num_times_in_ref = ref_ngrams.count(left_context_ngram)
                num_times_in_hyp = hyp_ngrams.count(left_context_ngram)
                if num_times_in_ref == num_times_in_hyp == 1:
                    # Find the position of ngram that matched the reference.
                    pos = position_of_ngram(left_context_ngram, reference)
                    # Add the positions of the ngram.
                    worder.append(pos + len(left_context_ngram) - 1)
                    break
return worder

def find_increasing_sequences(worder):
"""
Given the worder list, this function groups monotonic +1 sequences.
>>> worder = [7, 8, 9, 10, 6, 0, 1, 2, 3, 4, 5]
>>> list(find_increasing_sequences(worder))
[(7, 8, 9, 10), (0, 1, 2, 3, 4, 5)]
:param worder: The worder list output from word_rank_alignment
:param type: list(int)
"""
items = iter(worder)
a, b = None, next(items, None)
result = [b]
while b is not None:
a, b = b, next(items, None)
if b is not None and a + 1 == b:
result.append(b)
else:
if len(result) > 1:
yield tuple(result)
result = [b]
def kendall_tau(worder, normalize=True):
"""
Calculates the Kendall's Tau correlation coefficient given the worder
list of word alignments from word_rank_alignment(), using the formula:
tau = 2 * num_increasing_pairs / num_possible pairs -1
Note that the no. of increasing pairs can be discontinuous in the worder
list and each each increasing sequence can be tabulated as choose(len(seq), 2)
no. of increasing pairs, e.g.
>>> worder = [7, 8, 9, 10, 6, 0, 1, 2, 3, 4, 5]
>>> number_possible_pairs = choose(len(worder), 2)
>>> round(kendall_tau(worder, normalize=False),3)
-0.236
>>> round(kendall_tau(worder),3)
0.382
:param worder: The worder list output from word_rank_alignment
:type worder: list(int)
:param normalize: Flag to indicate normalization
:type normalize: boolean
:return: The Kendall's Tau correlation coefficient.
:rtype: float
"""
worder_len = len(worder)
# Extract the groups of increasing/monotonic sequences.
increasing_sequences = find_increasing_sequences(worder)
# Calculate no. of increasing_pairs in *worder* list.
num_increasing_pairs = sum(choose(len(seq), 2) for seq in increasing_sequences)
# Calculate no. of possible pairs.
num_possible_pairs = choose(worder_len, 2)
# Kendall's Tau computation.
tau = 2 * num_increasing_pairs / (num_possible_pairs) - 1
if normalize:  # If normalized, the tau output falls between 0.0 to 1.0
    return (tau + 1) / 2
else:  # Otherwise, the tau outputs falls between -1.0 to +1.0
    return tau

def spearman_rho(worder, normalize=True):
"""
Calculates the Spearman's Rho correlation coefficient given the worder
list of word alignment from word_rank_alignment(), using the formula:
rho = 1 - sum(d**2) / choose(len(worder)+1, 3)
Given that d is the sum of difference between the worder list of indices
and the original word indices from the reference sentence.
Using the (H0,R0) and (H5, R5) example from the paper
>>> worder =  [7, 8, 9, 10, 6, 0, 1, 2, 3, 4, 5]
>>> round(spearman_rho(worder, normalize=False), 3)
-0.591
>>> round(spearman_rho(worder), 3)
0.205
:param worder: The worder list output from word_rank_alignment
:param type: list(int)
"""
worder_len = len(worder)
sum_d_square = sum((wi - i) ** 2 for wi, i in zip(worder, range(worder_len)))
rho = 1 - sum_d_square / choose(worder_len + 1, 3)
if normalize:  # If normalized, the rho output falls between 0.0 to 1.0
    return (rho + 1) / 2
else:  # Otherwise, the rho outputs falls between -1.0 to +1.0
    return rho`

Initially it was giving "division by zero error" for both sentence_ribes and corpus_ribes.
I made some changes in the  code it works fine when I use sentence_ribes,but outputs same error "division by zero error" when corpus_ribes  is used.
Can someone please point out my mistake.

# -*- coding: utf-8 -*-
# Natural Language Toolkit: RIBES Score
#
# Copyright (C) 2001-2020 NLTK Project
# Contributors: Katsuhito Sudoh, Liling Tan, Kasramvd, J.F.Sebastian
#               Mark Byers, ekhumoro, P. Ortiz
# URL: <http://nltk.org/>
# For license information, see LICENSE.TXT
""" RIBES score implementation """

from itertools import islice
import math

from nltk.util import ngrams, choose

def sentence_ribes(references, hypothesis, alpha=0.25, beta=0.10):
    """
    The RIBES (Rank-based Intuitive Bilingual Evaluation Score) from
    Hideki Isozaki, Tsutomu Hirao, Kevin Duh, Katsuhito Sudoh and
    Hajime Tsukada. 2010. "Automatic Evaluation of Translation Quality for
    Distant Language Pairs". In Proceedings of EMNLP.
    http://www.aclweb.org/anthology/D/D10/D10-1092.pdf
    The generic RIBES scores used in shared task, e.g. Workshop for
    Asian Translation (WAT) uses the following RIBES calculations:
        RIBES = kendall_tau * (alpha**p1) * (beta**bp)
    Please note that this re-implementation differs from the official
    RIBES implementation and though it emulates the results as describe
    in the original paper, there are further optimization implemented
    in the official RIBES script.
    Users are encouraged to use the official RIBES script instead of this
    implementation when evaluating your machine translation system. Refer
    to http://www.kecl.ntt.co.jp/icl/lirg/ribes/ for the official script.
    :param references: a list of reference sentences
    :type reference: list(list(str))
    :param hypothesis: a hypothesis sentence
    :type hypothesis: list(str)
    :param alpha: hyperparameter used as a prior for the unigram precision.
    :type alpha: float
    :param beta: hyperparameter used as a prior for the brevity penalty.
    :type beta: float
    :return: The best ribes score from one of the references.
    :rtype: float
    """
    best_ribes = -1.0
    # Calculates RIBES for each reference and returns the best score.

        # Collects the *worder* from the ranked correlation alignments.

    worder = word_rank_alignment(str(reference), str(hypothesis))
    nkt = kendall_tau(worder)

        # Calculates the brevity penalty
    bp = min(1.0, math.exp(1.0 - len(reference) / len(hypothesis)))

        # Calculates the unigram precision, *p1*
    p1 = len(worder) / len(hypothesis)

    _ribes = nkt * (p1 ** alpha) * (bp ** beta)

    if _ribes > best_ribes:
        best_ribes = _ribes

    return best_ribes

def corpus_ribes(list_of_references, hypotheses, alpha=0.25, beta=0.10):
    """
    This function "calculates RIBES for a system output (hypothesis) with
    multiple references, and returns "best" score among multi-references and
    individual scores. The scores are corpus-wise, i.e., averaged by the number
    of sentences." (c.f. RIBES version 1.03.1 code).
    Different from BLEU's micro-average precision, RIBES calculates the
    macro-average precision by averaging the best RIBES score for each pair of
    hypothesis and its corresponding references
    >>> hyp1 = ['It', 'is', 'a', 'guide', 'to', 'action', 'which',
    ...         'ensures', 'that', 'the', 'military', 'always',
    ...         'obeys', 'the', 'commands', 'of', 'the', 'party']
    >>> ref1a = ['It', 'is', 'a', 'guide', 'to', 'action', 'that',
    ...          'ensures', 'that', 'the', 'military', 'will', 'forever',
    ...          'heed', 'Party', 'commands']
    >>> ref1b = ['It', 'is', 'the', 'guiding', 'principle', 'which',
    ...          'guarantees', 'the', 'military', 'forces', 'always',
    ...          'being', 'under', 'the', 'command', 'of', 'the', 'Party']
    >>> ref1c = ['It', 'is', 'the', 'practical', 'guide', 'for', 'the',
    ...          'army', 'always', 'to', 'heed', 'the', 'directions',
    ...          'of', 'the', 'party']
    >>> hyp2 = ['he', 'read', 'the', 'book', 'because', 'he', 'was',
    ...         'interested', 'in', 'world', 'history']
    >>> ref2a = ['he', 'was', 'interested', 'in', 'world', 'history',
    ...          'because', 'he', 'read', 'the', 'book']
    >>> list_of_references = [[ref1a, ref1b, ref1c], [ref2a]]
    >>> hypotheses = [hyp1, hyp2]
    >>> round(corpus_ribes(list_of_references, hypotheses),4)
    0.3597
    :param references: a corpus of lists of reference sentences, w.r.t. hypotheses
    :type references: list(list(list(str)))
    :param hypotheses: a list of hypothesis sentences
    :type hypotheses: list(list(str))
    :param alpha: hyperparameter used as a prior for the unigram precision.
    :type alpha: float
    :param beta: hyperparameter used as a prior for the brevity penalty.
    :type beta: float
    :return: The best ribes score from one of the references.
    :rtype: float
    """
    c=0
    for i in range(len(list_of_references)):
        for j in range(len(list_of_references[i])):
            print(type(list_of_references[i][j].split()))
            c+=sentence_ribes(list_of_references[i][j], hypotheses[i])
    print(c/len(hypotheses))

def position_of_ngram(ngram, sentence):
    """
    This function returns the position of the first instance of the ngram
    appearing in a sentence.
    Note that one could also use string as follows but the code is a little
    convoluted with type casting back and forth:
        char_pos = ' '.join(sent)[:' '.join(sent).index(' '.join(ngram))]
        word_pos = char_pos.count(' ')
    Another way to conceive this is:
        return next(i for i, ng in enumerate(ngrams(sentence, len(ngram)))
                    if ng == ngram)
    :param ngram: The ngram that needs to be searched
    :type ngram: tuple
    :param sentence: The list of tokens to search from.
    :type sentence: list(str)
    """
    # Iterates through the ngrams in sentence.
    for i, sublist in enumerate(ngrams(sentence, len(ngram))):
        # Returns the index of the word when ngram matches.
        if ngram == sublist:
            return i

def word_rank_alignment(reference, hypothesis, character_based=False):
    """
    This is the word rank alignment algorithm described in the paper to produce
    the *worder* list, i.e. a list of word indices of the hypothesis word orders
    w.r.t. the list of reference words.
    Below is (H0, R0) example from the Isozaki et al. 2010 paper,
    note the examples are indexed from 1 but the results here are indexed from 0:
        >>> ref = str('he was interested in world history because he '
        ... 'read the book').split()
        >>> hyp = str('he read the book because he was interested in world '
        ... 'history').split()
        >>> word_rank_alignment(ref, hyp)
        [7, 8, 9, 10, 6, 0, 1, 2, 3, 4, 5]
    The (H1, R1) example from the paper, note the 0th index:
        >>> ref = 'John hit Bob yesterday'.split()
        >>> hyp = 'Bob hit John yesterday'.split()
        >>> word_rank_alignment(ref, hyp)
        [2, 1, 0, 3]
    Here is the (H2, R2) example from the paper, note the 0th index here too:
        >>> ref = 'the boy read the book'.split()
        >>> hyp = 'the book was read by the boy'.split()
        >>> word_rank_alignment(ref, hyp)
        [3, 4, 2, 0, 1]
    :param reference: a reference sentence
    :type reference: list(str)
    :param hypothesis: a hypothesis sentence
    :type hypothesis: list(str)
    """
    worder=[]
    hypothesis=hypothesis.split()
    reference=reference.split()
    hyp_len = len(hypothesis)
    # Stores a list of possible ngrams from the reference sentence.
    # This is used for matching context window later in the algorithm.
    ref_ngrams = []
    hyp_ngrams = []
    for n in range(1, len(reference) + 1):
        for ng in ngrams(reference, n):
            ref_ngrams.append(ng)
        for ng in ngrams(hypothesis, n):
            hyp_ngrams.append(ng)
    for i, h_word in enumerate(hypothesis):

        # If word is not in the reference, continue.
        if h_word not in reference:
            continue
        # If we can determine one-to-one word correspondence for unigrams that
        # only appear once in both the reference and hypothesis.
        elif hypothesis.count(h_word) == reference.count(h_word) == 1:
            worder.append(reference.index(h_word))

        else:
            max_window_size = max(i, hyp_len - i + 1)
            for window in range(1, max_window_size):
                if i + window < hyp_len:  # If searching the right context is possible.
                    # Retrieve the right context window.
                    right_context_ngram = tuple(islice(hypothesis, i, i + window + 1))
                    num_times_in_ref = ref_ngrams.count(right_context_ngram)
                    num_times_in_hyp = hyp_ngrams.count(right_context_ngram)
                    # If ngram appears only once in both ref and hyp.
                    if num_times_in_ref == num_times_in_hyp == 1:
                        # Find the position of ngram that matched the reference.
                        pos = position_of_ngram(right_context_ngram, reference)
                        worder.append(pos)  # Add the positions of the ngram.
                        break
                if window <= i:  # If searching the left context is possible.
                    # Retrieve the left context window.
                    left_context_ngram = tuple(islice(hypothesis, i - window, i + 1))
                    num_times_in_ref = ref_ngrams.count(left_context_ngram)
                    num_times_in_hyp = hyp_ngrams.count(left_context_ngram)
                    if num_times_in_ref == num_times_in_hyp == 1:
                        # Find the position of ngram that matched the reference.
                        pos = position_of_ngram(left_context_ngram, reference)
                        # Add the positions of the ngram.
                        worder.append(pos + len(left_context_ngram) - 1)
                        break
    return worder

def find_increasing_sequences(worder):
    """
    Given the *worder* list, this function groups monotonic +1 sequences.
        >>> worder = [7, 8, 9, 10, 6, 0, 1, 2, 3, 4, 5]
        >>> list(find_increasing_sequences(worder))
        [(7, 8, 9, 10), (0, 1, 2, 3, 4, 5)]
    :param worder: The worder list output from word_rank_alignment
    :param type: list(int)
    """
    items = iter(worder)
    a, b = None, next(items, None)
    result = [b]
    while b is not None:
        a, b = b, next(items, None)
        if b is not None and a + 1 == b:
            result.append(b)
        else:
            if len(result) > 1:
                yield tuple(result)
            result = [b]

def kendall_tau(worder, normalize=True):
    """
    Calculates the Kendall's Tau correlation coefficient given the *worder*
    list of word alignments from word_rank_alignment(), using the formula:
        tau = 2 * num_increasing_pairs / num_possible pairs -1
    Note that the no. of increasing pairs can be discontinuous in the *worder*
    list and each each increasing sequence can be tabulated as choose(len(seq), 2)
    no. of increasing pairs, e.g.
        >>> worder = [7, 8, 9, 10, 6, 0, 1, 2, 3, 4, 5]
        >>> number_possible_pairs = choose(len(worder), 2)
        >>> round(kendall_tau(worder, normalize=False),3)
        -0.236
        >>> round(kendall_tau(worder),3)
        0.382
    :param worder: The worder list output from word_rank_alignment
    :type worder: list(int)
    :param normalize: Flag to indicate normalization
    :type normalize: boolean
    :return: The Kendall's Tau correlation coefficient.
    :rtype: float
    """

    worder_len = len(worder)
    # Extract the groups of increasing/monotonic sequences.
    increasing_sequences = find_increasing_sequences(worder)
    # Calculate no. of increasing_pairs in *worder* list.
    num_increasing_pairs = sum(choose(len(seq), 2) for seq in increasing_sequences)
    # Calculate no. of possible pairs.
    num_possible_pairs = choose(worder_len, 2)
    # Kendall's Tau computation.
    tau = 2 * num_increasing_pairs / (num_possible_pairs) - 1
    if normalize:  # If normalized, the tau output falls between 0.0 to 1.0
        return (tau + 1) / 2
    else:  # Otherwise, the tau outputs falls between -1.0 to +1.0
        return tau

def spearman_rho(worder, normalize=True):
    """
    Calculates the Spearman's Rho correlation coefficient given the *worder*
    list of word alignment from word_rank_alignment(), using the formula:
        rho = 1 - sum(d**2) / choose(len(worder)+1, 3)
    Given that d is the sum of difference between the *worder* list of indices
    and the original word indices from the reference sentence.
    Using the (H0,R0) and (H5, R5) example from the paper
        >>> worder =  [7, 8, 9, 10, 6, 0, 1, 2, 3, 4, 5]
        >>> round(spearman_rho(worder, normalize=False), 3)
        -0.591
        >>> round(spearman_rho(worder), 3)
        0.205
    :param worder: The worder list output from word_rank_alignment
    :param type: list(int)
    """
    worder_len = len(worder)
    sum_d_square = sum((wi - i) ** 2 for wi, i in zip(worder, range(worder_len)))
    rho = 1 - sum_d_square / choose(worder_len + 1, 3)

    if normalize:  # If normalized, the rho output falls between 0.0 to 1.0
        return (rho + 1) / 2
    else:  # Otherwise, the rho outputs falls between -1.0 to +1.0
        return rho

Initially it was giving "division by zero error" for both sentence_ribes and corpus_ribes.
I made some changes in the code of nltk implementation of ribes_score it works fine when I use sentence_ribes,but outputs same error "division by zero error" when corpus_ribes is used.
Can someone please point out my mistake.

Division by zero error is constantly thrown for corpus_ribes() function.
To replicate:
Take the example from the docs (https://www.nltk.org/api/nltk.translate.html) and shorten some
of the input sentences. In this case I only shortened hyp2 and hyp2a.
from nltk.translate.ribes_score import corpus_ribes

hyp1 = ['It', 'is', 'a', 'guide', 'to', 'action', 'which', 'ensures', 'that', 'the', 'military', 'always', 'obeys', 'the', 'commands', 'of', 'the', 'party']
ref1a = ['It', 'is', 'a', 'guide', 'to', 'action', 'that',  'ensures', 'that', 'the', 'military', 'will', 'forever',  'heed', 'Party', 'commands']
ref1b = ['It', 'is', 'the', 'guiding', 'principle', 'which', 'guarantees', 'the', 'military', 'forces', 'always', 'being', 'under', 'the', 'command', 'of', 'the', 'Party']
ref1c = ['It', 'is', 'the', 'practical', 'guide', 'for', 'the',  'army', 'always', 'to', 'heed', 'the', 'directions',  'of', 'the', 'party']

hyp2 = ['he', 'read', 'the']
ref2a = ['he', 'was', 'interested', 'in', 'world', 'history', 'because', 'he']

list_of_references = [[ref1a, ref1b, ref1c], [ref2a]]
hypotheses = [hyp1, hyp2]
corpus_ribes(list_of_references, hypotheses)

Output:
--------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-175-b02599af69e7> in <module>
      1 list_of_references = [[ref1a, ref1b, ref1c], [ref2a]]
      2 hypotheses = [hyp1, hyp2]
----> 3 corpus_ribes(list_of_references, hypotheses)

/anaconda3/envs/torch/lib/python3.7/site-packages/nltk/translate/ribes_score.py in corpus_ribes(list_of_references, hypotheses, alpha, beta)
    117     # Iterate through each hypothesis and their corresponding references.
    118     for references, hypothesis in zip(list_of_references, hypotheses):
--> 119         corpus_best_ribes += sentence_ribes(references, hypothesis, alpha, beta)
    120     return corpus_best_ribes / len(hypotheses)
    121

/anaconda3/envs/torch/lib/python3.7/site-packages/nltk/translate/ribes_score.py in sentence_ribes(references, hypothesis, alpha, beta)
     53         # Collects the *worder* from the ranked correlation alignments.
     54         worder = word_rank_alignment(reference, hypothesis)
---> 55         nkt = kendall_tau(worder)
     56
     57         # Calculates the brevity penalty

/anaconda3/envs/torch/lib/python3.7/site-packages/nltk/translate/ribes_score.py in kendall_tau(worder, normalize)
    288     num_possible_pairs = choose(worder_len, 2)
    289     # Kendall's Tau computation.
--> 290     tau = 2 * num_increasing_pairs / num_possible_pairs - 1
    291     if normalize:  # If normalized, the tau output falls between 0.0 to 1.0
    292         return (tau + 1) / 2

ZeroDivisionError: division by zero

I'm having same issue

Hi, in what version is this fix available? (because I am using 3.6.2 and I still have the same error)

@fabrien In the upcoming version, NLTK 3.6.3. This new version should launch soon.
If you do not want to wait, then you can use the following:
pip install --upgrade git+git://github.com/nltk/nltk.git

This will install the version currently on the development branch of NLTK, which has the fix applied.


Made sure the label doesn't overrun on team dropdown

Also added LoDash.

code looks ok to me, and has the desired effect on long departments. However, the Travis error makes me a bit twitchy about merging this in. I think it's ok as it appears to be something to do with the old dummy app. @threedaymonk can you confirm?

@fordie I think that's just because it's based on the point before I fixed the Travis configuration. If you rebase this onto origin/master, it should pass.

@fordie I have rebased it, so let's see if this works.

I also suggest squashing these two commits together before merging.


fix(package): update mathjs to version 3.12.2

Closes #55
https://greenkeeper.io/

Coverage remained the same at 76.567% when pulling b9f066dc1f61b4db0caea09319b855be65457132 on greenkeeper/mathjs-3.12.2 into 4102a896340306fd2f59750c35f8e513655bf01e on master.


EditorConfig highlighting should adhere to latest EditorConfig spec regarding inline comments

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version: 1.74.2

Steps to Reproduce:
In .editorconfig file, add the following:
abc = value1;value2

Note that ;value2 is highlighted as a comment. It shouldn't be highlighted as a comment per the current EditorConfig spec

We use the built in properties/ini grammar to do syntax highlighting for .editorconfig files. It looks like this generic properties grammar isn't ideal for that. This is something an extension could fix.

We try to keep VS Code lean and we think the functionality you're asking for is great for a VS Code extension. Maybe you can already find one that suits you in the VS Code Marketplace. Just in case, in a few simple steps you can get started writing your own extension. See also our issue reporting guidelines.
Happy Coding!


validator improvement (pattern list of lists)

I had a pattern that was a list of lists. This should be a type error, but it validated. I am sorry I lost the example.

I had a malformed match_dict today that was missing patterns entirely, and somehow wasn't caught?


Left/Right cb counts and error highlights are misleading above 4k (excluding pump)

Describe the bug
The Left/Right cb counts and error highlights seem to only take 4 columns into account rather than, say, 7 on kb7 files. Several hits on the error plot remain dim when cycling highlights, but I'm not sure exactly what columns are ignored or displayed properly. Issue present on kb7, solo, and doubles. Pump works as it should
To Reproduce
Play >4k file (note: not pump)
Press down arrow on result screen to cycle highlights
Expected behavior
Show hits correctly
Screenshots

You mention CB counts but only give an example of hand highlighting.
This was fixed in 0.65.1: https://github.com/etternagame/etterna/commit/cf0c5e7123df17545624de3a039e2a147bd6ce01

Woops, little oversight

fixed and handled by a563a8febe5f047892317802e4b07cef895bdc8d cae1e35063f88e5941b31bde69cefaba0af12c86


Feature Request: Enable joining a tailscale network

After creating a cluster via Terraform, I delete firewall rules (rel. #6) and add the cluster to a Tailscale network. I am able to generate a Tailscale key using Terraform, and it would be great to be able to pass that key to my Crunchybridge cluster to have it join the Tailscale network

What is the source / format of the tailscale key? Does it come from a tailscale provider or is it something else?

I can imagine a couple different workflows for managing the keys:

The Terraform-managed workflow that uses tailscale_tailnet_key to create the key and pass it to Crunchybridge
A more manual workflow where the user injects the key via a variable

In both cases, I think it's fair to expect a string similar to what is expected in the forms at https://crunchybridge.com/clusters/<some_cluster_id>/networking/tailscale


Issue With Elasticsearch Hadoop Config, Please help me

hadoop : 2.6.0
Hive : apache-hive-1.2.0-bin
Elasticsearch : 1.4.4
PATH enabled with variable HIVE_AUX_JARS_PATH also with hive-site.xml / hive-env.sh.
Jars : elasticsearch-hadoop-2.1.0.BUILD-20150604.023552-423.jar / elasticsearch-hadoop-1.3.0.M2.jar / elasticsearch-hadoop-1.3.0.M3.zip / elasticsearch-hadoop-2.0.2.jar /elasticsearch-hadoop-2.0.2.zip
Tried with all the available options but still getting the following error. Please help me.
hive> CREATE EXTERNAL TABLE wikitable (
>     title string,
>     redirect_page string )
> STORED BY 'org.elasticsearch.hadoop.hive.ESStorageHandler'
> TBLPROPERTIES('es.resource' = 'wikipedia_river/page/_search?q=*',
>               'es.host' = 'MongoDB-1.tcgsoftwareinc.com',
>               'es.port' = '9200');
FAILED: SemanticException Cannot find class 'org.elasticsearch.hadoop.hive.ESStorageHandler'

For questions use the public forum, not the issue tracker since this is not a bug.
Also I recommend reading on the docs - there is no ESStorageHandler - that's an old class in 1.3, one should use EsStorageHandler.
Also adding multiple versions of the library to the classpath (1.3.0.M2, M3, 2.0.2) doesn't make sense - you should use only one version.
Otherwise, the runtime will pick an arbitrary version and chaos will ensue.
This is the third time you are politely redirected to the public forum and again you chose to ignore it; there won't be any further updates / answers from my side in the future.

Really Sorry....I will find out the public forum...


[FLINK-28533] SchemaChange supports updateColumnNullability and updateColumnComment

Note: need to rebase master after FLINK-28573 is merged


GDScript: Fix some export annotation issues

Add missing static variable check for @export_custom.
Move the @export_storage implementation into a separate method and add a comment about the reasons.
Document that PROPERTY_USAGE_SCRIPT_VARIABLE flag is always added.
Fix Editor Help displaying rough types for annotation arguments.
Add autocompletion for @export_custom arguments.
Resolve #89816 (don't add property_hint for PROPERTY_HINT_NONE).
Fix class_name and usage for enum arrays, improve utils.notest.gd to track changes in tests.
Add more test cases in export_variable.gd.

Thanks!


mutate(Country = fct_recode(Country,        "US Virgin Islands" = "United States Virgin Islands")) -> latin_am3

latin_am2 %>%
bind_cols(flags1) %>%
mutate(Country = fct_recode(Country,
"US Virgin Islands" = "United States Virgin Islands")) -> latin_am3
can you explain what you did with this part of your code please, how will you modify this for Africa  . I am trying to do similar thing for Africa and I am now learning, not  experienced.

I did bind_cols to glue two datasets, the main dataset and the other one with iso codes for geom_flags. Second, I did mutate and fct_recode to abbreviate United States Virgin Islands to US Virgin Islands, which is to make its name shorter.
Does it answer your question?

thank you very much, I understand it now, can you check what I have done so far
the final result is not displaying what could be the problem, the codes have been running since yesterday

title: "week-31"
author: "Wonder Akwei"
date: "7/30/2021"
output: html_document
olympics <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2021/2021-07-27/olympics.csv')

library(tidyverse)
library(ggflags)
library(ggthemes)
library(ggtext)
library(countrycode)
library(ggplot2)

# data with country and continents
mobile <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-11-10/mobile.csv') %>%
  dplyr::select(entity, continent) %>%
  filter(continent == "Africa") -> country

country %>%
  left_join(olympics, by = c("entity" = "team")) %>%
  filter(!entity %in% c("Africa") &
           season == "Summer") %>%
  filter(!is.na(medal)) %>%
  distinct(entity, games, event, medal) %>%
  count(entity, medal) -> africa
head(africa)
africa

africa %>%
  pivot_wider(id_cols = entity, names_from = medal, values_from = n) %>%
  mutate_at(c("Bronze", "Gold", "Silver"), ~ replace_na(., 0)) %>%
  mutate(Total = Gold + Silver + Bronze) %>%
  group_by(entity) %>%
  arrange(desc(Gold)) %>%
  ungroup(entity) %>%
  mutate(Rank = row_number()) ->africa1
head(africa1)

africa1 %>%
  #  bind_cols(flag_country) %>%
  pivot_longer(c("Gold", "Silver", "Bronze"),
               names_to = "medal",
               values_to = "n") %>%
  dplyr::select(Rank, entity, medal, n, Total) %>%
  rename(Country = entity) ->africa2
head(africa2)

flags <-africa2 %>%
  pull(Country)
countrycode(flags, 'country.name', 'iso2c') %>%
  as_tibble() %>%
  rename(flag = value) %>%
  mutate(flag = str_to_lower(flag)) -> flags1

africa2 %>%
  bind_cols(flags1) -> african3

library(extrafont)
library(showtext)
showtext_auto()
font_add_google("Lato", "lato")
font_add_google("Play", "play")

african3 %>%
  ggplot(aes(x = Rank, y = n,
             country = flag)) +
  geom_point(african3 %>%
               filter(medal == "Gold"),
             mapping = aes(y = 45),
             size = 10,
             color = "gold",
             show.legend = FALSE) +
  geom_text(african3 %>%
              filter(medal == "Gold"),
            mapping = aes(label = n, y = 45),
            fontface = "bold") +
  geom_point(african3 %>%
               filter(medal == "Silver"),
             mapping = aes(y = 55),
             size = 10, color = "#d6d6d6",
             show.legend = FALSE) +
  geom_text(african3 %>%
              filter(medal == "Silver"),
            mapping = aes(label = n, y = 55),
            fontface = "bold") +
  geom_point(african3 %>%
               filter(medal == "Bronze"),
             mapping = aes(y = 65),
             size = 10,
             color = "#cd7f32",
             show.legend = FALSE) +
  geom_text(african3 %>%
              filter(medal == "Bronze"),
            mapping = aes(label = n, y = 65),
            fontface = "bold") +
  coord_flip() +
  scale_x_reverse() +
  scale_y_continuous(limits = c(5, 70)) -> fig1

symbol_ol <- ""

update.packages("rcpp")
fig1 +
  geom_richtext(african3 %>%
                  group_by(Country) %>%
                  filter(n == min(n)),
                mapping = aes(y = 30, label = Country),
                hjust = 1, size = 6, label.color = NA) +
  geom_flag(african3 %>%
              group_by(Country) %>%
              filter(n == min(n)),
            mapping = aes(y = 33), hjust = 1, size = 7) +
  labs(x = NULL, y = NULL,
       title = "Africa summer Olympics medals",
       subtitle = symbol_ol,
       caption = "Data from Athens 1896 to Rio 2016 | Source: Kaggle | Figure: @wonderakwei") +
  theme_void(base_family = "lato") +
  theme(axis.title.y = element_blank(),
        axis.title.x = element_blank(),
        axis.text.y = element_blank(),
        panel.grid.minor = element_blank(),
        plot.caption = element_markdown(hjust = 0.5,
                                        color = "gray60",
                                        size = 11),
        plot.subtitle = element_markdown(hjust = 0.5,
                                         margin=margin(0,0,-5,0)),
        plot.title = element_markdown(family = "play",
                                      face = "bold",
                                      size = 20,
                                      hjust = 0.5)) ->africa4
africa4
ggsave("fig.png", bg = "white", height = 12)

I am not sure, your code is running well. I got the figure using your code below:
olympics <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2021/2021-07-27/olympics.csv')

library(tidyverse)
library(ggflags)
library(ggthemes)
library(ggtext)
library(countrycode)
library(ggplot2)

# data with country and continents
mobile <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-11-10/mobile.csv') %>%
  dplyr::select(entity, continent) %>%
  filter(continent == "Africa") -> country

country %>%
  left_join(olympics, by = c("entity" = "team")) %>%
  filter(!entity %in% c("Africa") &
           season == "Summer") %>%
  filter(!is.na(medal)) %>%
  distinct(entity, games, event, medal) %>%
  count(entity, medal) -> africa
head(africa)

africa %>%
  pivot_wider(id_cols = entity,
              names_from = medal,
              values_from = n) %>%
  mutate_at(c("Bronze", "Gold", "Silver"), ~ replace_na(., 0)) %>%
  mutate(Total = Gold + Silver + Bronze) %>%
  group_by(entity) %>%
  arrange(desc(Gold)) %>%
  ungroup(entity) %>%
  mutate(Rank = row_number()) -> africa1
head(africa1)

africa1 %>%
  #  bind_cols(flag_country) %>%
  pivot_longer(c("Gold", "Silver", "Bronze"),
               names_to = "medal",
               values_to = "n") %>%
  dplyr::select(Rank, entity, medal, n, Total) %>%
  rename(Country = entity) -> africa2
head(africa2)

flags <- africa2 %>%
  pull(Country)
countrycode(flags, 'country.name', 'iso2c') %>%
  as_tibble() %>%
  rename(flag = value) %>%
  mutate(flag = str_to_lower(flag)) -> flags1

africa2 %>%
  bind_cols(flags1) -> african3

library(extrafont)
library(showtext)
showtext_auto()
font_add_google("Lato", "lato")
font_add_google("Play", "play")

african3 %>%
  ggplot(aes(x = Rank, y = n,
             country = flag)) +
  geom_point(african3 %>%
               filter(medal == "Gold"),
             mapping = aes(y = 45),
             size = 10,
             color = "gold",
             show.legend = FALSE) +
  geom_text(african3 %>%
              filter(medal == "Gold"),
            mapping = aes(label = n, y = 45),
            fontface = "bold") +
  geom_point(african3 %>%
               filter(medal == "Silver"),
             mapping = aes(y = 55),
             size = 10, color = "#d6d6d6",
             show.legend = FALSE) +
  geom_text(african3 %>%
              filter(medal == "Silver"),
            mapping = aes(label = n, y = 55),
            fontface = "bold") +
  geom_point(african3 %>%
               filter(medal == "Bronze"),
             mapping = aes(y = 65),
             size = 10,
             color = "#cd7f32",
             show.legend = FALSE) +
  geom_text(african3 %>%
              filter(medal == "Bronze"),
            mapping = aes(label = n, y = 65),
            fontface = "bold") +
  coord_flip() +
  scale_x_reverse() +
  scale_y_continuous(limits = c(5, 70)) -> fig1
symbol_ol <- ""

fig1 +
  geom_richtext(african3 %>%
                  group_by(Country) %>%
                  filter(n == min(n)),
                mapping = aes(y = 30, label = Country),
                hjust = 1, size = 6, label.color = NA) +
  geom_flag(african3 %>%
              group_by(Country) %>%
              filter(n == min(n)),
            mapping = aes(y = 33), hjust = 1, size = 7) +
  labs(x = NULL, y = NULL,
       title = "Africa summer Olympics medals",
       subtitle = symbol_ol,
       caption = "Data from Athens 1896 to Rio 2016 | Source: Kaggle | Figure: @wonderakwei") +
  theme_void(base_family = "lato") +
  theme(axis.title.y = element_blank(),
        axis.title.x = element_blank(),
        axis.text.y = element_blank(),
        panel.grid.minor = element_blank(),
        plot.caption = element_markdown(hjust = 0.5,
                                        color = "gray60",
                                        size = 11),
        plot.subtitle = element_markdown(hjust = 0.5,
                                         margin=margin(0,0,-5,0)),
        plot.title = element_markdown(family = "play",
                                      face = "bold",
                                      size = 20,
                                      hjust = 0.5)) ->africa4

ggsave("fig.png", bg = "white", height = 12)

It is still running here, haha I don't even know what the problem is. Thank you very much.


trouble with Key: dependencyClasspath (sbt 0.11.0)

I am working with the ProguardPlugin and a Multi-Build Project.  I have two subprojects and a root project.  The root project will be a library that contains common code for the two subprojects, this root project does not YET contain any code whatsoever, it only contains the sbt build definition for my overall project.  So my two subprojects are set to depend on my root project even though there isn't any code to depend on yet.  With this setup I attempt to use the ProguardPlugin on one of the subprojects.
The ProguardPlugin executes the following code:
def proguardInJarsTaskImpl: Initialize[Task[Seq[File]]] = {
	(dependencyClasspath in Compile, proguardInJars, proguardLibraryJars) map {
		(dc, pij, plj) =>
			import Build.data
		data(dc).filterNot(plj.contains) ++ pij
	}
}

When it runs this code the Key: "dependencyClasspath in Compile" has the dependency of my root project included:
"C:\Users\username\gits\rootProject\target\scala-2.9.1\classes" which is great except that there is no code in the root project yet, so "C:\Users\username\gits\rootProject\target\scala-2.9.1\classes" is never created.  And I end up with a Proguard error when it looks for that dependency: No such file or directory.
 [error] {file:/C:/Users/cvanvranken.URMC-SH/gits/datawarehousequery/}datawarehousequery-client/*:proguard: java.io.IOException: Can't read [C:\Users\cvanvranken.URMC-SH\gits\datawarehousequery\target\scala-2.9.1\classes(;;;;!META-INF/MANIFEST.MF)] (No such file or directory)

So the question is where should this problem be tackled?  Within sbt, or within the ProguardPlugin?  My feeling is that sbt should always create the "\target\scala-2.9.1\classes" directory if it is going to report that it is a dependency.  Alternatively it should not report a classpath dependency for a project if the classes folder is not created.

Please don't open an issue while the mailing list discussion is still active.  I also see that you opened the issue on the proguard plugin, so I'm closing this.


CSOM docs missing

The CSOM API index is missing from this page.  It was here at one point.  Where did all the CSOM documentation go?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 94956608-25b4-0605-f43d-d161ebdde905
Version Independent ID: 5a5b71db-be77-2ac8-7b9f-82f13961d4fa
Content: SharePoint .NET Server, CSOM, JSOM, and REST API index
Content Source: docs/sp-add-ins/sharepoint-net-server-csom-jsom-and-rest-api-index.md
Product: sharepoint
GitHub Login: @spdevdocs
Microsoft Alias: spdevdocs

6 Months Later & this still hasn't been updated

@VesaJuvonen - this really needs to be fixed as there's still lots of value in .net based CSOM development over JS Dev, especially for external applications


Extract shared code from tweet and toot tools

These two tools do the same thing, just one talks to Twitter the other Mastodon. It seems likely that the main tool could be made generic over some trait.

Fixed by #112


Completions sometimes not working

I've noticed the completions are sometimes not working. This seems be happening only on top level DOM elements (like the first <div> inside <body>).
Haven't really worked out when exactly this happens. Well, sometimes™
Has anyone else noticed this?

Fixed in v 1.10.0. Release just went out. Update via package control should be available in the next 60 minutes.


Styling: Clarification of responsibilities

Context: Currently, foundational aspects of styling (here: fonts, viewport breakpoints) are defined both in SAP themes as well as project styles.css. We see duplication and blurring of responsibility.
Goals: Responsibilities are clarified, style definitions are provided by the designated artifact.
Examples:

Font definition variables exist both in Styles.css and Theme
Breakpoints (used by the header) are currently defined in styles.css

let's do this on the go, case by case, and discuss then as needed


generator: Use Swift 6.0.2 by default

Testing:

EndToEndTests continue to pass locally, but cannot run in CI
All other tests pass locally and in CI

@swift-ci test


Check for proper typing in inputs in bit_code and phase_code

Found this bug while looking into #785, but does not close that issue.

Actually, could someone help me get the checks to pass here? I wrote the test functions to check for invalid inputs, but that causes the GitHub actions to fail?


Support GML as WFS output format

We should add support for GML as WFS output format.

I can't remember why i wanted this so i'll close this for know.


How to create a "set" of lv.point_t()

I'm a python novice, and I am struggling with C-binding use.
Translating the line example from the C docs, I want to draw a line through 5 points
`line1=lv.obj()
line1 = lv.line(lv.scr_act())
line1.set_points(line_points, 5)`
but I can't work out how to correctly instantiate the line_points object? I've tried various combinations of lists, dictionaries and arrays, and the closest I've got is an error "TypeError: can't convert 'lv_point_t' object to str implicitly" when using an array.
I'm really excited about the possibilities lvgl on micropython opens up, especially with the PC emulation. Thanks for a great project.

Hi @patrickomatik
I'm happy to see you are interested in this project!
We have an online (web) simulator, here is a script that draws a line
The interesting code is
l = lv.line(scr)
l.set_points([{'x':100,'y':100},{'x':150,'y':100},{'x':150,'y':150}],3)

Another option is to create lv.point_t() objects, set their x and y properties, and pass an array of these objects.
The support for arrays was added on dev-6.0 branch, which will be released soon.
You can try it here.

Awesome - thanks for the speedy response! Emscripten support rocks.
I've pulled the dev-6.0 branch, but make -C ports/unix fails due to missing the directory ../../lib/lv_bindings/lvgl/src
The src directory also doesn't seem to be present in the lvgl repo - do I need to do something specific to create it? Building the lv_binding main branch behaves as expected.

@patrickomatik  Did you run git submodule update --recursive?

I think it should be git submodule update --init --recursive.

I cloned a fresh dev-6.0 repo using submodule recursive. @embeddedt 's suggestion seems to have done the trick, thanks!

Ooops
GEN build/frozen_mpy.c Traceback (most recent call last):   File "../../tools/mpy-tool.py", line 875, in <module>     main()   File "../../tools/mpy-tool.py", line 863, in main     raw_codes = [read_mpy(file) for file in args.files]   File "../../tools/mpy-tool.py", line 717, in read_mpy     raise Exception('incompatible .mpy version') Exception: incompatible .mpy version make: *** [../../py/mkrules.mk:119: build/frozen_mpy.c] Error 1
Suggestions please?

@patrickomatik make clean in both the mpy-cross and unix directories.

Great. I then get an error mpy-cross executable does not exist, which I fixed by making that directory first.
But the build fails with many, many errors of which this is just the first:
In file included from build/lvgl/lv_mpy.c:20:
../../py/obj.h:638:45: note: expected ‘mp_uint_t’ {aka ‘long unsigned int’} but argument is of type ‘void *’
mp_obj_t mp_obj_new_int_from_uint(mp_uint_t value);
~~~~~~^

@patrickomatik Please clone a fresh, new repository using these exact instructions and see if it works. I just tested a fresh copy and it worked properly for me.
git clone https://github.com/littlevgl/lv_micropython.git
cd lv_micropython
git checkout dev-6.0
git submodule update --init --recursive
cd mpy-cross
make -j1
cd ../ports/unix
make -j1

@embeddedt @patrickomatik I think https://github.com/littlevgl/lv_micropython.git is broken.
I'll fix it soon.
Until then you can try using https://github.com/amirgon/lv_mpy/tree/dev-6.0

I think https://github.com/littlevgl/lv_micropython.git is broken.

Exactly what is broken? I can clone and build it just fine.

Exactly what is broken? I can clone and build it just fine.

@embeddedt I meant the dev-6.0 branch. There's some mixup with submodule versions. I'm going to fix this soon when I make some commits and also align to latest dev-6.0 of lvgl itself.

@embeddedt,
Last night I tried merging my changes with your changes on dev-6.0 branch (your Emscripten port) and ran into trouble.

I noticed that you updated Micropython sources from upstream.
I think it would be better to rebase instead of merge when you update Micropython upstream. It could make it much easier to understand what changed and differentiate our changes from upstream changes.
Did you update to some tag/release, or to current master of Micropython upstream?
After you made your changes, did you try building some other targets besides Emscripten? (the unix port, esp32, stm32 ports?)

Did you update to some tag/release, or to current master of Micropython upstream?

I merged the current master. Is that a problem?

After you made your changes, did you try building some other targets besides Emscripten?

Yes, I can build the 'unix` port just fine on my computer.

Yes, I can build the unix port just fine on my computer.

@embeddedt I just tried cloning and building a clean version - it fails.
git clone https://github.com/littlevgl/lv_micropython.git
git submodule update --recursive --init
git checkout dev-6.0
cd ports/unix/
make DEBUG=1

Then I get
make: ../../mpy-cross/mpy-cross: Command not found
../../py/mkrules.mk:112: recipe for target 'build/frozen_mpy/upip.mpy' failed

Did I do something wrong? Could you try it on your side?

You need to build the mpy-cross directory first.

You need to build the mpy-cross directory first.

@embeddedt You are right, of course! I forgot this step.
Thanks!

Great. I then get an error mpy-cross executable does not exist, which I fixed by making that directory first.
But the build fails with many, many errors of which this is just the first:
In file included from build/lvgl/lv_mpy.c:20:
../../py/obj.h:638:45: note: expected ‘mp_uint_t’ {aka ‘long unsigned int’} but argument is of type ‘void *’
mp_obj_t mp_obj_new_int_from_uint(mp_uint_t value);

@embeddedt @patrickomatik I think https://github.com/littlevgl/lv_micropython.git is broken.
I'll fix it soon.
Until then you can try using https://github.com/amirgon/lv_mpy/tree/dev-6.0

@patrickomatik I just pushed a new version to dev-6.0. Everything should work now.
Please try it out, and let me know if there are any problems.
(don't forget to build mpy-cross first, like I did :wink:)

don't forget to build mpy-cross first

Just for the record, the reason we were seeing the mpy-cross issues is because MicroPython changed the process on their end: https://github.com/micropython/micropython/issues/3255

Closing for now.
Please re-open if there is still a problem.


Add print statement for config

To make it clear project config was discovered

The configs you say is the TestCaseConfig or the AbstractProjectConfig?

ProjectConfig.
Something like:
"Config discovered....."
On Sat, 29 Jun 2019 at 14:13, Gustavo Fão Valvassori <
<EMAIL_ADDRESS>wrote:

The configs you say is the TestCaseConfig or the AbstractProjectConfig?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kotlintest/kotlintest/issues/841?email_source=notifications&email_token=AAFVSGTBCKFSIQMJM7TTCQTP46X6NA5CNFSM4HZKBTLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY36RVQ#issuecomment-506980566,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAFVSGS2BRO5MA4PQIEDJGDP46X6NANCNFSM4HZKBTLA
.

Can be something like this?

Looks awesome.
I would make the true or false or single value ones on the same line though
to avoid taking up too much space.
On Sun, 30 Jun 2019, 13:20 Gustavo Fão Valvassori<EMAIL_ADDRESS>wrote:

Can be something like this?
[image: image]
https://user-images.githubusercontent.com/7588480/60400476-83a49f80-9b4a-11e9-8ce5-1ebc52eb8a4e.png
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kotlintest/kotlintest/issues/841?email_source=notifications&email_token=AAFVSGRDJN4OT4MV6FGOMVLP5D2NNA5CNFSM4HZKBTLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY4RCKQ#issuecomment-507056426,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAFVSGTW6ME4ZIANVUCXAI3P5D2NNANCNFSM4HZKBTLA
.


Refs #5388 Whitelist parsing checks. [5393]

Build status:

Linux
Mac
Windows

Errors seem unrelated to this PR, rebuilding.

Build status:

Linux
Mac
Windows


Update consume-retail-server-api.md

The client secret must be created in the client app registration (Create the app registrtation for the client), NO in the Retail app registration. If you do it in the Retail App registration is giving errors by trying to get the access token.

:white_check_mark: Organization membership check passed
Requester is not a member of the MicrosoftDocs organization.


expand additional_bindings tag to use object form

Description

Expand the additional_bindings tag to use object form as to allow for Typescript transposing of the Provenance Protobuf API.
update proto-all make command to also run statik doc updates.

Note, this PR also updates the swagger docs for some previously merged proto changes.
closes: #1405

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[ ] Targeted PR against correct branch (see CONTRIBUTING.md)
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Wrote unit and integration tests
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added relevant godoc comments.
[ ] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[ ] Re-reviewed Files changed in the Github PR explorer
[ ] Review Codecov Report in the comment section below once CI passes

Note that no changes were made to the swagger endpoints after running make proto-all but I still wanted to test a few to make sure.

make proto-all
make clean build run-config
./build/provenanced config set api.enable true api.swagger true --home ./build/run/provenanced
make run
add contract-secification

./provenanced tx metadata write-contract-specification contractspec1q0w6ys5g6jm509v2830374aprsrq260w62 tp1f27w2zsxk84836s2apk4n6ghrg405tt9dkga39 "owner" "hashvalue" "myclassname" \
--home ~/workspace/provenance/build/run/provenanced \
--from validator \
--gas auto --gas-prices<PHONE_NUMBER>nhash --gas-adjustment 2

add scope-specification

./provenanced tx metadata write-scope-specification scopespec1qn7jh3jvw4gytq9r5x770e8yj74s9t479r tp1f27w2zsxk84836s2apk4n6ghrg405tt9dkga39 "owner" contractspec1q0w6ys5g6jm509v2830374aprsrq260w62 \
--home ~/workspace/provenance/build/run/provenanced \
--from validator \
--gas auto --gas-prices<PHONE_NUMBER>nhash --gas-adjustment 2

test URL endpoints for scope and contract spec: http://localhost:1317/swagger/


Fix array length

fix issue with array length, which broke the vue-reactivity example (and thus the todo-vue example)

Pull Request Test Coverage Report for Build 918127956

6 of 6 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.6%) to 75.68%

Totals

Change from base Build 846492725:
0.6%

Covered Lines:
331

Relevant Lines:
407

💛  - Coveralls


[HSDEV-2550][HSDEV-1475][TC] measure keypress TC latency

@erivni

nice, how about to add simple test for it?
you can add time to rtps packet and see that the sample contains the earliest one..
check out:
https://github.com/erivni/webrtc/blob/master/pkg/media/samplebuilder/samplebuilder_test.go#L72


Internal field

adding internal field support was pretty quick and easy after i figured out how the code generation was working.

Looks good, merged


move domain services possible?

is it please possible to move azure ad domain services from one subscription to another?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 76c1c185-685f-27e0-e634-844b510b1834
Version Independent ID: 09375778-5354-ab44-c5c6-94c2aebe34bc
Content: Move Azure resources to a new subscription or resource group
Content Source: articles/azure-resource-manager/resource-group-move-resources.md
Service: azure-resource-manager
GitHub Login: @tfitzmac
Microsoft Alias: tomfitz

@jurajgh - you can find move support by resource type at Move operation support for resources. If you are referring to Microsoft.AAD/domainservices, it does not support move.

#please-close