mopidy-mopify: 1.5.17 -> 1.6.0

Semi-automatic update. These checks were performed:

built on NixOS
found 1.6.0 with grep in /nix/store/k988x3a5gm4ff987yynn97gzcxh16hbl-mopidy-mopify-1.6.0
found 1.6.0 in filename of file in /nix/store/k988x3a5gm4ff987yynn97gzcxh16hbl-mopidy-mopify-1.6.0

cc "@Gonzih"

Failure on x86_64-darwin (full log)
Partial log (click to expand)
Package ‘gst-plugins-bad-1.12.3’ in /Users/graham/nix-borg/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/development/libraries/gstreamer/bad/default.nix:29 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Partial log (click to expand)
Installing collected packages: Mopidy-Mopify
Successfully installed Mopidy-Mopify-1.6.0
/tmp/nix-build-mopidy-mopify-1.6.0.drv-0/mopidy-mopify-1.6.0
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/k988x3a5gm4ff987yynn97gzcxh16hbl-mopidy-mopify-1.6.0
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/k988x3a5gm4ff987yynn97gzcxh16hbl-mopidy-mopify-1.6.0/lib
patching script interpreter paths in /nix/store/k988x3a5gm4ff987yynn97gzcxh16hbl-mopidy-mopify-1.6.0
checking for references to /tmp/nix-build-mopidy-mopify-1.6.0.drv-0 in /nix/store/k988x3a5gm4ff987yynn97gzcxh16hbl-mopidy-mopify-1.6.0...

Success on aarch64-linux (full log)
Partial log (click to expand)
Successfully installed Mopidy-Mopify-1.6.0
/build/mopidy-mopify-1.6.0
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/r7m24bn12wak81521zdrql9s1kby811s-mopidy-mopify-1.6.0
strip is /nix/store/lvx1acn1ig1j2km8jds5x3ggh3f2wa8v-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/r7m24bn12wak81521zdrql9s1kby811s-mopidy-mopify-1.6.0/lib
patching script interpreter paths in /nix/store/r7m24bn12wak81521zdrql9s1kby811s-mopidy-mopify-1.6.0
checking for references to /build in /nix/store/r7m24bn12wak81521zdrql9s1kby811s-mopidy-mopify-1.6.0...
/nix/store/r7m24bn12wak81521zdrql9s1kby811s-mopidy-mopify-1.6.0

Failure on x86_64-darwin (full log)
Partial log (click to expand)
Package ‘gst-plugins-bad-1.12.3’ in /private/var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/development/libraries/gstreamer/bad/default.nix:29 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Partial log (click to expand)
Installing collected packages: Mopidy-Mopify
Successfully installed Mopidy-Mopify-1.6.1
/tmp/nix-build-Mopidy-Mopify-1.6.1.drv-0/Mopidy-Mopify-1.6.1
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/shr7pcax3pczqp37gpk769ppylr2y2nv-Mopidy-Mopify-1.6.1
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/shr7pcax3pczqp37gpk769ppylr2y2nv-Mopidy-Mopify-1.6.1/lib
patching script interpreter paths in /nix/store/shr7pcax3pczqp37gpk769ppylr2y2nv-Mopidy-Mopify-1.6.1
checking for references to /tmp/nix-build-Mopidy-Mopify-1.6.1.drv-0 in /nix/store/shr7pcax3pczqp37gpk769ppylr2y2nv-Mopidy-Mopify-1.6.1...

Failure on x86_64-darwin (full log)
Partial log (click to expand)
Package ‘gst-plugins-bad-1.12.3’ in /Users/graham/nix-borg/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-ndnd/pkgs/development/libraries/gstreamer/bad/default.nix:29 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Partial log (click to expand)
Installing collected packages: Mopidy-Mopify
Successfully installed Mopidy-Mopify-1.6.1
/tmp/nix-build-Mopidy-Mopify-1.6.1.drv-0/Mopidy-Mopify-1.6.1
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/shr7pcax3pczqp37gpk769ppylr2y2nv-Mopidy-Mopify-1.6.1
strip is /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/shr7pcax3pczqp37gpk769ppylr2y2nv-Mopidy-Mopify-1.6.1/lib
patching script interpreter paths in /nix/store/shr7pcax3pczqp37gpk769ppylr2y2nv-Mopidy-Mopify-1.6.1
checking for references to /tmp/nix-build-Mopidy-Mopify-1.6.1.drv-0 in /nix/store/shr7pcax3pczqp37gpk769ppylr2y2nv-Mopidy-Mopify-1.6.1...

Failure on aarch64-linux (full log)
Partial log (click to expand)
shrinking /nix/store/brppjm02cjk6xr0msck4vz8q5w25hi89-glib-networking-2.54.1/lib/gio/modules/libgiolibproxy.so
shrinking /nix/store/brppjm02cjk6xr0msck4vz8q5w25hi89-glib-networking-2.54.1/libexec/glib-pacrunner
strip is /nix/store/viv0il4zpdqj4p7yamwbmn2k9pb8dcnv-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/brppjm02cjk6xr0msck4vz8q5w25hi89-glib-networking-2.54.1/lib  /nix/store/brppjm02cjk6xr0msck4vz8q5w25hi89-glib-networking-2.54.1/libexec
patching script interpreter paths in /nix/store/brppjm02cjk6xr0msck4vz8q5w25hi89-glib-networking-2.54.1
checking for references to /build in /nix/store/brppjm02cjk6xr0msck4vz8q5w25hi89-glib-networking-2.54.1...
strip is /nix/store/viv0il4zpdqj4p7yamwbmn2k9pb8dcnv-binutils-2.28.1/bin/strip
cannot build derivation '/nix/store/1g1l4xvhpfznzgcqryd1k2dgbb1dw6h4-mopidy-2.1.0.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/pgfdbsgh75yn71y1zjw5nx0s1cwpxgnx-Mopidy-Mopify-1.6.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/pgfdbsgh75yn71y1zjw5nx0s1cwpxgnx-Mopidy-Mopify-1.6.1.drv' failed


Fullscreen issues

Windows 11 here. I used a hotkey library to bind the escape key to leave fullscreen. Codewise this does App.MainWindow.SetFullScreen(false);. While it does leave the fullscreen mode as desired it does not fall back to the original window size, hence resulting in a maximized window. And secondly more annoying: my app is using the chromeless window, but after changing fullscreen to false, im suddenly getting the chrome decoration back.

@IceReaper We can reproduce this. Thanks for the heads up. We are getting ready to publish a new version now, but will try to get to this right after.

@IceReaper Thanks for pointing this out. We're about to publish a new version and will try to fix this right after.

This will require window state management on each OS in an appropriate fashion.
@IceReaper you can implement this by writing the current size and position of the window into a file and set them after returning from the fullscreen state.


Problem with launching with the mod

I can't launch Minecraft with catia and it has a message saying," Forge Mod loader has found a problem with your Minecraft installation. Forge: minimum version <IP_ADDRESS> something. file 'logs/fml-client-latest.log'
I don't know what to do please help me here is the crash report maybe it can help to solve this problem?
crash-2020-12-03_10.41.39-fml.txt

Also forgot to say its for 1.8.9


autoheal constantly restarts on linux/arm/v7

Tried deploying a container with this image on Armbian (an OrangePi PC board), but it constantly restarts.
First, tried running it with the following DockerCompose (I previously deployed the exact same Compose on a linux/amd64 host, and there works fine):
version: '3'

services:
  autoheal:
    # https://github.com/willfarrell/docker-autoheal
    container_name: autoheah
    image: willfarrell/autoheal:latest
    network_mode: none
    environment:
      - AUTOHEAL_CONTAINER_LABEL=all
      - AUTOHEAL_INTERVAL=10
      - AUTOHEAL_START_PERIOD=60
      - AUTOHEAL_DEFAULT_STOP_TIMEOUT=25
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
    restart: always

The container is constantly restarting, producing the following output (restarts about once each ~+60 seconds because I set the AUTOHEAL_START_PERIOD to 60):
2021-08-28T11:13:04.616302674Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:14:07.059787538Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:15:09.648184993Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:16:11.946453731Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:17:14.241354914Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:18:16.564788868Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:19:18.855314561Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:20:21.163773499Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:21:23.453673455Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:22:25.853702825Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:23:28.171957971Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:24:30.817528380Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:25:33.123744990Z Monitoring containers for unhealthy status in 60 second(s)
2021-08-28T11:26:35.328544128Z Monitoring containers for unhealthy status in 60 second(s)

Running the proposed docker run command:
docker run -d \
    --name autoheal \
    --restart=always \
    -e AUTOHEAL_CONTAINER_LABEL=all \
    -v /var/run/docker.sock:/var/run/docker.sock \
    willfarrell/autoheal

makes the container to be constantly restarting, and no output is given. If I set the restart policy to none, the container exists with code 28 in both cases.
Host system info:
 OS: Debian 10 buster
 Kernel: armv7l Linux 5.10.43-sunxi
 Uptime: 4d 21h 58m
 Packages: 479
 Shell: 17285
 CPU: ARMv7 rev 5 (v7l) @ 4x 1.368GHz [53.0°C]
 GPU:
 RAM: 362MiB / 999MiB

I have the same issue with the later versions of this image. It seems to be working well from an older version (although I am not sure what has changed since.) willfarrell/autoheal@sha256:0ad8b27083d065b8c22ea4db6b245097b8e3d3e44090196b11559de88801020c is a digest that currently works on linux/arm/v7 (specifically an Rpi4). According to the ReadMe, latest is built daily, but I have not yet looked into what may have changed that would cause this to break.

@itrogers Now that you mention that a previous build worked for you, I think I found the problem, which I had some weeks ago with python:alpine images (https://github.com/docker-library/python/issues/637). It seems that since Alpine 3.13.0, some breaking change involving 32bit platforms was introduced - I'm not really into the details, but it's well explained here: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0#time64_requirements
I tried the same fix (described in https://github.com/docker-library/python/issues/637#issuecomment-904544160) and it worked!
I created a simple compose that deploys a failing container for testing the functionality:
version: "3"

services:
  failing-healthcheck:
    container_name: failing-healthcheck-test
    image: debian:11
    command: sleep infinity
    network_mode: none
    healthcheck:
      test: ["CMD", "ls", "foo"]
      interval: 5s
      timeout: 1s
      retries: 3
      start_period: 0s

The autoheal container was working properly:
01-10-2021 15:19:38 Container /failing-healthcheck-test (87f1e0d23036) found to be unhealthy - Restarting container now with 10s timeout
01-10-2021 15:20:11 Container /failing-healthcheck-test (87f1e0d23036) found to be unhealthy - Restarting container now with 10s timeout
01-10-2021 15:20:43 Container /failing-healthcheck-test (87f1e0d23036) found to be unhealthy - Restarting container now with 10s timeout
01-10-2021 15:21:17 Container /failing-healthcheck-test (87f1e0d23036) found to be unhealthy - Restarting container now with 10s timeout
01-10-2021 15:21:49 Container /failing-healthcheck-test (87f1e0d23036) found to be unhealthy - Restarting container now with 10s timeout
01-10-2021 15:22:22 Container /failing-healthcheck-test (87f1e0d23036) found to be unhealthy - Restarting container now with 10s timeout
01-10-2021 15:22:55 Container /failing-healthcheck-test (87f1e0d23036) found to be unhealthy - Restarting container now with 10s timeout
01-10-2021 15:23:28 Container /failing-healthcheck-test (87f1e0d23036) found to be unhealthy - Restarting container now with 10s timeout

I guess another way for making it work on armv7 would be to use an Alpine base image version before 3.13. This may be automated using build ARGs, and parametrizing the architectures in the build workflow. What do you think @willfarrell ? (I could try working on a PR if you agree)

seems to be broken for RPi4 using latest image. @itrogers suggested image works.

still broken on RPi4 :(

You can use this instead https://github.com/qdm12/deunhealth. It does the same thing and works on a RPI4.

seems to be broken for RPi4 using latest image. @itrogers suggested image works.

+1
@willfarrell  Is this image still maintained?


Add Support custom subscription config/creds for stress tests

This PR adds support for "bring your own subscription" when deploying test resources. This will enable a new us to bill deployed resources to a separate team that owns their own test subscription (in this case expensive storage and cosmos test resources).
The process is very similar to how we do this for live testing. We set up a new secret for a partner team which they can update with service principal credentials and a subscription id to their test subscription. Then additionally they can specify which subscription config they would like to target on a per-scenario or all-scenario basis in scenario matrix config.
See the README updates in this PR for more details on the process.

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow

/check-enforcer override

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow


[hwameistor UI] Local Volumes--Operations missing "End at"

add "end at" to local volume operation list, but not sure which field is to be shown

This field is not yet enabled and can be hidden for now. Once the current task is executed, it will be deleted, and there will be no record.

hide "end at" column right now


No RSS?

Hi!
Thanks for creating this! Do you know what is going wrong?
$ python3 ytdl-current-video-archiver.py -f feeds.txt -v 2
-Initialized.-
-The current video cache file doesn't exist. It will be recreated.-
-Getting RSS feeds...-
Got feed https://www.youtube.com/feeds/videos.xml?channel_id=Dutchtuber2 from https://www.youtube.com/channel/Dutchtuber2.
-Checking current video cache...-
Parsing feed https://www.youtube.com/feeds/videos.xml?channel_id=Dutchtuber2...
Traceback (most recent call last):
  File "/Users/me/git/github.com/PrivateCaller/ytdl-current-video-archiver/ytdl-current-video-archiver.py", line 171, in <module>
    YouTubeDL_Current_Video_Archiver(arg.feed_file, arg.additional_urls, arg.output, arg.current_videos_cache_file, arg.video_depth, arg.downloader, arg.downloader_args).do_cycle()
  File "/Users/me/git/github.com/PrivateCaller/ytdl-current-video-archiver/ytdl-current-video-archiver.py", line 117, in do_cycle
    self.check_current_video_cache()
  File "/Users/me/git/github.com/PrivateCaller/ytdl-current-video-archiver/ytdl-current-video-archiver.py", line 65, in check_current_video_cache
    video = feed.entries[video]
            ~~~~~~~~~~~~^^^^^^^
IndexError: list index out of range
            ~~~~~~~~~~~~^^^^^^^
IndexError: list index out of range

I'll take a look at it tonight.

@sbkg0002 Is the video depth you entered greater than the number of videos on the channel? The script didn't account for that, but it should be fixed in the latest commit.


Possible to use int8 during pretraining stage?

Since the LLaMA model is frozen during pretraining, I was wondering if it was possible to run the model with int8 to reduce the vram used during training.
In my attempts to do so, I successfully started training with llama as int8 (by just adding the usual load_in_8bit=True, device_map=... combo), but observed that the loss would always collapse to 0.0 immediately. I also noticed the same issue when I simply loaded the llama model as fp16, rather than as bf16.

also just an fyi but the references to fastchat/train/train_mem.py in the README should probably be changed to llava/train/train_mem.py

Hi @152334H, thanks for your interest in our work!  Thanks for pointing out the script path issue, and I have just corrected it.
Regarding using INT8 during the pretraining stage, there may be issues doing so.  The reason is that the Linear layer that is being projected, are optimized with the gradient that is backward propagated all the way from the last layer to the first layer of the LLM.  When directly doing this INT8 loading, I guess some gradients will underflow / overflow.
I do know that some Alpaca-LORA projects use INT8 training, and I am interested in integrating these techniques into our project as well.  I will also look into this after finishing all related release stuff.  Please let me know if you are also interested in this and would like to contribute!

Hi, I am closing this issue, due to the inactivity.  Hope your problem has been resolved.  If you have further concerns, please feel free to re-open or open a new issue.  Thanks!

No problem.


Emissions

/cc @jstrachan adding the distinctUntilChanged on each store did help a bit, as did doing a zip. I'm not 100% sure, but I think that might be it?

There were the following issues with your Pull Request

Commit: ed2aa47cf9c2c749f679f1d97459a47a06310ada

Your commit message does not contain a body

Commit: 8de97f0ac6c0714f23b3ff2598b9b032f8c478ef

Your commit message does not contain a body

Guidelines are available at https://github.com/fabric8io/fabric8-ui

This message was auto-generated by https://gitcop.com

@jstrachan what I couldn't work out what was what uniquely identifies the build and build config objects we get from openshift - with this I might be able to better filter the streams. I'm going to dig a bit deeper tomorrow.

Not sure the zip is correct though.

[test]


updating recipient ids in honeycombio_trigger doesn't update recipient ids

Versions
0.13.0
Steps to reproduce

Write this module like this (copied from the docs on honeycombio_trigger, substituting the channel values for Slack channels in your own Slack instance, using a honeycombio_dataset that has traces with child spans in it so the trace.parent_id column exists, and updating time_range to a working value):

resource "honeycombio_dataset" "test-yourdataset" {
  name              = "test-yourdataset"
}

resource "honeycombio_slack_recipient" "frameworks-alerts-test1" {
  channel = "#test1"
}

resource "honeycombio_slack_recipient" "frameworks-alerts-test2" {
  channel = "#test2"
}

data "honeycombio_query_specification" "example" {
  calculation {
    op     = "P90"
    column = "duration_ms"
  }

  filter {
    column = "trace.parent_id"
    op     = "does-not-exist"
  }

  time_range = 2000
}

resource "honeycombio_query" "example3" {
  dataset    = honeycombio_dataset.test-yourdataset.name
  query_json = data.honeycombio_query_specification.example.json
}

resource "honeycombio_trigger" "example" {
  name        = "Requests are slower than usual"
  description = "Average duration of all requests for the last 10 minutes."

  query_id = honeycombio_query.example3.id
  dataset  = honeycombio_dataset.test-yourdataset.name

  frequency = 600 // in seconds, 10 minutes

  alert_type = "on_change" // on_change is default, on_true can refers to the "Alert on True" checkbox in the UI

  threshold {
    op    = ">"
    value = 4000000 # FIXME an absurdly high number so it doesn't fire.
  }

  recipient {
    id = honeycombio_slack_recipient.frameworks-alerts-test1.id
  }

Run terraform plan
Run terraform apply
Update the recipient block to:

   recipient {
    id = honeycombio_slack_recipient.frameworks-alerts-test1.id
  }

Run terraform plan and notice that says:

Terraform will perform the following actions:

  # honeycombio_trigger.example will be updated in-place
  ~ resource "honeycombio_trigger" "example" {
        id          = "J8ri22RPZ5d"
        name        = "Requests are slower than usual"
        # (6 unchanged attributes hidden)

      ~ recipient {
          ~ id     = "Bj7FG65MoWU" -> "fTwUjLv3GtS"
            # (2 unchanged attributes hidden)
        }

        # (1 unchanged block hidden)
    }

Run terraform apply and see it finishes successfully
Run terraform plan and see it still says that the recipient id needs to be changed in the exact same way:

Terraform will perform the following actions:

  # honeycombio_trigger.example will be updated in-place
  ~ resource "honeycombio_trigger" "example" {
        id          = "J8ri22RPZ5d"
        name        = "Requests are slower than usual"
        # (6 unchanged attributes hidden)

      ~ recipient {
          ~ id     = "Bj7FG65MoWU" -> "fTwUjLv3GtS"
            # (2 unchanged attributes hidden)
        }

        # (1 unchanged block hidden)
    }

Additional context
I believe a colleague worked around this problem previously by setting type and target in these recipient blocks as discussed in the comments of #262. That workaround possibly opened us up to #262 or some of the difficulties in #262 are another expression of this bug

I've attempted to write an acceptance test in a branch on my fork: jeff.hodges/debug-trigger-recipient-update
Since recipients are global objects across environments and I have only access to my org's set of environments, I wasn't able to get full isolation like y'all could.
But with its own environment and dataset set, I was able to get a failure that might be a reproduction of this issue. I'm open, though, to see it fixed up by y'all or take feedback to make it correct.
$  TF_ACC=1 go test -v -run=TestAccHoneycombIOTrigger_changingRecipientID ./...
?   	github.com/honeycombio/terraform-provider-honeycombio	[no test files]
testing: warning: no tests to run
PASS
ok  	github.com/honeycombio/terraform-provider-honeycombio/client	(cached) [no tests to run]
?   	github.com/honeycombio/terraform-provider-honeycombio/client/internal/httputil	[no test files]
=== RUN   TestAccHoneycombIOTrigger_changingRecipientID
    resource_trigger_test.go:213: Step 2/2 error: Check failed: honeycombio_trigger.test: Attribute 'recipient.0.id' expected "xmbNbyhSY91", got "rX2sUPHwbjn"
    testing_new.go:84: Error running post-test destroy, there may be dangling resources: exit status 1

        Error: 409 Conflict: Recipient rX2sUPHwbjn is being used by a trigger or burn alert and cannot be deleted

--- FAIL: TestAccHoneycombIOTrigger_changingRecipientID (4.38s)
FAIL
FAIL	github.com/honeycombio/terraform-provider-honeycombio/honeycombio	4.703s
?   	github.com/honeycombio/terraform-provider-honeycombio/honeycombio/internal/hashcode	[no test files]
?   	github.com/honeycombio/terraform-provider-honeycombio/honeycombio/internal/verify	[no test files]
FAIL

This is also happening on honeycombio_burn_alerts with recipient ids.
Calling terraform refresh, oddly, fixes the honeycombio_triggers to the right state, but honeycombio_burn_alerts both in the terraform state and in the Honeycomb API will have the old recipients.

Any updates on this one? We're still seeing it in production

Also, I seem to be wrong about terraform refresh fixing it.

@jmhodges-color sorry for the delay here! We'll pick this up today and see what we can sort out.

Hitting this again on dozens of triggers I'm having to fix manually

Just hit this again for another 6 triggers (and it's not even my usual job to manage these)

Any news on this one?

@jmhodges-color work is underway (https://github.com/honeycombio/terraform-provider-honeycombio/pull/305) to migrate honeycombio_trigger to the ~new Terraform Plugin Framework as I'm hopeful it will give us the control required -- which the current Plugin SDK makes rather challenging/hacky -- to squash this collection of related bugs .

Any news?

The trigger-recipient-fixes branch has a working solution but only for Terraform Core (CLI) 1.4.x.
Bit of a gut bunch as I was going to get it merged. Working on a solve for Terraform <1.4 as requiring everyone to update doesn't really feel like a path forward unless we're ready to cut version 1.0 of the provider


Failed to generate code from example

$ go generate ./example/api

DEBU[0000] args: ["/var/folders/3z/8t297jqn2ss3jjvp32t_6bk80000gp/T/go-build920965508/b001/exe/requestgen" "-type" "NoParamRequest" "-url" "/v1/bullet" "-method" "GET" "-debug"]
DEBU[0000] found APIClient field client -> github.com/c9s/requestgen.APIClient
DEBU[0000] add package import: fmt
DEBU[0000] add package import: net/url
DEBU[0000] add package import: encoding/json
DEBU[0000] add package import: regexp
DEBU[0000] add package import: net/url
DEBU[0000] add package import: context
DEBU[0001] importing github.com/c9s/requestgen/example/api from github.com/c9s/requestgen/example/api, same package object (pointer), no import  template-function=qualifier
DEBU[0001] importing github.com/c9s/requestgen/example/api from github.com/c9s/requestgen/example/api, same package object (pointer), no import  template-function=qualifier
DEBU[0001] importing github.com/c9s/requestgen/example/api from github.com/c9s/requestgen/example/api, same package object (pointer), no import  template-function=qualifier
DEBU[0001] importing github.com/c9s/requestgen/example/api from github.com/c9s/requestgen/example/api, same package object (pointer), no import  template-function=qualifier
DEBU[0001] importing github.com/c9s/requestgen/example/api from github.com/c9s/requestgen/example/api, same package object (pointer), no import  template-function=qualifier
DEBU[0001] importing github.com/c9s/requestgen/example/api from github.com/c9s/requestgen/example/api, same package object (pointer), no import  template-function=qualifier
DEBU[0001] importing github.com/c9s/requestgen/example/api from github.com/c9s/requestgen/example/api, same package object (pointer), no import  template-function=qualifier
DEBU[0001] apiClientField: 0x1400a30e928 apiUrl: 0x14000098f30
DEBU[0001] importing github.com/c9s/requestgen/example/api from github.com/c9s/requestgen/example/api, same package object (pointer), no import  template-function=qualifier
DEBU[0001] type interface{} is types.Interface, do not use pointer
DEBU[0001] type interface{} is types.Interface, do not use reference
DEBU[0000] args: ["/var/folders/3z/8t297jqn2ss3jjvp32t_6bk80000gp/T/go-build2438071941/b001/exe/requestgen" "-debug" "-type" "PlaceOrderRequest" "-responseType" ".Response" "-responseDataField" "Data" "-responseDataType" ".Order"]
DEBU[0002] ident Response matches type selector member Response
FATA[0002] can not find type matches the type selector &{Package:. Member:Response IsSlice:false} in the packages [github.com/c9s/requestgen/example/api]
exit status 1
example/api/place_order_request.go:87: running "go": exit status 1

https://github.com/narumiruna/requestgen-example-docker


@c0m1t - Hook intro

This PR was originally made by @c0m1t. Because he/she targeted master directly, I had to close the original one, and submit this PR to fa branch.

Hey.there is no new changes for this PR?
I am checking to accept the changes and merge them...
@sorousht  @c0m1t

@zahrajoulaei It's been a long time since @c0m1t have opened this PR. I know it isn't well written and does not meet our standards for being published, so I'm going to retouch and improve as much as possible by tonight.


Fix problem with shared preferences set to null

allows to pass null as a default value to StringPreference
fixes bug in Android (<=4.3) with setting the value to null

Current coverage is 64.83%

Merging #41 into master will increase coverage by +0.31% as of 6650856

@@            master     #41   diff @@
======================================
  Files           16      16
  Stmts          234     236     +2
  Branches        18      19     +1
  Methods          0       0
======================================
+ Hit            151     153     +2
  Partial          7       7
  Missed          76      76

Review entire Coverage Diff as of 6650856

Uncovered Suggestions

+2.54% via ...LayoutViewPager.java#26...31
+2.11% via ...ew/ToolbarUtils.java#16...20
+2.11% via ...LayoutViewPager.java#38...42
See 7 more...

Powered by Codecov. Updated on successful CI builds.

:+1:


Fixed link to CONTRIBUTING.md in README.md

Hi,
I was following the README and it seems the link to CONTRIBUTING.md is not apt. So this is my bit to fix that.

Duplicate to #32 👍


The Karaca's Encryption Algorithm

Before jumping into Pr first comment for assign.All Problem from Edabit. Link is mandatory to add
Problem statement:
Make a function that encrypts a given input with these steps:
	Input: "apple"
	Step 1: Reverse the input: "elppa"
	Step 2: Replace all vowels using the following chart:
	a => 0
	e => 1
	i => 2
	o => 2
	u => 3
	# "1lpp0"
	Step 3: Add "aca" to the end of the word: "1lpp0aca"
	Output: "1lpp0aca"
	Examples
	encrypt("banana") ➞ "0n0n0baca"
	encrypt("karaca") ➞ "0c0r0kaca"
	encrypt("burak") ➞ "k0r3baca"
	encrypt("alpaca") ➞ "0c0pl0aca"
	Notes
	All inputs are strings, no uppercases and all output must be strings.

Problem Link: https://edabit.com/challenge/JzBLDzrcGCzDjkk5n

Before submitting a PR please Check some details.Check Edabit for problem task

[ x] fileName formet should be in camelcase
example- returnTheNextNumber.py
[ x]  Add label for Problem
[ x]  In file there should be problem statement and link to probelm
example -

'''
    Probem Task : This program will add two numbers
    Problem Link : https://edabit.com/challenge/SFzHtm63XT6EYNHWY
'''
*/
def main():
    return a + b

This repository is vaild for HacktoberFest2020
Read before making PR - How to contribute on Github
Good Luck,

can this be assign to me please ?

Could you please assign this task to me?

I will do it.

can i work on this?

come on!! guys!.... you cant work on it if you aren't assigned to it yet ... the pull request will be rejected!! ... and besides ... i authored the issue and requested for it first ... it's a first come first serve thing guys ... the README says so ...


Compatibility issues with OctoberCMS v2.1.0

Hi,
I receive the following error message when I add CumulusGuard component to my CMS page:
Trying to get property 'slug' of non-object
~\plugins\initbiz\cumuluscore\eventhandlers\RainlabUserHandler.php line 86

return $model->clusters()->whereSlug($cluster->slug)->first() ? true : false;

Also when the user is added only to one cluster and I visit a page that has ClusterList component added, then I receive "ERR_TOO_MANY_REDIRECTS" error message on the page.
Could you please help me what could be the problem?
Am I doing something wrong or the plugin is not compatible with the latest build of OctoberCMS?
Versions that I use:

OctoberCMS: 2.1.0
Rainlab.User: 1.5.4
CumulusCore: 3.4.3

Thanks in advance for your help!

Hi @vimre,
Sorry for the late reply. I think that the problem might be caused by not properly configured the page. When you're adding the CumulusGuard component to the page, ensure you have the :slug or similar property set in the URL and corresponding value in the component's slug property. It is necessary for the component to know in which cluster the current session is. The too many redirects issue is probably caused by the same problem.
Hope that helps. If not, re-open the issue and let us know about any more problems.
Regards,
Tomasz Strojny


Add command to get an STS token for a role from an ocm token

What type of PR is this?
feature
What this PR does / Why we need it?
Adds a command to fetch STS credentials for given role
Which Jira/Github issue(s) does this PR fix?
OSD-16489
Pre-checks (if applicable)

[x] Ran unit tests locally
[ ] ~Validated the changes in a cluster~
[x] Included documentation changes with PR

To test locally:
In an AWS account, create a role with the trust policy in the attached ticket. Run this command with the arn for that role passed in as --role-arn, i.e. backplane cloud token --role-arn [role arn]. If successful, the command should output AWS STS Credentials as a JSON payload.

Codecov Report

Merging #112 (3113dad) into main (275dbf6) will increase coverage by 0.07%.
The diff coverage is 54.71%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #112      +/-   ##
==========================================
+ Coverage   50.12%   50.19%   +0.07%
==========================================
  Files          43       45       +2
  Lines        2715     2753      +38
==========================================
+ Hits         1361     1382      +21
- Misses       1122     1137      +15
- Partials      232      234       +2

Impacted Files
Coverage Δ

cmd/ocm-backplane/cloud/token.go
37.14% <37.14%> (ø)

pkg/utils/jwt.go
85.71% <85.71%> (ø)

cmd/ocm-backplane/cloud/cloud.go
66.66% <100.00%> (+6.66%)
:arrow_up:

cmd/ocm-backplane/login/login.go
81.25% <100.00%> (+3.04%)
:arrow_up:

Closing, as changes are encompassed by https://github.com/openshift/backplane-cli/pull/114


scroll animation still active upon changing states

Upon using ion-scroll, if the state is changed while still scrolling the animation continues,
expecting the elements to still be on the DOM, however these elements may not be on the DOM.
It thinks that the "container" element exists, while it is no longer on the DOM.
This produces an error:
Uncaught TypeError: Cannot read property 'querySelectorAll' of undefined

I would expect there not to be an error upon such behaviour.
Steps to reproduce:

use the chrome mobile emulator ( refresh to fully activate emulator mode )
scroll the "click me pls" buttons
click on a button ( this scrolls and changes state [ while scrolling ])
look at the error in the console

Other information:
run as MOBILE,
I tested it in mobile chrome emulator
Which Ionic Version? 1.3.1
Plunker that shows an example of your issue
http://plnkr.co/edit/4AhEQMShybHtipnyI8JL?p=preview

This issue was moved to driftyco/ionic-v1#94


Fix remotely held items @commands broken by #3379, brought by #3337

Commands brought with ThirdPersonRemoteClientSystem(#3337) were broken since #3379. This PR fixes them. These commands provide ability to set position and rotation of items held by remote players for local client trough ingame console. For debugging purposes
#3379
Contains
Edited ingame commands for remote held item translation rotation and mountpoint scale (the 3rd one is probaly useless though, might as well get rid of it). These commands, introduced by #3337 stopped working after #3379 changed the code to suit the refactored logic.
Changes: shortened, made it work. Redundant checks for relatesToLocalPlayer(remotePlayer) removed as only real remote player entities should have RemotePersonHeldItemMountPointComponent already (this check is performed when adding those). Unnecessary call to ensureClientSideEntityOnHeldItemMountPoint() deleted.
How to test
Start multiplayer game with 2 players minimum. Aim your screen at the other player/s. Fire up your console ("~") and use one of the commands:
setRemotePlayersHeldItemMountPointTranslations <Float x> <Float y> <Float z> for moving remote held items about. (in meters, 1 meter is one block i suppose?)
setRemotePlayersHeldItemMountPointRotations <Float x> <Float y> <Float z> for rotating remote held items about. (in degrees)
observe visual changes of items held by other players :)
Outstanding before merging
Probably nothing.

Would have added this to the previous PR but its' branch is deleted

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

Tested - works great! Nice work :-)


Fixed a bug (typo) in yolo_detection

I'm only learning onnxruntime extension but this seems to me like a typo.

This, however, doesn't fix the issue I'm struggling with:
from pathlib import Path
from onnxruntime_extensions.tools.add_pre_post_processing_to_model import yolo_detection

model_path = Path("yolov8n-pose.onnx")
new_model_path = model_path.with_name("yolov8n-pose_nms.onnx")

yolo_detection(model_path, new_model_path)

# ----------------

import numpy as np
import onnxruntime as ort
from onnxruntime_extensions import get_library_path

so = ort.SessionOptions()
so.register_custom_ops_library(get_library_path())

path = "yolov8n-pose_nms.onnx"

session = ort.InferenceSession(path, so, providers=['CPUExecutionProvider'])

outname = [i.name for i in session.get_outputs()]  # ['image_out']
inname = [i.name for i in session.get_inputs()]    # ['image']

im = np.ones((640, 640, 3), dtype=np.uint8)

inp = {inname[0]: im}
outputs = session.run(outname, inp)

Traceback (most recent call last):
  File "/home/dizcza/Projects/Airtouch/airpy/onnxruntime_demo.py", line 22, in <module>
    outputs = session.run(outname, inp)
  File "/home/dizcza/miniconda3/envs/mediapipe/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 217, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: image Got: 3 Expected: 1 Please fix either the inputs or the model.

If I flatten the image with im = im.reshape(-1) as suggested here, I get another error:
2023-11-07 17:30:09.644952955 [E:onnxruntime:, sequential_executor.cc:514 ExecuteKernel] Non-zero status code returned while running DecodeImage node. Name:'pre_process_0' Status Message: 2: [DecodeImage] Invalid input. Failed to decode image.
Traceback (most recent call last):
  File "/home/dizcza/Projects/Airtouch/airpy/onnxruntime_demo.py", line 22, in <module>
    outputs = session.run(outname, inp)
  File "/home/dizcza/miniconda3/envs/mediapipe/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 217, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running DecodeImage node. Name:'pre_process_0' Status Message: 2: [DecodeImage] Invalid input. Failed to decode image.

All right, apart from a strange ubyte input complaint, I think I cannot simply apply the same post-processing steps to the YoloPose model (not YoloDetect).
Long story short, how to merge NMS as a post-processing step to the YoloPose v8 model?

You can see the individual pre/post processing steps used for an object detection yolo model here
Whilst most of those steps are applicable, there are some differences for pose models that require a slightly different pipeline and some changes to existing steps to support the keypoint data.
For reference:

Object detection post-processing

https://github.com/ultralytics/ultralytics/blob/e7bd159a44cf7426c0f33ed9b413ef4439505a03/ultralytics/models/yolo/detect/predict.py#L23

Pose post-processing:

https://github.com/ultralytics/ultralytics/blob/e7bd159a44cf7426c0f33ed9b413ef4439505a03/ultralytics/models/yolo/pose/predict.py#L31

Differences:
For object detection there are 80 classes and no mask data (guessing 'mask' originally comes from a segmentation model usage but in this case that's where the keypoint data is).
For pose there's 1 class and 51 items of mask data (17 keypoints x 3 values for each).
Detection:

Pose:

To handle the differences SplitOutBoxAndScore and SelectBestBoundingBoxesByNMS need to be augmented to pass through the mask data. would probably require making some things configuration based. e.g. inferring whether there is an object confidence score or not (used in older yolo detection models) gets messy if you also want to infer the number of classes/mask size, and it's simpler to explicitly specify all those values when configuring the post-processing step.
There are some additional changes required to extract the key points from the mask data. That requires plugging in their shape. For this particular model it is [17, 3] as there are 17 keypoints (nose, eyes etc.) with x, y and a confidence score. Other models could have more/less keypoints and not have thhe confidence score. The x and y values from the keypoints need to be scaled as well (similar to what ScaleBoundingBoxes does).
https://github.com/ultralytics/ultralytics/blob/e7bd159a44cf7426c0f33ed9b413ef4439505a03/ultralytics/models/yolo/pose/predict.py#L48-L49
Those additional changes would need to be implemented as new post-processing steps, but maybe you don't need that to be done in the model at this point.

@skottmckay thanks for the detailed response.

Those additional changes would need to be implemented as new post-processing steps, but maybe you don't need that to be done in the model at this point.

But that's precisely what I want: I'm running YoloPose v8 on Android and OpenCV backend gives me 1.5 FPS - way too slow. To speed things up, I need to

quantize the model
merge NMS to the graph (currently I'm doing NMS as it's done here)

My naive idea was that I could simply borrow your script for yolo detection - I was wrong: it seems like a lot of work.

Anyway, yolo pose post processing is a feature request.
And this PR just fixes a negligible issue.

/azp run onnxruntime-extensions.CI

Anyway, yolo pose post processing is a feature request.
And this PR just fixes a negligible issue.

@dizcza The change actually creates a different issue: https://github.com/microsoft/onnxruntime-extensions/issues/584#issuecomment-1800741416
The naming was bad and there could have been comments explaining things better.
Can you describe your intended usage to help prioritize adding support for the pose model to the post processing? Is this for a production scenario and if so is there a timeline?

@dizcza The change actually creates a different issue: #584 (comment)
The naming was bad and there could have been comments explaining things better.

Oh, so this PR should have been closed then.

Can you describe your intended usage to help prioritize adding support for the pose model to the post processing? Is this for a production scenario and if so is there a timeline?

I'm using the yolov8 pose to monitor people interacting with the camera, mostly kids. I need to parse their poses, extract hand-relative orientations, and do some logic. The app runs on Windows, Linux, and Android. The Android part is the most challenging since a straightforward approach brings only 1.5 FPS - that's why I started looking into your repository specifically for (1) quantization and (2) NMS included in the model to get rid of manually parsing a large tensor and then calling cv::dnn::NMSBoxes. And NMS is the only post-processing step I'm interested in.
This is for a production scenario.
I think I can do quantization without any (significant) modification of your code.

We'll look at adding the post-processing changes to simplify usage. Possibly that will also cover most if not all of the new segmentation yolo model post-processing as well as a bonus.
FWIW in the next ORT release we're look at adding the one missing operator for the NNAPI EP to have full coverage of the model. Using the NNAPI EP could give better performance, but NNAPI performance is highly unpredictable from device to device. Also adding the 2 missing operators for CoreML to have complete coverage of the model and CoreML tends to give a far more consistent performance gain.
IIRC generally the internals of the model are quantized and the model input/output would still be fp32, so hopefully the pre/post processing can be used as-is.

This, however, doesn't fix the issue I'm struggling with:
from pathlib import Path
from onnxruntime_extensions.tools.add_pre_post_processing_to_model import yolo_detection

model_path = Path("yolov8n-pose.onnx")
new_model_path = model_path.with_name("yolov8n-pose_nms.onnx")

yolo_detection(model_path, new_model_path)

# ----------------

import numpy as np
import onnxruntime as ort
from onnxruntime_extensions import get_library_path

Traceback (most recent call last):
File "/home/dizcza/Projects/Airtouch/airpy/onnxruntime_demo.py", line 22, in
outputs = session.run(outname, inp)
File "/home/dizcza/miniconda3/envs/mediapipe/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 217, in run
return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: image Got: 3 Expected: 1 Please fix either the inputs or the model.

Hi, this is as we add a pre-process step in the model so that we can only accept the raw jpg/png image as input.
open('xx.jpg').read()

Hi, this is as we add a pre-process step in the model so that we can only accept the raw jpg/png image as input. open('xx.jpg').read()

Thanks, I thought that "raw bytes" means numpy ubyte data type. But I see how convenient it gets in Android native C++.
BTW, an ONNX Android with Native C++ integration example is missing, I'm really struggling collecting bits and pieces for such a use-case. And, of course, I want to reuse already working C++ ONNX backend code in Android. But this is a question for ONNX guys, I'll ask one somewhere else.

What would be different about Android C++ vs any other C++ usage? I believe the AAR has the C++ headers and the native shared library (libonnxruntime.so) is separate from the jni layer (libonnxruntime4j_jni.so).

What would be different about Android C++ vs any other C++ usage? I believe the AAR has the C++ headers and the native shared library (libonnxruntime.so) is separate from the jni layer (libonnxruntime4j_jni.so).

I'm talking about

a complete example of how to setup an Android project with ONNX native C++ bindings. Such things are always tricky unless you do it every day. I found only one working unofficial example https://github.com/Rohithkvsp/OnnxRuntimeAndorid
limitations of ONNX Java bindings. See this issue: https://github.com/microsoft/onnxruntime/issues/18369

Typically we'd expect an Android app developer to not be writing C++ directly given ORT has Java bindings. Is there a significant performance advantage to not using the ORT Java bindings?

Typically we'd expect an Android app developer to not be writing C++ directly given ORT has Java bindings.

Why? Because Java devs are afraid of C++? :) Won't you say such a thing to Android devs (including myself), who have lots of OpenCV code in C++, and all of a sudden they are forced to rewrite their C++ code to Java if OpenCV stops providing an interface to pass all their objects as pointers from Java to C++? We'd rather not support Android platforms at all then...

Is there a significant performance advantage to not using the ORT Java bindings?

Nope. The only reason I'd like to use ONNX in C++ on Android is to reuse the working code I have on C++. Luckily, there is not much code of ONNX in C++ and I think I can port it to Java with some effort.

Craigacp is the expert on the Java bindings so let's see what he says about exposing the native handler.
Worst case is it not possible to load the model as bytes in Java and pass that down to the C++ level to use ORT from then on? e.g. https://github.com/microsoft/onnxruntime-inference-examples/blob/afc83c003c2b262929d1667d8934021bfde8f394/mobile/examples/object_detection/android/app/src/main/java/ai/onnxruntime/example/objectdetection/MainActivity.kt#L98

Worst case is it not possible to load the model as bytes in Java and pass that down to the C++ level to use ORT from then on? e.g. https://github.com/microsoft/onnxruntime-inference-examples/blob/afc83c003c2b262929d1667d8934021bfde8f394/mobile/examples/object_detection/android/app/src/main/java/ai/onnxruntime/example/objectdetection/MainActivity.kt#L98

I cannot do this because neither OrtEnv nor SessionOptions have the getNativeHandle() function public. Citing myself,

Even if they had, why not provide the same functionality, getNativeHandle(), for OrtSession in the first place?

Sorry, I got your point. You suggest creating an environment and session options in C++... yeah, that could and should work.

With such an approach, you're really narrowing down the usage. I don't see any reason why the getNativeHandle function should not be public for all OrtX Java classes.

I'm not a Java expert so not sure if there's a reason to keep the function internal. I'm guessing the approach was to keep as much internal as possible to minimize the surface area of misuse (misuse == github issues == time to investigate) and maximize the flexibility of the implementation (anything not exposed can be changed at will).
if this is a valid use case where exposing the handle simplifies things for a developer who wants to directly use C++ we can definitely consider exposing it. however if the only reason it's required is to read a raw resource the cost/benefit of exposing it may fall on the side or more potential cost than benefit if you can read the model in java and pass it down to the C++ layer and everything else is done there.

@dizcza This PR should provide the necessary pieces to run NMS in the model and return bounding boxes and keypoints scaled to the original image.
https://github.com/microsoft/onnxruntime-extensions/pull/605
I have manually tested. Need to add unit tests.
I think you should be able to take vision.py and general.py from the PR and drop them into an existing installed onnxruntime-extensions python package under the tools/pre_post_processing/steps directory, replacing the old versions. After that tutorials/yolov8_pose_e2e.py from the PR would hopefully work as-is.
Alternatively clone the repo, checkout the PR branch, and run pip install -e . from the top level directory of the repo for the python package to point to the repo.
We don't have a custom operator to draw the keypoints, so default is to output essentially the same data as the original model for the selected bounding boxes with all the co-ordinates scaled to match the original image. Details in the comments in the tutorial. Only difference is the value for the selected class is included in the output, which is redundant when there's only one class, so there's 57 elements instead of the 56 in the exported onnx model.
Pre-processing supports providing bytes from a jpg or png. You can adjust the pre/post processing pipelines as needed. They're defined directly in the e2e tutorial for simplicity.

@skottmckay wow, that's quite a lot of work!
I tested it, and it works as expected. I have two questions then:

Would it work out of the box with the C++ Onnxruntime extensions, as described on the official docs https://onnxruntime.ai/docs/extensions/#c?
How to feed uint8 frames of shape (3, W, H) from a videocapture as the input to this exported model?

How to feed uint8 frames of shape (3, W, H) from a videocapture as the input to this exported model?

I think in this case I don't need the first step, ConvertImageToBGR, in the pre-processing steps. But the following steps refer to this node and substituting them for image, as the model input name, doesn't help.

Basically this puts the pre/post processing inside the model, so the C++ code (or GPU/NPU with NNAPI/CoreML) can handle it.
You only need extensions at runtime for custom operators like DecodeImage/EncodeImage/DrawBoundingBoxes.
If you have already decoded to {C, H, W} you don't need DecodeImage. Assuming you handle any actions using the bounding boxes/key points you don't need any other custom ops. If that is the case, extensions is providing the tools to put the pre/post processing in the model but doing it using ONNX operators, so no runtime dependency on the onnxruntime extensions packages.
Assuming the channel order matches the model (most of RGB but some are BGR) you can

change the model input name and shape to whatever you want - line 47

the first pre-processing step automatically uses the input name
shape can use names if you don't know the H and W values until runtime

e.g. inputs = [create_named_value("rgb_chw_bytes", onnx.TensorProto.UINT8, [3, "dynamic_H", "dynamic_W"])]
but if you do know the actual shapes it's better to use them

remove the ConvertImageToBGR
add layout=CHW to the Resize
LetterBox needs and update to support CHW though.

should be simple enough to add that. I'll see if I can add it to this PR

obviously remove the ChannelsLastToChannelsFirst step

For the sake of testing out the pre/post processing you could temporarily changing the input to be H, W, C.

I have hacked up a version that fills most of the gaps and seems to work with CHW byte input.

automatically connect first pre-processing step to graph input names

all the other steps auto-connect but I overlooked the very first one doesn't

adds layout awareness to LetterBox and ScaleNMSBoundingBoxesAndKeyPoints

both need to know where to get the H and W values from, and LetterBox needs to create the correctly ordered pads

add Identity step so you can pass in decoded bytes with shape

needs a Step to be able to connect up with ScaleNMSBoundingBoxesAndKeyPoints to determine original image shape
you can use this as the first pre-processing step. no perf cost - just allows connecting the shape up with the post-processing

I'll clean it up a bit more next week but hopefully works as-is for now.

All right, I tested your latest commit and was able to feed a decoded RGB image of shape (H, W, C) (I modified the code to this shape format).
When I tested it on C++, however, it always returns an empty tensor of shape (0, 57)... but at least it ran without errors. I'll look into this next week.

I find your constants strange:
        SelectBestBoundingBoxesByNMS(iou_threshold=0.7, score_threshold=0.25, max_detections=100, has_mask_data=True),

I'm using
        SelectBestBoundingBoxesByNMS(iou_threshold=0.5, score_threshold=0.70, max_detections=100, has_mask_data=True),

Also, max_detections is unclear: is it before or after the NMS suppression - is it input to NMS or its output? It should be clarified in the docs.

Updated to have max_boxes_per_class and max_detections parameters. The latter runs as the last step (if specified) to reduce the total to the best k boxes.

Thanks @skottmckay I think I can use your script as it is now. Compiling NMS into the graph hasn't yielded a significant performance increase, though I haven't tried testing the model on a video with lots of people - maybe in this case I'll see such a performance gain. Of course, the most important thing in including pre- and post-processing steps into a graph is to reduce the code (and headache) to do it manually either in C++ or Android.
The only thing left for me is quantization with ONNX, which I will probably postpone as the CPU desktop performance is already great, and Android inference will not get sped up by a factor of more than 2, which means FPS 1.5 --> 3.0 at most, which is still beyond our accepted range, which forces me to consider Android ML Kit that provides FPS 15.0 with float weights.

Did you try the XNNPACK EP? It may have some additional ARM fp32 kernels that speed things up. If ML Kit is also using CPU I'd be very surprised that it would be 5 to 10x faster than the equivalent model using ORT.
There's a PR to fill the 2 missing operators for the model to run on CoreML (Softmax and Split) and we'll try and add the one missing operator for NNAPI this weeek (Split). NNAPI is a bit hit and miss though as performance is very device specific.

Did you try the XNNPACK EP?

I've just tried XNNPACK and it hasn't yielded any performance gain at all. First of all, the docs at https://onnxruntime.ai/docs/execution-providers/Xnnpack-ExecutionProvider.html#usage are outdated as there is no such public method (AppendExecutionProvider) for Android platform. I found the addXnnpack function and used it like so:
    private fun createOrtSession(environment: OrtEnvironment): OrtSession {
        val so = SessionOptions()
        val po = mapOf<String, String>()
        so.addXnnpack(po)
        so.setOptimizationLevel(SessionOptions.OptLevel.ALL_OPT)
        return environment.createSession(readModel(), so)
    }

If ML Kit is also using CPU I'd be very surprised that it would be 5 to 10x faster than the equivalent model using ORT.

I meant ML Kit Pose Android implementation - it's shipped with a completely different model (MediaPipe Poses) that, on second thought, could be already quantized for the end user: just plug and play. It's limited though to one person in an image and that's quite a limitation for our app.

The documentation is for usage with the C/C++ API and is correct. I didn't realize the Android usage had diverged. I'll get the documentation updated to mention that and see if we can make the Android API a little more equivalent with a more generic AppendExecutionProvider function.
If it's this ML Kit model the input is also significantly smaller than the YOLOv8 model. 640x640 vs 480x360 is nearly 2.5x the amount of data to process. Throw in single detection vs. multiple and the performance difference starts to make more sense.

If it's this ML Kit model the input is also significantly smaller than the YOLOv8 model. 640x640 vs 480x360 is nearly 2.5x the amount of data to process. Throw in single detection vs. multiple and the performance difference starts to make more sense.

Yes, I understand: a model with a smaller input layer always wins overall...
What I don't understand is why XNNPACK does not speed up the original Yolo V8 Pose floating point model; I expect the conv and matmul operations to run faster with XNNPACK. I saw a profiling API method for such debugging, I'm not sure if I'm into this though.

 The XNNPACK EP has additional ones, but if the model doesn't require them (i.e. the MLAS kernels cover the majority of compute intensive operators) the performance wouldn't differ much. Both MLAS and XNNPACK are CPU based.


Fix temp.

Closes #725
Apparently temp() had no test coverage. A name change during the alpha stage of databroker 2.x development wasn't applied in the module databroker.v2. This fixes databroker.v2.temp and databroker.v1.temp.
In [1]: import databroker.v1

In [2]: import databroker.v2

In [3]: databroker.v1.temp()
OBJECT CACHE: Will use up to 6_290_642_534 bytes (15% of total physical RAM)
Out[3]: <databroker.v1.Broker at 0x7fcdbcc49f10>

In [4]: databroker.v2.temp()
OBJECT CACHE: Will use up to 6_290_642_534 bytes (15% of total physical RAM)
Out[4]: <Catalog {}>

The tests are passing now after the CI restart. I'll merge the PR to unblock downstream library maintainers. Maybe we'll need a new beta release soon.


ROS issues rs::depth_frame data topic and subscribes it at other nodes

The RGB image passed to the ros topic was very smooth, but there was a problem with subscribing to the depth information.
I want to subscribe “depth” from the ROS topic and calculate the 3D Points corresponding to the pixel coordinates using function "rs2_deproject_pixel_to_point" in the code below.
       rs2::frameset data = pipe.wait_for_frames();        data = align_to.process(data);        auto depth = data.get_depth_frame();        rs2_intrinsics intr = depth.get_profile().as<rs2::video_stream_profile>().get_intrinsics();         rs2_deproject_pixel_to_point(point, &intr, pixel, dist);
I have compiled the official ROS sample code successfully.
I would like to ask which topic I need to subscribe to so that I can use the depth_frame type of data at the subscribing node.thx:)

[Please Ignore - RealSense system comment]

Close this issue for lack of activity.
Please feel free to reopen if the issue is still relevant.


Double-click in ```{r x = TRUE} in markdown selects entire line

Should select just term double-clicked on

https://github.com/Microsoft/RTVS/commit/3e3a3608f8f8389744e7684a6ad45011ebd653fd


Re-launch Missing Member Page for 2019

-Hopefully by last week of July.
-Same interface design as 2017-2018 is fine.
-Ability to manually edit "Missing Member" status on Admin 2.0 will be critical.
-Auto-removing Missing Members if when a "town hall" event is added to the database for the timespan of this Congress (Jan 1, 2019 - Dec 31 2020)

Missing Member count is currently too high (as of 7/24).
E.g. Scott Peters had held a town hall in 2019. But he's appearing as an MM on the chart.

fixed scott peters in particular (it wasn't including the eventbrite counts because they didn't have a user associated with them. I fixed that. Let me know if you come across more

Jared Huffman
Diana DeGette
Scott Tipton

Last need is making sure the poster downloads work (or removing the poster downloads for now--they are fun but not critical for this launch)


install failed due to nativescript-angular version bump not published

Note that regular install fails due to nativescript-angular version not published (1.4.1)
for a quick workaround- you can fork 'template-hello-world-ng' repo.
change the nativescript-angular version back to 1.3.0
then change the install.js in nativescript-ng2-magic to pull from your fork of 'template-hello-world-ng'.
and run install.js again.
hope it helps someone.

Fixed here, note new repo and install:
https://github.com/wwwalkerrun/nativescript-ngx-magic


Add Codeberg

add Codeberg support to main page.

Thanks for this request (and sorry for being quite long to merge ...)


PreTokenGenerationTriggerEvent => claims_to_suppress : NoneType' object does not support item assignment

What were you trying to accomplish?
I'm currently trying to add some claims to suppress to the token in the event object. Working with manual way (without your lib).
Expected Behavior

No exception
Current Behavior

Exception
Possible Solution

Steps to Reproduce (for bugs)
def handler(event, context):
    event: PreTokenGenerationTriggerEvent = PreTokenGenerationTriggerEvent(event)
    user_pool_id = event.user_pool_id
    claims_override_details = event.response.claims_override_details
    claims_override_details.claims_to_suppress = ["Email"]

Environment

Powertools version used: 1.22.0
Packaging format (Layers, PyPi):
AWS Lambda function runtime: Python 3.9
Debugging logs

    "exception": "Traceback (most recent call last):\n  File \"/var/task/main.py\", line 130, in handler\n    claims_override_details.claims_to_suppress = []\n  File \"/var/task/package/aws_lambda_powertools/utilities/data_classes/cognito_user_pool_event.py\", line 487, in claims_to_suppress\n    self._data[\"claimsToSuppress\"] = value\nTypeError: 'NoneType' object does not support item assignment"

How to enable debug mode**

2021-12-05 21:13:10,898 aws_lambda_powertools.logging.logger [DEBUG] Adding filter in root logger to suppress child logger records to bubble up
2021-12-05 21:13:10,898 aws_lambda_powertools.logging.logger [DEBUG] Marking logger service_undefined as preconfigured

hey @filol thank you so much for filing this. Would you happen to have a copy of the event with any sensitive data redacted to help us reproduce/test it?
cc @michaelbrewer
Thank you!

I will have a look @filol and try to include it in our test coverage to be sure we don't regress.

@filol was claimsOverrideDetails in your event null? Would it be possible to post it with any values masked. I would be curious about the data structure.

@filol - i have PR (https://github.com/awslabs/aws-lambda-powertools-python/pull/878) that could fix this, if you could verify that the event did some from a real integration test and that is looks something like this:
{
  "version": "1",
  "triggerSource": "TokenGeneration_Authentication",
  "region": "us-west-2",
  "userPoolId": "us-west-2_example",
  "userName": "testqq",
  "callerContext": {
    "awsSdkVersion": "aws-sdk-unknown-unknown",
    "clientId": "71ghuul37mresr7h373b704tua"
  },
  "request": {
    "userAttributes": {
      "sub": "0b0a57c5-f013-426a-81a1-f8ffbfba21f0",
      "email_verified": "true",
      "cognito:user_status": "CONFIRMED",
      "email"<EMAIL_ADDRESS>    },
    "groupConfiguration": {
      "groupsToOverride": [],
      "iamRolesToOverride": [],
      "preferredRole": null
    }
  },
  "response": {
       "claimsOverrideDetails": null
  }
}

Hi @michaelbrewer, I don't have the event right now on the hand. I will come back to you with my sample monday

@michaelbrewer There is my event from aws, so yes claimsOverrideDetails is null by default
{
    "version": "1",
    "triggerSource": "TokenGeneration_RefreshTokens",
    "region": "eu-west-1",
    "userPoolId": "eu-west-1_xxxxxxxxx",
    "userName": "xxxxxxxxxx",
    "callerContext": {
        "awsSdkVersion": "aws-sdk-unknown-unknown",
        "clientId": "xxxxxxxxxxxxxxxxxxxxxxxxx"
    },
    "request": {
        "userAttributes": {
            "sub": "c7605922-xxxx-xxxx-a402-881ba30eeabc",
            "zoneinfo": "UTC",
            "cognito:email_alias": "xxxx",
            "cognito:user_status": "CONFIRMED",
            "email_verified": "true",
            "phone_number_verified": "false",
            "family_name": "xxxx",
            "email": "xxxx"
        },
        "groupConfiguration": {
            "groupsToOverride": [
                "xxx",
                "xxx",
            ],
            "iamRolesToOverride": [],
            "preferredRole": null
        }
    },
    "response": {
        "claimsOverrideDetails": null
    }
}


Extra lines included in doc comments

serde has this doc comment on its Serialize trait:
////////////////////////////////////////////////////////////////////////////////

/// A **data structure** that can be serialized into any data format supported
/// by Serde.
///

docs.rs doesn't render the //////////////////////////////////////////////////////////////////////////////// line, but rust-analyzer does:

According to the reference, lines starting with more than 3 forward slashes aren't doc comments:
OUTER_LINE_DOC :
/// (~/ ~[\n IsolatedCR]*)?

According to the reference, lines starting with more than 3 forward slashes aren't doc comments:
OUTER_LINE_DOC :
/// (~/ ~[\n IsolatedCR]*)?

According to the reference, lines starting with more than 3 forward slashes aren't doc comments:
OUTER_LINE_DOC :
/// (~/ ~[\n IsolatedCR]*)?

Fixed in #4184.


Embed videos with black screen

Script version 5.6 on Violentmonkey
playing on any website causes this problem https://steamcommunity.com/sharedfiles/filedetails/?id=3209239259

OS: Windows 10 22H2
Browser Firefox
Version 127.0

OS: Windows 10
Browser: Firefox
Tamper Monkey Version: 5.1.1
Adblock Thing Version: 5.6
Same issue.
Yes this is a dup but I wanted to stay informed.


Remove Underscore

The project will still maintain references to "underscore" as required by dependencies that depend on Underscore

Coverage remained the same at 100.0% when pulling 4677b47da2157148a5e43cd6c34aa7168ebebd6b on bugfix/remove-underscore-mixins into 18d60af1e23536a63a6c367713d675f11f2d0af1 on master.

 Remove underscore mixins module


Adding teams

https://github.com/MicrosoftDocs/office-docs-powershell/issues/7500

Docs Build status updates of commit 16b3357:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

skype/skype-ps/skype/Disable-CsOnlineSipDomain.md
:white_check_mark:Succeeded
View (skype-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit a7ee42a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

skype/skype-ps/skype/Disable-CsOnlineSipDomain.md
:white_check_mark:Succeeded
View (skype-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Release 2.0 - change to an ansible collection

Preparing release
@staticdev can the release drafter manage all the changes here or should we edit it ? I noted that is has overwritten my comments on the draft release last time.

Preparing release @staticdev can the release drafter manage all the changes here or should we edit it ? I noted that is has overwritten my comments on the draft release last time.

Release manager considers all PRs to main, except with tag skip-changelog as one change. In this case, makes sense to edit manually since it is a special case.

Preparing release @staticdev can the release drafter manage all the changes here or should we edit it ? I noted that is has overwritten my comments on the draft release last time.

Release Drafter considers all PRs to main, except with tag skip-changelog as one change. In this case, makes sense to edit manually since it is a special case.

Ok but I have a doubt : last time I edited the release draft to add a comment that this was the last release as a role, but this comment is not in the 1.8.0 release.
unless someone removed it ?

I expect troubles with galaxy when this PR is merged.
I haven't found doc about converting a role into a collection for galaxy (and keeping stats)
could be a nice exp for when transferring to nextcloud (but it will have a larger exposure so good stats may not be that hard to get)
I didn't found doc about transferring a role in galaxy to an other organization either...

I expect troubles with galaxy when this PR is merged.
I haven't found doc about converting a role into a collection for galaxy (and keeping stats) could be a nice exp for when transferring to nextcloud (but it will have a larger exposure so good stats may not be that hard to get)
I didn't found doc about transferring a role in galaxy to an other organization either...

@aalaesar I also had trouble with that. Probably there was a merge of PR after your edit and then Release-Drafter overwritten it. This is why currently I edit after the release to be 100% safe =p

@aalaesar I think you will need somehow to rebase to be able to merge since Github says there is a conflict in meta/main.yml

Lets go.


Add conversation apis to delegators

The Conversations API was recently added to the Ruby library and as a result, we need to update the Rails gem to include those services in def_delegators.

Hey @fabianrbz, you're totally right. Thanks for that. I removed the extra ones, and left just the #conversations method as it is defined in client.


Support access tokens and authorization codes

More info

+1 :)

@gurisko Just wondering if you had the chance to look into the iOS part of this.. thank you ;)

Hey @1N50MN14. I will be able to look at this only after February 8.

Cool thanks for the update @gurisko

Hi @gurisko , don't mean to nag, just wondering if you got the chance to look into this one - thank you ;)

It's been a long time @1N50MN14 . Are you still available to test this feature?

If anyone feels like it, pull request #31 is open for reviews. If there won't be any comments in around 12 hrs, it will be merged and released under 1.3.0.

hey @gurisko I thought you disappeared! ;) I paid $ to get the iOS part done few months ago and its tested maybe you want to take a look before merging your PR see if there's special in there?

@1N50MN14 I was too busy and I stopped to use this plugin so there was a long break. Now I am trying to finish the planned features. I would definitely like to have a look if you don't mind. And thank you for testing. Hopefully we will make it better and you can get back to our public releases :)

@gurisko Awesome! Yeah it would be great to get back to a public release I had a concern about the long term maintainability of the plugin. The invite link should be in your inbox, couple of notes:

I enforced useAccessToken in the iOS code; there's a minor issue where the arg isn't interpreted correctly and since my iOS skills are non-existent I simply enforced the use of access token.

A themeColor option is added to both iOS and Android.

For Android I faced an issue where the AccountKit activity window by default uses FB's blue color on the status bar which looks kind of weird, what I did is replace the activity tag in AndroidManifest.xml with: ```
And in style.xml under res/values I changed the statusbar color, I added the theme like so:
<resources>
<style name="AccountKitTheme" parent="Theme.AccountKit">
    <item name="com_accountkit_primary_text_color">@android:color/white</item>
    <item name="com_accountkit_status_bar_color">#888888</item>
    <item name="com_accountkit_button_background_color">#71b7e6</item>
</style>
</resources>

It might be a good idea to bake this into the plugin options (at least the status bar color) so that the experience is a bit more seamless.
Welcome back! :-)

Thank you @1N50MN14. I did a quick review of your code and the core functionality seems similar, so hopefully we won't introduce any bugs into our new release.
I will have a look at the theming next.

Hi @gurisko, Is theme option available in latest release? I need to change button color in plugin.


Performance markers logged from C++ no longer work in Android

Same problem reported before here https://github.com/facebook/react-native/issues/23771.
Description
These performance markers are not working anymore for Android using react-native 0.64.2:
NATIVE_MODULE_SETUP_START
NATIVE_MODULE_SETUP_END
RUN_JS_BUNDLE_START
RUN_JS_BUNDLE_END

React Native version:
react-native version 0.64.2
Steps To Reproduce

Init a new RN project on any version above 0.64.2
Add the following code to start logging markers:

ReactMarker.addListener(new ReactMarker.MarkerListener() {
  @Override
  public void logMarker(
          ReactMarkerConstants name, @Nullable String tag, int instanceKey) {
    Log.d("ReactNativeEvent", "name: "+name.name() + " tag: "+tag);
  }
});

Expected Results
It should log the markers stated above.

I figured that we have this bug when react-native-navigation is in the project. As this is related to react-native-navigation not react-native i am closing the issue in this repo.


Please, help

0160:err:ole:CoUninitialize Mismatched CoUninitialize after starting game, but if I change graphics API to Vulkan, it returns error: Cannot create  a Win32 Vulkan surface -9

I've never faced this before. Can you dump the full output? add a screenshot as well.


⚠ Bofrost.at has degraded performance

In b271f68, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2269 ms

Resolved: Bofrost.at performance has improved in e67cb54 after 6 minutes.


wasm client gets into weird state after idling for a while

while testing spaceships example, if i leave a chrome wasm client connected for a while and come back later, it is in some weird state where position warps around a bit. will try and get a video and update this issue.

What do you mean by "a while"?

with just 1 client connected, after a while of inactivity the server spews a bunch of:
System 'bevy_ecs::event::event_update_system<.......>' has not run for<PHONE_NUMBER> ticks. Changes older than<PHONE_NUMBER> ticks will not be detected.
then when i try to play, something is pretty messed up with position and rotation.
will try to get a more accurate idea of how long soon.

The System 'bevy_ecs::event::event_update_system<.......>' has not run for<PHONE_NUMBER> ticks. Changes older than<PHONE_NUMBER> ticks will not be detected. logs are expected; bevy's change detection doesn't work after more than ~15minutes.
However lightyear should recover seamlessly from this. It doesn't recover by itself?

ok yeah the "system has not run" thing is a red herring, it happened before i saw those messages this time.
i restarted the server, connected a chrome wasm client, flew about smoothly, then unfocussed the chrome window for 15 mins.
when i came back, the normally buttery smooth movement looks like this:
https://github.com/cBournhonesque/lightyear/assets/29747/011aa985-cc3e-4f31-91e5-d9d2fc77674a
nothing logged on client or server to give a clue.

ah, this weird behaviour is the same if a leave a native client alone for 15m too..

Yep thats defo tick wrapping then, idk why the existing fix doesnt work tho

could you elaborate on what the tick wrapping issue is?

Either the bevy ticks (u32) or the lightyear ticks (u16) are wrapping around after a certain amount of time (around 15minutes for lightyear) which might be causing issues when comparing ticks.
To avoid this, I have:

unit tests that check that tick wrapping is handled (but they are probably not comprehensive)
a system that runs regularly and updates internal ticks to account for the wrapping

I refactored some of the replication code recently, maybe it broke some of the tick-wrapping specific logic.
The issue in your video is for the white ball's movement, right?

hard to pinpoint what is wrong in the video – the ball is janky, the player movement feels stuttery, the bullets appear to move backwards at some point.

Fixed this in https://github.com/cBournhonesque/lightyear/pull/497 (only in cb/0.16 branch)
Some of the tick wrapping logic had indeed been broken with the replication refactor.
I added a stronger unit test so hopefully these kind of silent breakages won't resurface


"W" is truncated in the "Warning" log-level chip in the select button on Windows.

Bug
As the title says

yscope-log-viewer version
b15d9e082f5b0dc45c97e42293f0b1b50fa619ba
Environment
Windows 11 Pro 24H2 26120.2702 (Windows Feature Experience Pack 1000.26100.42.0)
Microsoft Edge Version 132.0.2957.55 (Official build) stable app, beta channel (64-bit)
Node.js: v22.11.0
Reproduction steps

Load the log viewer with a demo file: http://localhost:3010/?filePath=https://yscope.s3.us-east-2.amazonaws.com/sample-logs/yarn-ubuntu-resourcemanager-ip-172-31-17-135.log.1.clp.zst
Click the log-level select button in the bottom-right corner and select "TRACE", which will have all log-levels more severe than "TRACE" selected, which includes "WARN".
Observe the "W" getting truncated in the chip in the button.

Related to #83 which made --ylv-ui-font-family as the global UI font: on Windows, which remove application of "Inter" on the log level chip.


encoding H.264, bad packets

I have some simple code to pull frames from a video, manipulate them, and then encode them back to video.  When I use 'mpeg4' to encode the output video it works fine (and it's fast! yay!), but when I switch to 'h264', muxing the output packet throws this error:
Argument 'packet' has incorrect type (expected av.packet.Packet, got NoneType)
It looks like the encoding step produces bad packets or none at all, am I missing something here?
the short version of the code:
# open output video file for writing
output_file = av.open(outputname,'w','mp4')
# add a video stream to the output at 5fps
output_video_stream = output_file.add_stream('h264', 5)

#... a bunch of code to loop through frames, and process images with openCV

    # store the numpy array as a frame
    frame_out.planes[0].update(newimg)
    # encode an output frame
    output_packet = output_video_stream.encode(frame_out)
    # code breaks here when using 'h264'
    output_file.mux(output_packet)

The h264 encoder works fine when I convert videos using ffmpeg on the command line, so it's there, but not playing well with PyAV.

H.264 has B-frames, so it has to buffer up a few frames before giving you packets. Stream.encode() can give you None, and you need to test for that.

Hi Mike
thanks for the quick reply!  That solved it.  I was testing with 10 frames,
and that wasn't enough to produce a packet.
cheers
dave
On Thu, Jul 14, 2016 at 11:01 AM, Mike Boers<EMAIL_ADDRESS>wrote:

H.264 has B-frames
https://en.wikipedia.org/wiki/Video_compression_picture_types#Bi-directional_predicted_frames.2Fslices_.28B-frames.2Fslices.29,
so it has to buffer up a few frames before giving you packets.
Stream.encode() can give you None, and you need to test for that.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mikeboers/PyAV/issues/179#issuecomment-232743731, or mute
the thread
https://github.com/notifications/unsubscribe/AC2wn1FyV1_hI3s7Ec__3Bo2aMtfZtPSks5qVnmTgaJpZM4JMr1g
.


is ft_regex_tokenizer ready for use?

Hello,
I use Sparklyr 0.7.0-9106 and I see that a function ft_regex_tokenizer is available.
However, I do not see this ft transformer being mentioned on the main website https://github.com/rstudio/sparklyr/blob/master/vignettes/guides-mllib.Rmd#transformers
That does mean the function is not ready for production use?
Thanks again for the clarification. I really need to use it.

@kevinykuo @javierluraschi I am asking this because I have noticed the following problem.
I have a parquet dataframe with about 35 million rows.
Running the following code will work as expected:
df_new %>%
  head() %>%
  ft_regex_tokenizer(input.col='myweirdstring',
                     output.col = 'myoutput', pattern = '\\b[^\\d\\W]+\\b', gaps =FALSE) %>%
  select(myoutput, myweirdstring) %>%
  mutate(myword = explode(myoutput)) %>%
  group_by(myword) %>%
  summarize(mycount = n())

# Source:   lazy query [?? x 2]
# Database: spark_connection
   myword    mycount
   <chr>       <dbl>
 1 country       2.00
 2 original      4.00
 3 nice          5.00
 4 hello         2.00

Here I simply use ft_regex_tokenizer on a character column (looking for words but not numbers) and then I explode the result so that I can actually count the occurrences of each word in the dataset. Note that I use head() here.
Now here is the big issue: if I run that same code on the full 35 million rows (that is without the head() ) I immediately get the error:
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task 9 in stage 13.0 failed 4 times, most recent failure: Lost task 9.3 in stage 13.0 (TID 1321, 10.11.1111.111, executor 2): org.apache.spark.SparkException: Failed to execute user defined function($anonfun$createTransformFunc$2: (string) => array<string>)
what does that even mean??? is this a bug??
Thanks again for your help!!!!

@kevinykuo @javierluraschi actually after a few tests, using head(5000) will generate the same error which likely rule out some possible memory issues I guess. Do you have any idea?
What does the  function($anonfun$createTransformFunc$2: (string) => array<string>) error mean?
Thanks!

@randomgambit I'd try to identify the rows that contain the problematic string and eyeball them.
ft_regex_tokenizer() is good to go and this looks like a Spark or data issue.

thanks. I would add to the documentation that all of the spark ml functions are available. not only the one you mention on the sparklyr website because this is confusing (at least to me).
thanks!!

@kevinykuo can I please ask a follow up question related to that?
I have a variable that contains strings like "KEVIN10 KUO SPARK KUO30 HELLO"
Assume I want to filter the rows that contain KUO somewhere in the string.
Then doing
df %>% filter(mycolumn %rlike% 'KUO')
works as expected.
HOWEVER, if I want to filter out the rows where I have KUO immediately followed by a number (such as KUO30, doing the following:
filter(mycolumn %rlike% 'KUO\\d+')
never works...
Is there something special with the syntax I must use here?

Try "KUO[0-9]" for the pattern

@kevinykuo Okay thanks that worked!
So now we are back to the fundamental question of this issue: lets assume I want to extract the number that is next to 'KUO'.
I tried the following:
df_new  %>%
ft_regex_tokenizer(input.col='mystring',
                              output.col = 'myoutput',
                              pattern = '(?<=KUO)([0-9]+)', gaps =FALSE) %>%
  mutate(myword = explode(myoutput))

but this always returns an empty frame. Are the lookarounds supported by ft_regex_tokenizer ?

ft_regex_tokenizer() should take Java regular expressions so I'd google with that

@kevinykuo I believe I am using the right syntax here... even
pattern = "(?<=KUO)([0-9])"
does not work.
In contrast, using
pattern = ".*(?<=.*KUO)([0-9]).*"
seem to run forever... I am lost here :(

HOLY COW. I found the bug
running
pattern = "(?<=kuo)([0-9])" with lower case kuo will match against all the upper case KUOs in the data). Any reason for that?
I see some hidden option in ft_regex_tokenizer(to_lower_case = TRUE) but no documentation about it on https://www.rdocumentation.org/packages/sparklyr/versions/0.6.4/topics/ft_regex_tokenizer. Could it be it?
Thanks!

Check out the latest version of the docs https://www.rdocumentation.org/packages/sparklyr/versions/0.7.0/topics/ft_regex_tokenizer


Brazil census data

Census data could be used to generate a country-wide address list in Brazil, but it would require a dedicated script to parse and process. I will describe the situation so you can decide if this makes sense for openaddresses.
There are two datasets that would need to be combined:

An address list in each municipality (or, sometimes, a district within a municipality). For rural addresses, a GPS reading is included. For urban addresses, the id number of a city block and sidewalk segment is given. (This is at ftp://ftp.ibge.gov.br/Censos/Censo_Demografico_2010/Cadastro_Nacional_de_Enderecos_Fins_Estatisticos/)
Vector maps with the sidewalk segments. (This is at ftp://geoftp.ibge.gov.br/recortes_para_fins_estatisticos/malha_de_setores_censitarios/censo_2010/base_de_faces_de_logradouros/)

Thus, at least the block corresponding to each address can be determined.
I can't say much about the quality of the data; I suppose it varies a lot throughout the country. One issue is that the geospatial information is often not precisely aligned with OSM, see the pictures below. In any case, this is the best data one can hope to gather in Brazil (apart from a few bigger cities that may release their own addresses) at least until the next census comes out, in 5 to 10 years.

This sounds similar to US Census interpolated address ranges ala Tiger.
+1 for OpenAddresses adding support for this data type (at least interpolation points at each end of line).

On Dec 30, 2016, at 06:03, astoff<EMAIL_ADDRESS>wrote:
Census data could be used to generate a country-wide address list in Brazil, but it would require a dedicated script to parse and process. I will describe the situation so you can decide if this makes sense for openaddresses.
There are two datasets that would need to be combined:
An address list in each municipality (or, sometimes, a district within a municipality). For rural addresses, a GPS reading is included. For urban addresses, the id number of a city block and sidewalk segment is given. (This is at ftp://ftp.ibge.gov.br/Censos/Censo_Demografico_2010/Cadastro_Nacional_de_Enderecos_Fins_Estatisticos/)
Vector maps with the sidewalk segments. (This is at ftp://geoftp.ibge.gov.br/recortes_para_fins_estatisticos/malha_de_setores_censitarios/censo_2010/base_de_faces_de_logradouros/)
Thus, at least the block corresponding to each address can be determined.
I can't say much about the quality of the data; I suppose it varies a lot throughout the country. One issue is that the geospatial information is often not precisely aligned with OSM, see the pictures below. In any case, this is the best data one can hope to gather in Brazil (apart from a few bigger cities that may release their own addresses) at least until the next census comes out, in 5 to 10 years.
―
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@nvkelso These are not address ranges. They are more like the Japanese dataset: individual addresses geolocated up to city block (as opposed to parcel or rooftop).

Interesting!
Can you provide a fully formed Brazil example address, and what this dataset would allow? Is it 1:1 or missing something?

On Dec 30, 2016, at 10:27, astoff<EMAIL_ADDRESS>wrote:
@nvkelso These are not address ranges. They are more like the Japanese dataset: individual addresses geolocated up to city block (as opposed to parcel or rooftop).
―
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I have attached a little example of what one can get (a part of Curitiba)
c-uniq.csv.gz.  Judging by the amount of dots, I'd say the dataset is fairly complete.
In another case (Porto Alegre) there are 638K dwellings (house number + unit), and 288K unique buildings (i.e., after discarding units). Again, that seems about right for a fairly dense city of 1.5 million.

I have attached a little example of what one can get (a part of Curitiba)
c-uniq.csv.gz.  Judging by the amount of dots, I'd say the dataset is fairly complete.
In another case (Porto Alegre) there are 638K dwellings (house number + unit), and 288K unique buildings (i.e., after discarding units). Again, that seems about right for a fairly dense city of 1.5 million.

Wow, that's amazing! Let's get this added :)
Import strategy: Since there is some manual process involved, I suggest breaking the import into sources per state (27 total, 26 for states and one for the capital).
Accuracy: It looks like a lot of these addresses are stacked on top of each other (so there are multiple addresses per "street block"), with "block" level precision. @migurski is there an accuracy config to use for this? Looks like 4 or 5 now, or maybe add one for 6 to indicate block level?
Units: Looks like many of the "stacked" records have an empty unit property in the CSV table. Is there a unit available in the source data?
Projection: For the alignment issues, we sometimes see that in other sources. It is possible to override the projection information specified in the shapefile in the OpenAddress src config if we can figure out a better one to use after the per-state files are generated.

+1, very interested in Brazil countrywide data. Units could be useful for libpostal purposes as well.

This data is great. I would say "5" for accuracy, and maybe we can reevaluate what we're using for accuracy anyway. The integer values are not sure, it'd be nice to just use the strings.

OK, great to hear you like this. Let's discuss the concern with units.
In the vast majority of cases, the "units" in this dataset refer to apartments of an apartment tower, and not different buildings within the same parcel or different street-level entrances to a given building. I imagined you would not want to retain this level of detail — are you saying you actually do want to keep it?
I should also remark that every single building (residential or not) appears in the listing, but only residential units are listed.

We do have a place in the OA schema to put units, so if they exist in the data it wouldn't hurt to add them! For @thatdatabaseguy’s purposes, I believe that the presence of units has also helped with libpostal training data sets.

Yes, the libpostal parser takes data sets like OpenAddresses that are already separated into fields, recreates what the full address would look like, and then trains a model to parse full addresses (from geocoder input, CSVs, etc) into components. Having the unit information helps the system handle more types of input, for example recognizing that "Apto 202" is a unit, and thus most geocoding systems can ignore it for the purposes of determining the lat/lon (sometimes they can get confused by extra information like this, so it's useful to determine which parts of a geocoder query are important). They're not mission-critical but nice to have!

One more question before I am satisfied with the script: Should we try to clean up the original data in any way? (By the way, is a CSV enhancer (https://github.com/openaddresses/machine/issues/283) still on the roadmap?)
This particular dataset would benefit mostly from the following two things, the second of which may be quite important if this is to be used as training data:

Titlecase and add diacritics to street names.
Clean up the number_suffix field (which is appended to the number in the current form of https://github.com/openaddresses/openaddresses/pull/2315). This field is used for various random purposes:

An actual suffix for the number, like "A" in "7A"
"SN" for "sem número", which is most commonly rendered "s/nº"
"KM" in rural areas indicates not a house number but the milestone where the property is located.  Should typically be prepended, not appended, to the number.
Sometimes unit information is given in the number_suffix, the most common cases being "FRENTE" and "FUNDOS" (front side versus back side of the parcel)
Various annotations:

"CASA" or "ED", to indicate it's a single-unit respectively multi-unit building. Safe to discard.
The name of a utility or government agency ("DMAE", "CEEE", "FUNASA", etc.) indicates an "issuing authority" for the number.  Common in informal or very recent settlements. This I'd say should be kept.
Many other random annotations, which usually seem safe to discard.

I would prefer to see caching/downloading scripts be as "dumb" as possible and to constrain themselves to only downloading the data from the source in as raw of a format as possible. Doing this means that we end up with the raw data downloaded to our system to start with and can adjust the transform/conform process to include different data later.
The CSV enhancer/cleaner steps are still on our roadmap, and I think these changes you list would be an interesting set of "fixes" to apply to the output.

Agreed, it's usually reasonable to keep the raw data raw and update the transform over time. The number_suffix changes are a bit more intricate than what the OA transforms can currently handle, but AFAICT from the São Paulo data, these cases are fairly infrequent and shouldn't affect consumers much.
Libpostal has its own cleanup/normalization which can accommodate all of the above, so no worries there, but useful to know the edge cases.
Re: diacritics, it looks like they're already stripped in the source, no? For certain known words like "Praca" => "Praça" it should be possible to recover them, and libpostal has dictionaries for Portuguese that can handle cases like that, but I'd think it would be a bit more difficult for random streets (not impossible, could be done by building an index of sans-diacritics forms to their most common unnormalized forms using a data set that's known to use proper diacritics e.g. OSM roads).  Hopefully this solves the issue where some entire cities are missing.
By the way, I was automatically kicked out of the organization sometime ago because I didn't have two-factor authentication set up. Can someone include me again?

I'll try running my old script. Hopefully this solves the issue where some entire cities are missing.
By the way, I was automatically kicked out of the organization sometime ago because I didn't have two-factor authentication set up. Can someone include me again?

@astoff I tried to run the script, but the 2019 version changed a bit. The files are packed differently and the internal structure changed too. In case you need to download all the files again I'm seeding than as torrents. Download might be faster than IBGE FTP: https://github.com/vgeorge/cnefe

@astoff I tried to run the script, but the 2019 version changed a bit. The files are packed differently and the internal structure changed too. In case you need to download all the files again I'm seeding than as torrents. Download might be faster than IBGE FTP: https://github.com/vgeorge/cnefe