Upgrading Waypoint proxies to 1.24.0 causes brief outage with filter_chain_not_found errors

Is this the right place to submit this?

[X] This is not a security vulnerability or a crashing bug
[X] This is not a question about how to use Istio

Bug Description
I am running Istio in Ambient mode and have followed the upgrade guide to upgrade Istio from version 1.23.2 to version 1.24.0. The steps I followed in order were:

Upgrade istio-base helm chart
Upgrade istio-cni helm chart
Restart all meshed workloads (not necessary but did so for good measure)
Upgrade istio-ztunnel helm chart
Edit Waypoint Gateway proxies to change label istio.io/rev: 1-23-2 to istio.io/rev: 1-24-0

Everything continued to work as normal until step 5. As soon as I upgraded the Waypoint proxies, all ingress traffic to the cluster started failing with the same type of error from the Waypoint proxy:
{
  bytes_received 0
  bytes_sent 0
  downstream_local_address <IP_ADDRESS>:8877
  downstream_remote_address <IP_ADDRESS>:58622
  http {
    status_code 0
  }
  response_code 0
  response_code_details filter_chain_not_found
  response_flags NR
  start_time 2024-11-13T16:51:29.341Z
}

This lasted for approximately 10 minutes until everything started working as normal again. Clearly something happened in the background that fixed the cause of the issue but I'm struggling to work out what it was and how to avoid this happening on a production cluster.
Please note that the helm chart upgrades were deployed via ArgoCD rather than via the Helm CLI.
Version
➜ istioctl version
client version: 1.24.0
istiod version: 1.23.2
istiod version: 1.23.2
istiod version: 1.23.2
istiod version: 1.24.0
istiod version: 1.24.0
istiod version: 1.24.0
data plane version: 1.24.0 (10 proxies)

➜ kubectl version
Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.9

➜ istioctl proxy-status
NAME                               CLUSTER        CDS              LDS              EDS              RDS         ECDS        ISTIOD                             VERSION
waypoint-7445477dcb-672l2.load     Kubernetes     SYNCED (36s)     SYNCED (36s)     SYNCED (20m)     IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-v8pds     1.24.0
waypoint-7445477dcb-cp4tl.load     Kubernetes     SYNCED (36s)     SYNCED (36s)     SYNCED (20m)     IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-69dq6     1.24.0
waypoint-7445477dcb-v6rcv.load     Kubernetes     SYNCED (36s)     SYNCED (36s)     SYNCED (20m)     IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-v8pds     1.24.0
ztunnel-49ff7.istio-system         Kubernetes     IGNORED          IGNORED          IGNORED          IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-v8pds     1.24.0
ztunnel-8swgn.istio-system         Kubernetes     IGNORED          IGNORED          IGNORED          IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-v8pds     1.24.0
ztunnel-fwkpw.istio-system         Kubernetes     IGNORED          IGNORED          IGNORED          IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-69dq6     1.24.0
ztunnel-n2khr.istio-system         Kubernetes     IGNORED          IGNORED          IGNORED          IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-v8pds     1.24.0
ztunnel-s9k9d.istio-system         Kubernetes     IGNORED          IGNORED          IGNORED          IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-j7fx4     1.24.0
ztunnel-vfs55.istio-system         Kubernetes     IGNORED          IGNORED          IGNORED          IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-v8pds     1.24.0
ztunnel-zfxm9.istio-system         Kubernetes     IGNORED          IGNORED          IGNORED          IGNORED     IGNORED     istiod-1-24-0-5b48fc56d7-j7fx4     1.24.0

Additional Information
istio-namespace: istio-system
full-secrets: false
timeout (mins): 30
include: {  }
exclude: { Namespaces: kube-node-lease,kube-public,kube-system,local-path-storage }
end-time: 2024-11-13 17:16:52.371509 +0000 UTC
Cluster endpoint: https://##################
CLI version:
version.BuildInfo{Version:"1.24.0", GitRevision:"Homebrew", GolangVersion:"go1.23.3", BuildStatus:"Homebrew", GitTag:"1.24.0"}
The following Istio control plane revisions/versions were found in the cluster:
Revision default:
no running Istio pods in namespace "istio-system"
Revision 1-23-2:
&version.MeshInfo{version.ServerInfo{Component:"istiod", Revision:"1-23-2", Info:version.BuildInfo{Version:"1.23.2", GitRevision:"da0b3f29e9f9eef387691bdf5870a6d665c704c9", GolangVersion:"", BuildStatus:"Clean", GitTag:"1.23.2"}}, version.ServerInfo{Component:"istiod", Revision:"1-23-2", Info:version.BuildInfo{Version:"1.23.2", GitRevision:"da0b3f29e9f9eef387691bdf5870a6d665c704c9", GolangVersion:"", BuildStatus:"Clean", GitTag:"1.23.2"}}, version.ServerInfo{Component:"istiod", Revision:"1-23-2", Info:version.BuildInfo{Version:"1.23.2", GitRevision:"da0b3f29e9f9eef387691bdf5870a6d665c704c9", GolangVersion:"", BuildStatus:"Clean", GitTag:"1.23.2"}}}
Revision 1-24-0:
&version.MeshInfo{version.ServerInfo{Component:"istiod", Revision:"1-24-0", Info:version.BuildInfo{Version:"1.24.0", GitRevision:"8825a6b7f8c9a2d66005a5f8b64e98aaee0dda99", GolangVersion:"", BuildStatus:"Clean", GitTag:"1.24.0"}}, version.ServerInfo{Component:"istiod", Revision:"1-24-0", Info:version.BuildInfo{Version:"1.24.0", GitRevision:"8825a6b7f8c9a2d66005a5f8b64e98aaee0dda99", GolangVersion:"", BuildStatus:"Clean", GitTag:"1.24.0"}}, version.ServerInfo{Component:"istiod", Revision:"1-24-0", Info:version.BuildInfo{Version:"1.24.0", GitRevision:"8825a6b7f8c9a2d66005a5f8b64e98aaee0dda99", GolangVersion:"", BuildStatus:"Clean", GitTag:"1.24.0"}}}
The following proxy revisions/versions were found in the cluster:
Revision 1-24-0: Versions {1.24.0}
Fetching logs for the following containers:
###################### (obfuscated for security)
Fetching Istio control plane information from cluster.
Fetching CNI logs from cluster.
Running Istio analyze on all namespaces and report as below:
Analysis Report:
Info [IST0102] (Namespace datadog) The namespace is not enabled for Istio injection. Run 'kubectl label namespace datadog istio-injection=enabled' to enable it, or 'kubectl label namespace datadog istio-injection=disabled' to explicitly mark it as not needing injection.
Info [IST0102] (Namespace default) The namespace is not enabled for Istio injection. Run 'kubectl label namespace default istio-injection=enabled' to enable it, or 'kubectl label namespace default istio-injection=disabled' to explicitly mark it as not needing injection.
Info [IST0102] (Namespace emissary-system) The namespace is not enabled for Istio injection. Run 'kubectl label namespace emissary-system istio-injection=enabled' to enable it, or 'kubectl label namespace emissary-system istio-injection=disabled' to explicitly mark it as not needing injection.
Info [IST0102] (Namespace keda) The namespace is not enabled for Istio injection. Run 'kubectl label namespace keda istio-injection=enabled' to enable it, or 'kubectl label namespace keda istio-injection=disabled' to explicitly mark it as not needing injection.
Info [IST0102] (Namespace kube-node-lease) The namespace is not enabled for Istio injection. Run 'kubectl label namespace kube-node-lease istio-injection=enabled' to enable it, or 'kubectl label namespace kube-node-lease istio-injection=disabled' to explicitly mark it as not needing injection.
Info [IST0102] (Namespace kube-public) The namespace is not enabled for Istio injection. Run 'kubectl label namespace kube-public istio-injection=enabled' to enable it, or 'kubectl label namespace kube-public istio-injection=disabled' to explicitly mark it as not needing injection.
Info [IST0102] (Namespace kube-system) The namespace is not enabled for Istio injection. Run 'kubectl label namespace kube-system istio-injection=enabled' to enable it, or 'kubectl label namespace kube-system istio-injection=disabled' to explicitly mark it as not needing injection.
Info [IST0118] (Service load/admin-portal) Port name  (port: 80, targetPort: 8080) doesn't follow the naming convention of Istio port.
Creating an archive at /Users/###############
Time used for creating the tar file is 387.589959ms.
Cleaning up temporary files in /var/folders/km/txfjl8cj3j15s5wzrnv6l36r0000gq/T/bug-report.
Done.

At what point did you install the new istiod helm chart?

At what point did you install the new istiod helm chart?

Good point, I forgot to include that step. I installed Istiod 1.24.0 after istio-base was ugpraded. It was installed alongside istiod-1.23.2.

I've conducted some further tests and think I have found a stable upgrade path; rather than changing the revision label on Waypoint proxy (named waypoint) from istio.io/rev=1-23-2 to istio.io/rev=1-24-0, I instead deployed a new  Waypoint proxy, named waypoint-1-24-0, alongside the existing one with the label istio.io/rev=1-24-0. I allowed the new Waypoint pods to spin up then edited the label on the namespace (I use a single Waypoint per namespace) from istio.io/use-waypoint=waypoint to istio.io/use-waypoint=waypoint-1-24-0. This approached seemed stable and didn't appear to cause any downtime.
Further, I found that, for my initial approach where I edited the revision of the existing Waypoint, the issue seemed to resolve when I scaled down the istio-1-23-2 deployment to 0 pods. I.e. it was the presence of both 1.23.2 and 1.24.0 istiod pods that seemed to cause the issue.

I can reproduce and understand the root cause. Its fairly subtle..
When the old revision is the leader of status, it will write IP address to the Gateway address. That in turn messes with things


Add unit and integration tests

This PR provides a basic setup for unit and integration tests. It also contains example of the integration tests.
To do:

Write more tests 😄

Note: The branch was branched out of feature/use-namespace branch which has the latest code.

yeah I think this will be a good thing to get going!


GCFFlasher_internal no devices found

Hello,
I have a Conbee III  attached to a raspberry 4 on /dev/ttyUSB0.
I managed to run the container successfully by setting the baud rate option, and seems that the communication is fine (at least I can see the firmware id in the gateway page of Phoscon app), but if I run the command GCFFlasher_internal from inside the container I get "No device found".
# GCFFlasher_internal -l
no devices found
Path              | Serial      | Type
------------------+-------------+---------------

Any idea?
Thank you
William

It also happened to me with the latest "stable" version. If I use the image: deconzcommunity/deconz:2.23.02 my Raspbee is not only detected in the host but also in the docker container!

I don't know why this behavior has changed, but I'm leaning towards starting to recommend users to not use this container for firmware patching since Docker adds a layer of complexity into flashing a device connected to the host.
Instead it would make more sense to:

Download the correct flasher binary from here: https://github.com/dresden-elektronik/gcfflasher/releases and unpack.
Download the wanted firmware file from https://deconz.dresden-elektronik.de/deconz-firmware/ and place it next to the unpacked flasher binary.
If running, stop the deCONZ docker container to make sure nothing is using the device.
./GCFFlasher4 -l
./GCFFlasher4 -f firmware_file -d /dev/yourdevice -t 60 - modify device to what is listed from step 4.

Any issues with this method should be reported in the GCFflasher repo, since firmware flashing support is getting a bit out of scope for this project :)


Regex parse error when try to match string literals "{{xxx}}"

My code in Python works like this:
>>> content = "{{ static_prefix }}/home/js/asdf.js";
>>> a = re.compile("{{\\s*static_prefix\\s*}}");
>>> a.sub("/static", content)
'/static/home/js/asdf.js'

If I code the same logic with regex crate,
use regex::Regex;

fn main() {
    let content = "{{ static_prefix }}/home/js/asdf.js";
    let reg = Regex::new("{{\\s*static_prefix\\s*}}").unwrap();
    println!("{}", reg.replace(content, "/static"));
}

I got error:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
regex parse error:
    {{\s*static_prefix\s*}}
    ^
error: repetition operator missing expression

But if I escape {{ with backslash, it works:
use regex::Regex;

fn main() {
    let content = "{{ static_prefix }}/home/js/asdf.js";
    let reg = Regex::new("\\{\\{\\s*static_prefix\\s*\\}\\}").unwrap();
    println!("{}", reg.replace(content, "/static"));
}

I tried in Python and Go, to match this string, I don't need to escape {{ and }}, but with this regex crate in Rust, I need to put the extra escape, otherwise I got the above "parse error", regex can't compile.
Is this a bug?

Regular expressions are not generally portable across different regex libraries, and compatibility with other regex implementations is an explicit non-goal of this crate, so I don't think this is a bug.

Also, I would suggest using raw strings. You won't need to escape the backslashes.

@BurntSushi I need to read this regexp string from a configuration file, can't create a raw string directly, but thank you for the suggestion.
Also @ethanpailes thank you for the explanation.


Add ability to read an item from a shopcart

I implemented the function of read_item().
It finds a shopcart, if it exists, and goes through its items to see if there's any item matching item_id provided.
If so, it returns a JSON response of the item. If not, it returns 404.

Just merged some conflicts; it should be ready to go!

Merged conflicts and it's now ready for your review again. Thanks!

Updated with a JSON response, thanks!


simde/arm/neon.h fails to compile for an ARM64 target with Visual C++

Including "simde/arm/neon.h" fails to compile on Visual C++ for an ARM64 target since some of the SIMD_CONSTIFY_2_, SIMDE_CONSTIFY_4_, SIMDE_CONSTIFY_8_, and SIMDE_CONSTIFY_16_ macros used in some of the header files in the simde/arm/ directory give the following errors:
A link to a test C++ source file including the simde/arm/neon.h and the compiler result on Visual C++ for an ARM64 target can be found at https://godbolt.org/z/o4K7YGET7.
The following errors are produced by the SIMDE_CONSTIFY_ macros on Visual C++ when compiling for an ARM64 target:
https://simde.netlify.app/amalgamated/arm/neon.h(38025): warning C4003: not enough arguments for function-like macro invocation 'vext_f32'
https://simde.netlify.app/amalgamated/arm/neon.h(38025): warning C4002: too many arguments for function-like macro invocation '__float32x2_t_to_n64'
https://simde.netlify.app/amalgamated/arm/neon.h(38025): error C2059: syntax error: ')'

The above errors do not occur when compiling for x86 or x86_64 targets with Visual C++, and the errors do not occur when compiling for an ARM or ARM64 target with gcc or clang.

@johnplatts Can you retest using https://github.com/simd-everywhere/simde/releases/tag/v0.7.4-rc3 ?
Alas, I'm unable to update simde.netlify.app without Evan, so you'll need to bring the code another way

I ran into some of this and I think it has been fixed. We also have an MSVC arm64 build in our CI now. Let use know if we missed something!


Error Adding files to source control

Starting task: Adding files to source control

Executing the powershell script: ****\tasks\TFVCAdd\1.1.69\TfvcAdd.ps1
The specified module '****\tasks\TFVCAdd\1.1.69/vsts-tfvc-shared.psm1' was not loaded because no valid module file was found in any module directory.
Adding ItemSpec: ****\s\dev\Deployment\drops\biz\MH_BIZ_ProductsCatalog, Recursive: Full, Apply Ignorefile: false
The term 'Invoke-DisposeSourceProvider' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
The term 'Get-SourceProvider' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Finishing task: TFVCAdd

Task TFVCAdd failed. This caused the job to fail. Look at the logs for the task for more details.
Worker Worker-ea855c1a-4f22-41c9-9e74-0d7278ac0722 finished running job ea855c1a-4f22-41c9-9e74-0d7278ac0722

I manually copied script vsts-tfvc-shared.psm1 to the local (build agent) task folder, and the error was corrected. Deployment process has to be revisited.

This is strange... The task itself has the specified file in the vsix and when I execute the build on a clean accoutn the agent installs it and the file shows up. It would be interesting to share the logs from the agent\diag folder, I may need to raise this with the product team.
There's not much I can do to influence the actual deployment steps other than build the vsix.

My issue my be related to the fact I didn't install the extensions via marketplace. I used tfx build tasks upload!

That would be it.
You need to copy some files around in the build, you should be able to use:
./publish -force -release -publishlocal -serviceurl http://youraccount.visualstudio.com/DefaultCollection


syscall sleep not accurate when low delay

This was due to a bug in the reschedule function. The timer was not restarted. When a thread slept for < 1ms it would constantly run. Causing the timer inaccuracy to be visible.
Solved by fixing timer reset and only allowing for x * 1ms sleep. Less does not make any sense because the accuracy would be horrible!


Searchdomain crashes DHCP server

I am running latest KubeVirt on just released OpenShift 3.9, and when creating VM
apiVersion: kubevirt.io/v1alpha1
kind: VirtualMachine
metadata:
  annotations:
    presets.virtualmachines.kubevirt.io/presets-applied: kubevirt.io/v1alpha1
  clusterName: ""
  creationTimestamp: 2018-04-04T16:09:00Z
  generation: 0
  labels:
    kubevirt.io/nodeName: kubevirt-executor-lbednar-master1
  name: testvmgn56p
  namespace: kubevirt-test-default
  resourceVersion: "65056"
  selfLink: /apis/kubevirt.io/v1alpha1/namespaces/kubevirt-test-default/virtualmachines/testvmgn56p
  uid: 7cb2a2f0-3822-11e8-b93a-fa163e796a71
spec:
  domain:
    devices: {}
    features:
      acpi:
        enabled: true
    firmware:
      uuid: 52e2e942-8fcf-4a3e-bd7e-01d8e98db910
    machine:
      type: q35
    resources:
      requests:
        memory: 8Mi
  nodeSelector:
    kubernetes.io/hostname: kubevirt-executor-lbednar-master1
  terminationGracePeriodSeconds: 0
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: 2018-04-04T16:09:17Z
    message: unexpected EOF
    reason: Synchronizing with the Domain failed.
    status: "False"
    type: Synchronized
  interfaces:
  - ipAddress: <IP_ADDRESS>
  nodeName: kubevirt-executor-lbednar-master1
  phase: Scheduled

The virt-launcher is failing on following issue:
[root@kubevirt-executor-lbednar-master1 ~]# oc logs  -n kubevirt-test-default virt-launcher-testvmgn56p-fxf5k
level=info timestamp=2018-04-04T16:09:16.840080Z pos=virt-launcher.go:120 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmgn56p"
level=info timestamp=2018-04-04T16:09:16.840400Z pos=client.go:164 component=virt-launcher msg="Registered libvirt event notify callback"
level=info timestamp=2018-04-04T16:09:16.840495Z pos=virt-launcher.go:58 component=virt-launcher msg="Marked as ready"
level=info timestamp=2018-04-04T16:09:16.840716Z pos=monitor.go:241 component=virt-launcher msg="Monitoring loop: rate 1s start timeout 5m0s"
level=error timestamp=2018-04-04T16:09:17.385206Z pos=network.go:151 component=virt-launcher msg="Updated Mac for iface: eth0 - 3a:d1:fc:5e:f5:6e"
level=info timestamp=2018-04-04T16:09:17.397269Z pos=network.go:198 component=virt-launcher msg="Found nameservers in /etc/resolv.conf: \ufffd\u0010\u0000\u0014"
level=info timestamp=2018-04-04T16:09:17.397329Z pos=network.go:199 component=virt-launcher msg="Found search domains in /etc/resolv.conf: kubevirt-test-default.svc.cluster.local svc.cluster.local cluster.local openstacklocal"
level=info timestamp=2018-04-04T16:09:17.397342Z pos=dhcp.go:58 component=virt-launcher msg="Starting SingleClientDHCPServer"
level=error timestamp=2018-04-04T16:09:17.397404Z pos=network.go:176 component=virt-launcher msg="failed to run DHCP: Search domain is not valid: 'openstacklocal'"
panic: Search domain is not valid: 'openstacklocal'

goroutine 31 [running]:
kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/network.(*NetworkUtilsHandler).StartDHCP(0x1c8fc08, 0xc420185b80, 0xc4200714a0)
	/root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/network/network.go:177 +0x3c0
created by kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/network.SetupDefaultPodNetwork
	/root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/network/network.go:303 +0x4b0
virt-launcher exited with code 2

This openstacklocal domain is in /etc/resolv.conf like
# nameserver updated by /etc/NetworkManager/dispatcher.d/99-origin-dns.sh
# Generated by NetworkManager
search cluster.local openstacklocal
nameserver <IP_ADDRESS>

This OpenShift node is running as a VM inside of OpenStack cluster, and these values were generated by NetworkManager.
I will be happy for any suggestions!
Thanks,
Lukas.

"openstacklocal' is indeed an invalid domain per RFC. How do we want to handle this?

I would go in the way which @vladikr mentioned above.

In any case, we should skip an "incorrect" domain, but not crash.

That would end us up with incorrect config. One would assume that if it is in the resolv.conf in the pod it needs to be in the VM as well, to match functionality.
I think perhaps the domain validation should be loosened to allow partial domains like this.  'com' on its own is not a valid domain, but it IS a valid search domain as searching assumes it will be prepended with something.

@mlsorensen Thanks, your fix solved my problem!


Set Framerange doesn't apply after Template Builder used

In GitLab by @support-bot on Apr 17, 2023, 15:19

From Marc Gutmann

Problem
When we create a maya scene using Template Builder, then "set framerange" doesn't work for FPS.
How get error

Open Maya scene, use template builder
Change FPS in Ftrack
On Maya click on OpenPype > Set Framerange
FPS isn't update in the Maya scene

Softwares concerned
Maya, Ftrack, Fix

this now works

Can we close the ticket if it's all good ?

yes


PR gateway: Run the format (prog1) and audit (prog2) group in parallel.

This commit is to improve the exsiting PR gateway that distributes tasks..
When a webhook handler receive the event messages from the PRs,
The webhook handler call the PR gateway. Then, the PR gateway runs
the format (prog1) and audit (prog2) group in parallel.
Prog1 and Prog2 woudl be started in the background.
Signed-off-by: Geunsik Lim<EMAIL_ADDRESS>

:memo: Version: 1.20190225. Thank you for submitting PR #482. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnsuite.mooo.com/.

This PR passed the aging test in the real CI server. It is ready to be merged.


ceskatelevize: cannot download

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.03.14
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log
youtube-dl@stitny$ youtube-dl --verbose 'https://www.ceskatelevize.cz/porady/12058484398-katedrala-notre-dame-a-jeji-stavitele/21938775647/'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.ceskatelevize.cz/porady/12058484398-katedrala-notre-dame-a-jeji-stavitele/21938775647/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.03.14
[debug] Python version 3.8.8 (CPython) - Linux-5.11.4-1-default-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1
[debug] Proxy map: {}
[CeskaTelevizePorady]<PHONE_NUMBER>7: Downloading webpage
[CeskaTelevize] iFramePlayer.php: Downloading webpage
[CeskaTelevize] iFramePlayer.php: Downloading JSON metadata
[CeskaTelevize] iFramePlayer.php: Downloading JSON metadata
[CeskaTelevize] iFramePlayer.php: Downloading JSON metadata
[CeskaTelevize] iFramePlayer.php: Downloading JSON metadata
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 806, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 827, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/ceskatelevize.py", line 201, in _real_extract
    self._sort_formats(e['formats'])
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 1369, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

youtube-dl@stitny$

Description
The download used to work not that long time ago (a month), but it doesn't anymore.

Now, "no video with IDEC available"


Thank you.

@emptywee I am still looking into this, will let you know what I find out ASAP.  Good news is I can reproduce so I can now start to debug and figure out the issue.

just wanted to provide some updates, the stack trace is not stemming from this plugin but is stemming from one of the plugins we package with it.
https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/blob/master/lib/fluent/plugin/filter_kubernetes_metadata.rb
Line 297, based on the error I suspect es is nil.  I still need to determine why this is happening, the issue seems to be introduced with the https://github.com/SumoLogic/fluentd-kubernetes-sumologic/commit/7b100306d6c84335ee0d4ec6724a3218e8028893.

Thanks for the update. I used to add something similar to my configs (@timeout label) in v1.6 and it worked fine.

Ok so these logs are only happening when the concat plugin kicks in.  In my environment, I am running with mostly all default values. The default multiline pattern for the concat plugin is Julian dates.  I have one pod who's logging matches that pattern.  Everytime there is a multiline event in that pod, there is this stack trace:

The above is the final product sent to Sumo.  If we go look at the pod logs directly, we can see that this is the concatenation of 2 log lines.

And the stack trace is occuring at the exact same time, we get 2 because there are 2 messages being concated.

The good news is that this appears to be benign, other than generating noise in the logs.  No data is being dropped and the data makes it into Sumo concated together as expected.
@emptywee can you confirm that you are seeing the same behavior? E.g. that the stack traces occur during times of ingesting multi line log messages and the behavior above mirrors what you are seeing?

I'll check if logs aren't being lost. It's kinda tricky, since we have a really busy cluster and I need to find a way to make sure I am seeing what you are seeing :)

totally get it, i am going to try a few things in parallel and let you know what happens.

I think I have a fix, it is working locally, but want to let it run overnight to ensure all is good.  The fix will not be to this repo, but to the repo we depend on so timeline will depend on when they can merge.  if all continues to work as expected, I will submit a PR to that library.  We will leave this open for tracking until it is accepted and we can upgrade that library.

So it's really a fix? Not just a suppression of the error message?

I traced the issue.  When these stack traces appear, the event stream is actually empty in the k8s metadata filter plugin, which is downstream of the concat plugin.  I think it is a symptom of the concat plugin merging messages together.  Every time I get this stack trace that you are seeing, the stream is empty (I let it run with some debug logging for 24hours and confirmed every time it happened, it was empty).  The k8s metadata filter never expects an empty stream, so when it gets one, it causes this error. So I submitted a PR that ensures if the stream is empty, it does not process it (as there is nothing to process anyway).

update - fix was merged with the other repo.  Waiting on new gem to be published with fix. Then I will update the version we use and push a new image.  Hang tight, almost there.

@emptywee - ive pushed v1.11 which addressed the issue in my testing.  Before I push latest tag, wanna give her a go?

Absolutely. I will do it now and post results!

@frankreno everything looks solid. No issues has been observed so far. I think we can close this one. If anything comes up in the near future, I'll open a new one or we'll re-open this one if it has anything to do with it.

Thank you for fixing this 👍

Great, glad to hear and happy to help!


Update Dockerfile for Ruby 2.6

Description
ChefDK is now on Ruby 2.6, this should be reflected in the PATH.  Gems installed using chef gem install will otherwise not be on the PATH.

@bdwyertech This is a dupe of the PR I opened yesterday for the same #2109

Oops, didn't even look.  Thanks @tas50


No setup for League of Legends

When I select league of legends its just black keys no layers or anything tried to reinstall still nothing anyone lead me to a league.json or something any help would be appreciated.
Sorry I cleared all the chat out its late if you want to just delete ill make a New one couldnt figure out how to delete this topic.

Logitech910

League of Legends doesn't display anything by default as the lighting effects are sent from the game and are displayed on the keyboard through Aurora so you'll only see the effects when you are in-game.

Yeah I couldnt get it to work loading in the game tried to uninstall reinstall now i have a new problem Aurora wont load at all

Did you try deleting the save folder like mentioned in the other issue you commented on?

I did a search for it I couldnt find it ive uninstalled and reinstalled a couple times


Snakemake can't recognise input directories with relative path

Snakemake version

Used to work with snakemake 5.19.2
Fails with the snakemake 5.19.3 (latest currently available in bioconda)

Describe the bug
If the input of a rule is a directory, it suddenly only work with absolute paths, otherwise snake make complains about missing input.
Logs
(When running with --dryrun)
Snakemake 5.19.3 with relative paths:
VPIPE_BASEDIR = /Users/localadmin/V-test/V-pipe
Building DAG of jobs...
MissingInputException in line 1 of /Users/localadmin/V-test/V-pipe/rules/visualization.smk:
Missing input files for rule generate_web_visualization:
references/gffs/

Snakemake 5.19.2 or using absolute paths in 5.19.3:
VPIPE_BASEDIR = /Users/localadmin/V-test/V-pipe
Building DAG of jobs...
Job counts:

  {...job details...}

This was a dry-run (flag -n). The order of jobs does not reflect the order of execution.

Minimal example
Add a minimal example for reproducing the bug.
This snakemake file only works for 5.19.2:
rule all:
   input:
      gff_dir='references/gffs/'
   output:
      test_file='test.txt'
   shell:
      """
echo "b0rk" > {output.test_file}
      """

This config files work for 5.19.3 too:
rule all:
   input:
      gff_dir='/Users/localadmin/V-test/work/references/gffs/'
   output:
      test_file='test.txt'
   shell:
      """
echo "b0rk" > {output.test_file}
      """

Full real-world example
To exactly reproduce our use case:

you can follow the tutorial here
but use either this branch
or that branch
use the following config file:

[general]
aligner = bwa

[input]
samples_file = samples.tsv
fastq_suffix =
reference = references/NC_045512.2.fasta
gff_directory = references/gffs/
primers_file = references/primers/nCoV-2019.tsv

[output]
snv = True
local = True
global = False

visualization = True

[hmm_align]
leave_msa_temp = true

[coverage_intervals]
coverage=0

[snv]
mem=2048
threads=24

specific versions of snakemake can be tested with:

V=5.19.2; conda deactivate; conda env remove -n v-test ; conda create --yes -n v-test -c conda-forge -c bioconda snakemake-minimal="${V}"; conda activate v-test; snakemake -s "../V-pipe/vpipe.snake" --use-conda --dryrun; echo -ne "${V}\t"; snakemake --version

The problem is the trailing slash. Commonly, it is not needed. As a workaround, just remove it. It triggers a bug here that I will fix now.

Fixed in commit aedbf0f. Thank you!


Shared Library Error (libpython3.8.so.1.0) on builds using runner with python 3.9 or above

Receiving the following error trying to use snok/install-poetry action with python 3.9 job runner (does not contain any previous python versions).
/home/runner/.local/venv/bin/python: error while loading shared libraries: libpython3.8.so.1.0: cannot open shared object file: No such file or directory Error: Process completed with exit code 127.
Setting the python version to "3.8" resolves the issue:
steps:       - name: Set Up Python         uses: actions/setup-python@v3         with:           python-version: "3.8"       - name: Install Poetry         uses: snok/install-poetry@v1

Could you share a link to a failing workflow I could take a look at?


Continue assigning partitions when there's an error

Before, a topic error would stop messages for other topics from being assigned to a partition and therefore from being delivered. In order to better isolate topic errors, this change goes through all pending messages and writes those without errors to the produce operation buffer, leaving the rest in the pending message queue.

@bquorning I could use a review of this.


Prevent others from uploading to server

It's probably been documented somewhere and asked before, but what can I do to prevent random people from uploading to my private pomf? I'd prefer an api or user/password solution if possible that is compatible with ShareX.
Thank you.

This is beyond the scope of pomf.
Use HTTP Basic Authentication in your web server.

This isn't really suited for an Issue request - Help with pomf is better found at Rizon IRC - channel #pomfret. We won't code anything for you, but can answer most questions. lesderid's answer is a good start if you want.

I've decided to go the anonymous route and rename my upload.php. If that doesn't work, I'll look into Basic Auth. Thank you.

sr.ht is a good solution to this


Hosting instructions - readme

Add Vercel & Netlify details into the readme.

[ ] Test on Vercel
[ ] Test on Netlify

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


knexfile.js for migrations

I'm trying to configure knex migrations in my feathersjs app, I need to create a knexfile with the configurations for knex migrations to work, but I don't want to have my database configuration in two places (feathersjs config and knexfile), does anybody know how handle database configuration for both?

You can const app = require('./src/app);in the Knexfile and then get either the Knex client itself viaapp.get('knexClient')or the database configuration viaapp.get('')`.

You're right, that should work. Thanks

This helped me too, but @daffl I'm trying to sort out what the best practice is here. Migrating and seeding is a key feature, so would it make sense for feathers-knex to do this type of setup when installed?
In addition, the knex CLI isn't available when installed via the instructions in the README, is the preferred approach to also install knex globally to leverage the migration/seed apis?
Thanks!

Feathers adapters try to be as little in the way of the normal usage of the ORM/database as possible. So whatever the Knex documentation recommends will also work here. The main concern of this adapter is to connect a Knex model to the service interface and implement the common querying mechanism but that is pretty much it.

Cool, fair enough. Didn't see anything in the knex docs to accommodate the feathers setup but like I said the above recommendation worked great. To get migrations/seeding working i just installed knex globally, ran knex init and modified the knexfile.js file to:
const app = require('./src/app');

module.exports = app.get('postgres');

Then I could use the standard default/production feathers config files and add migration instructions there, e.g.:
  "postgres": {
    "client": "pg",
    "connection": "postgres://user:password@localhost:5432/database",
    "seeds": {
      "directory": "../src/seeds"
    }

I believe migrations will work as well with this strategy.
Thanks for the reply man!

If you would like to add that information to the Readme, a pull request would be very welcome :)

Thanks for the solution. May I add that it seems to be necessary to also comment out the parts in the models that create tables, as they seem to run before the migrations are called.

I know this is an old issue, but I echo the sentiment above re: migrations being a fundamental part of an application. This should really be addressed by the tool and in the documentation.

It's about adding the information above which can be done by editing the readme.


Fix Note Routes and Controller

Implement the following:

Remove use of disk storage in notes router.
Use middleware for image upload in notes controller.

Same here explain how you will ressolve these. What outcome we will get ?

@manjit-hub there were some issues in notes controller because the JWT token was not getting authenticated properly. So, I have worked on that issue. Also slight enhancement to display the name of the creater of a note on searching for notes.
And, one doubt, do you want to upload the document coming from uploadNote API on cloudinary or simply in the disk storage using multer, because the current codebase of your repo doesn't upload the note on cloudinary. So, I can add that as well.

@Shivansh-0403 go for it.


Problem with local dev config for demo issuer front-end

I'm trying to run the issuer front-end locally on my dev machine, either against the local agent (via ngrok) or the one at https://ssi.sphereon.com.
In .env.local, copied from .env per the instructions, should it have:
REACT_APP_OID4VP_PRESENTATION_DEF_ID=sphereon (like in the .env file)
or
REACT_APP_OID4VP_PRESENTATION_DEF_ID=sphereon2023 (like in the README)
Example packages/oid4vci-demo-frontend/.env.local:
REACT_APP_OID4VP_AGENT_BASE_URL=https://ssi.sphereon.com/agent
REACT_APP_OID4VP_PRESENTATION_DEF_ID=sphereon
REACT_APP_OID4VCI_AGENT_BASE_URL=https://ssi.sphereon.com/issuer/sphereon2023
REACT_APP_ENVIRONMENT=sphereon
PORT=5001

With REACT_APP_OID4VP_PRESENTATION_DEF_ID as sphereon, trying to log in with the Sphereon ID wallet app (latest iOS version) fails with:

Unable to retrieve information. Error: undefined is not an object (evaluating 'e.claims.vp_token')

The payload of the auth request JWT doesn't contain any claims, and the app isn't resilient to that, surfacing the low level error.

If I change it to sphereon2023, it gets further, but the wallet prompts for the guest credential, not the primary wallet ID credential. In the expected demo flow, this is only issued later, after authenticating.
If instead I enter the auth info manually, it goes through OK to issuing the guest credential
@nklomp I see there have been recent changes here. Could you please clarify?

I notice the docker .env file has sphereon2023: https://github.com/Sphereon-Opensource/OID4VC-demo/blob/develop/docker/.env.oid4vci-demo-frontend#L2

To use ngrok to tunnel to my locally running agent, I also had to change the credential_issuer and credential_endpoint in packages/agent/conf/dev/oid4vci_metadata/sphereon.json:
https://github.com/Sphereon-Opensource/OID4VC-demo/blob/771b94f90d7319975d63df9d63972bbbe9f81412/packages/agent/conf/dev/oid4vci_metadata/sphereon.json#L2
It would be nice if the agent endpoint could be configured purely through env vars.
Does the correlationId need to be a URL or can it be a URI that's not tied to the actual deployment address?

Through experimentation with the public demo issuer at , I found that if I change packages/oid4vci-demo-frontend/.env.local to have:
REACT_APP_OID4VP_PRESENTATION_DEF_ID=sphereonWallet

then the auth request includes the correct claims > vp_token >  presentation_definition for the identity challenge:
...
  "claims": {
    "vp_token": {
      "presentation_definition": {
        "id": "sphereonWallet",
        "purpose": "You need to prove your Wallet Identity data",
        "submission_requirements": [
          {
            "name": "Sphereon Wallet Identity",
            "rule": "pick",
            "min": 0,
            "max": 1,
            "from": "A"
          }
        ],
        "input_descriptors": [
          {
            "id": "SphereonWalletId",
            "purpose": "Checking your Sphereon Wallet information",
            "name": "Wallet Identity",
            "group": [
              "A"
            ],
            "schema": [
              {
                "uri": "https://sphereon-opensource.github.io/ssi-mobile-wallet/context/sphereon-wallet-identity-v1.jsonld"
              }
            ]
          }
        ]
      }
    }
  },
...

However, the Sphereon Wallet App now hangs and times out with a 504 gateway error when trying to fetch this. Fetching it at the same URL from my laptop works OK. Strange. Does the wallet use a proxy?

The demo agent doesn't have any definition with ID sphereonWallet though. Is that definition missing from the open source?


native-platform fails to run on FreeBSD 10 without gcc

The standard distribution for FreeBSD 10 does not include gcc, but includes clang instead.  This causes the native-platform to fail to run.  The native-platform should be updated to build with clang on FreeBSD so as to not require users to install gcc in addition to clang.

Strictly speaking, the problem isn't the compiler; it's the standard library.  FreeBSD 10 (and soon, 11) include libc++, but not libstdc++

This is fixed in 0.12. Thanks for reporting.


Fix #22 : Raise an exception when there are duplicates in links

I checked for the following keywords:

source
target
type
color

I don't think there are others that should be checked.
I checked that the examples in the repo still worrked after this commit.
(Fixes #22)

Thanks! I think your first commit was right -- it should be just source, target and type that are checked because they define the graph structure. In other words, if there are two links with the same source, target and type but different colours that is still a problem. Can you drop the second commit?

This should do it. Tell me if you want me to change anything. Retested the examples, everything seemed to work fine.

Looks good! Thanks for your help :tada:


UnicodeDecodeError when setup.cfg contains non-ASCII and LC_ALL=C

> cat >setup.py <<\EOF
# -*- coding: utf-8 -*-
from setuptools import setup
setup(
name='test',
version='1.0',
description='éàïñ',
)
EOF
> env LC_ALL=C python3 ./setup.py --description
éàïñ
> cat >setup.cfg <<\EOF
[metadata]
name = test
version = 1.0
description = éàïñ
EOF
> cat >setup.py <<\EOF
from setuptools import setup
setup()
EOF
> env LC_ALL=C python3 ./setup.py --description
Traceback (most recent call last):
  File "./setup.py", line 2, in <module>
    setup()
  File "/usr/lib/python3.6/distutils/core.py", line 121, in setup
    dist.parse_config_files()
  File "/usr/lib/python3.6/site-packages/setuptools/dist.py", line 355, in parse_config_files
    _Distribution.parse_config_files(self, filenames=filenames)
  File "/usr/lib/python3.6/distutils/dist.py", line 395, in parse_config_files
    parser.read(filename)
  File "/usr/lib/python3.6/configparser.py", line 697, in read
    self._read(fp, filename)
  File "/usr/lib/python3.6/configparser.py", line 1015, in _read
    for lineno, line in enumerate(fp, start=1):
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 75: ordinal not in range(128)

With PKG-INFO always written as UTF-8, I think it would make sense to load setup.cfg as UTF-8 too.
I see in the code that there's a setuptools/py36compat.py providing a patched parse_config_files but as seen in the traceback, it's not called. Is that normal?
According to the log it was added to fix #889, but adding a test for that fails:
diff --git i/setuptools/tests/test_config.py w/setuptools/tests/test_config.py
index 8bd2a494..2b118b70 100644
--- i/setuptools/tests/test_config.py
+++ w/setuptools/tests/test_config.py
@@ -288,6 +288,15 @@ class TestMetadata:
         with get_dist(tmpdir) as dist:
             assert set(dist.metadata.classifiers) == expected

+    def test_no_interpolation(self, tmpdir):
+        fake_env(
+            tmpdir,
+            '[metadata]\n'
+            'description = %(message)s\n'
+        )
+        with get_dist(tmpdir) as dist:
+            assert dist.metadata.description == '%(message)s'
+

 class TestOptions:

Anyway, here is my attempt at fixing: https://github.com/pypa/setuptools/compare/master...benoit-pierre:fix_889_and_non-ascii_in_setup.cfg
I can make proper PR(s) for those changes that are OK.

I started diving into this. I first found I could ignore most of the commits and focus on 9f81546. But even that single commit attempts to do too much, fixing two issues at the same time.
So I started to focus just on the first issue, that #889 seems not to be passing. I incorporated the test as dd404fd2e (thanks for that). But then when I went to find out why the test wasn't passing, all looked well except that, as you mentioned, the compatibility code isn't being called.
It seems that in 6913024 referencing #394, the compatibility code was overridden. And shame on me for not having a unit test to capture the regression.

Check da92ba46 for an alternative implementation for supporting this. Unfortunately, this can't be done for Python 2, as returning Unicode strings will break various things, e.g.: the sdist command when setup.cfg contains:
[sdist]
formats = zip

Because the distutils Python 2 code will check for str instances...

Okay, the fix for #889 is now in the master. There were two flaws. One was that the compatibility behavior was not being invoked. The other was that the compatibility behavior was not working properly because the interpolation=None had to be called in two places.

I should think that Setuptools should be able to support non-ascii characters in a setup.cfg file even if the target system is LANG=C. I believe LANG=C only affects default encoding behavior. Where setuptools can specify encoding behavior, the tests should run despite LANG=C.
But one thing I'm noticing is that these patches are affecting the behavior in py36compat Distribution_parse_config_files, where the docstring specifically cautions against editing that code. That code was copied from cpython and should match what's being released. If we want to tweak that behavior, we should probably contribute that behavior upstream first and then provide forward compatibility to Setuptools users by copying that code to the compat module.

The interpolation test still fails on Python 2 because Distribution_parse_config_files is not used. And if it is used, then interpolation=None cannot be used with Python 2, and parser.get(...,raw=True) must be used instead.
diff --git i/setuptools/py36compat.py w/setuptools/py36compat.py
--- i/setuptools/py36compat.py
+++ w/setuptools/py36compat.py
@@ -2,6 +2,8 @@ import sys
 from distutils.errors import DistutilsOptionError
 from distutils.util import strtobool
 from distutils.debug import DEBUG
+from setuptools.extern import six
+from setuptools.extern.six.moves.configparser import ConfigParser

 class Distribution_parse_config_files:
@@ -13,10 +15,9 @@ class Distribution_parse_config_files:
     as implemented in distutils.
     """
     def parse_config_files(self, filenames=None):
-        from configparser import ConfigParser

         # Ignore install directory options if we have a venv
-        if sys.prefix != sys.base_prefix:
+        if six.PY3 and sys.prefix != sys.base_prefix:
             ignore_options = [
                 'install-base', 'install-platbase', 'install-lib',
                 'install-platlib', 'install-purelib', 'install-headers',
@@ -33,7 +34,7 @@ class Distribution_parse_config_files:
         if DEBUG:
             self.announce("Distribution.parse_config_files():")

-        parser = ConfigParser(interpolation=None)
+        parser = ConfigParser()
         for filename in filenames:
             if DEBUG:
                 self.announce("  reading %s" % filename)
@@ -44,13 +45,13 @@ class Distribution_parse_config_files:

                 for opt in options:
                     if opt != '__name__' and opt not in ignore_options:
-                        val = parser.get(section,opt)
+                        val = parser.get(section,opt,raw=True)
                         opt = opt.replace('-', '_')
                         opt_dict[opt] = (filename, val)

             # Make the ConfigParser forget everything (so we retain
             # the original filenames that options come from)
-            parser.__init__(interpolation=None)
+            parser.__init__()

         # If there was a "global" section in the config file, use it
         # to set Distribution options.
@@ -69,12 +70,6 @@ class Distribution_parse_config_files:
                     raise DistutilsOptionError(msg)

-if sys.version_info < (3,):
-    # Python 2 behavior is sufficient
-    class Distribution_parse_config_files:
-        pass
-
-
 if False:
     # When updated behavior is available upstream,
     # disable override here.

@benoit-pierre I've published a call for help in distutils-sig. Given your passion and involvement, I want to acknowledge that you may be interested in assisting in this effort. If so, please comment in one of the tickets (#889 or the sister distutils ticket).

Hi there, does this issue still persist today for 2.7.x ?


[FE] Create flow to link a dataset to genesets

Need

Send POST request to /dp/v1/datasets/{dataset_uuid}/genesets with list of geneset uuids to link dataset to
If 404 returned invalidate geneset cache and rerequest for collection
Expected Request Body

{"add": [geneset_ids], "remove": ["geneset_ids]}

Expected Response Body - list of genesets linked to the dataset
[{"name":"geneset_name", "id": "geneset_uuid", "description":"geneset_description"}]

See Figma - https://www.figma.com/file/TKV4sJ00zWobEFkaNgklC3/Gene-Sets?node-id=178%3A1433

See Figma - https://www.figma.com/file/TKV4sJ00zWobEFkaNgklC3/Gene-Sets?node-id=178%3A1453

Open questions for @hthomas-czi: if we have to invalidate the cache due to a 404, do we need to show something in the UI indicating to the user that the page is going to refresh with new information?

@maniarathi Do we need to refresh the entire page when the gene set cache is invalidated...?

cc @tihuan ^^ Can you resolve with Harley? Thank you!


Safari browser support

Redirecting https://github.com/CatalystCode/3dtoolkit/issues/164#issue-304751718 to an issue in this repo on behalf of @zaxy78:
Hi !
I'm using the 3dtoolkit and noticed it's not compatible with Safari browser,
although Safari supports WebRTC since June 2017.
I've made it work on my side, mostly though fixes in adapter.js (PR PR) library.
Would it be a problem for you guys to include adapter.js intto 3dtoolkit?
At least in the Web Client Sample, but also potentially in the JS 3dtoolkit Lib itself.
It helps a lot with browsers differences, and might have saved you guys few tens of lines of code.
It is possible ofcouse to add the Safari APIs to the "adaptation lines of code" you guys used in the repo, manually.
I can suggest some pull request with the fixes, and allow the repo to enjoy support for Safari clients.
My only question is - would you take the easy path of including adapter.js , or would you insist in making the changes manually inside the lib?

@bengreenier so , what do u think?

@tylergibson, @anderm, or @sayar are probably more qualified to answer this one, unfortunately I don't have much context on this part of our stack. That said, it sounds reasonable to me 🚀

@zaxy78 I would definitely prefer we use adapter.js.  If you can integrate it and submit a PR, I'm all for integration. Unfortunately I don't think we will have time to do this work anytime in the near future.

@zaxy78 I am in favor as well, please submit a PR as I also don't have much time to work on this in the near future.


Support Windows

Fixes build script to support building on windows platform.
With this change, I have confirmed that examples work at the minimum on Windows. But, my testing is seriously lacking, so it may not works perfectly.

@paq Thank you for your PR. I'll try it tomorrow, so please wait a bit.

@paq Thank you for your PR. I'll try it tomorrow, so please wait a bit.

I checked running examples.
# Env
windows10
cmake version 3.19.4
clang version 11.0.0
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin

I checked running examples.
# Env
windows10
cmake version 3.19.4
clang version 11.0.0
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin

@paq Thank you fix CI and great PR! I will Merge and Release.

@paq Thank you fix CI and great PR! I will Merge and Release.

close: https://github.com/vaaaaanquish/lightgbm-rs/issues/3

close: https://github.com/vaaaaanquish/lightgbm-rs/issues/3


Contract events not being emitted with ganache-cli

I'm having an issues subscribing to contract events using Eventeum and ganache-cli.
I can see and subscribe to block events and successfully register and delete contract events but I don't get any notifications when they are raised.
my Event looks like this :
event PaymentScheduleCreated(         bytes32 id,         uint paymentLeeway,         address owner,         address destination,         uint subscriptionAmount     );
And I subscribe like this :
curl -X POST \ http://localhost:8060/api/rest/v1/event-filter \ -H 'Cache-Control: no-cache' \ -H 'Content-Type: application/json' \ -H 'Postman-Token: 616712a3-bf11-bbf5-b4ac-b82835779d51' \ -d '{ "id": "PaymentScheduleCreated", "contractAddress": "0x4a936E7Ce91FD8A47bbe9aeBFabE15AA13A11232", "eventSpecification": {   "eventName": "PaymentScheduleCreated",   "nonIndexedParameterDefinitions": [     {"position": 0, "type": "BYTES32"},     {"position": 1, "type": "UINT256"},     {"position": 2, "type": "ADDRESS"},     {"position": 3, "type": "ADDRESS"},     {"position": 4, "type": "UINT256"}] } }'
Which returns
{"id":"PaymentScheduleCreated"}
And I see this in the logs:

Already registered contract event filter with id: PaymentScheduleCreated

Am I missing something?

I can get other contract events to work, so it seems it's only that specific event that isn't working

Thats strange...have you checked that the ContractEventFilter entry in mongodb reflects the event filter that you've added?

AhI tracked it down, after looking in ContractEventFilter I noticed the UINT256  types where registered as INT256. I think that was a typo on my side when I originally tried them out. Once I fixed this everything worked.
Is INT256 ever valid? If not I would expect an error.
Thanks for the help.

I'm glad you figured out your issue!  INT256 types are valid yep; they're signed integers in solidity, whereas UINT's are unsigned.


Test all contact forms

Solution I'm going with right now is embedding gravity forms in a . I'd like to eventually roll contact and order forms from wp-rest api, but there's a lot of time, trial, and error involved especially for payment processor.
Gravity forms also contain all of their own features like conditional fields, validation, notifications, and assignments. I don't have the time it takes to roll my own versions of this,  nor do I think I could improve on them. Finally, Gravity Forms is well known in WordPress site owner and development circles, making it easier to create and maintain them in the back end.


[PoC] systemfields: initialize model parent

closes https://github.com/inveniosoftware/invenio-vocabularies/issues/156
Still in question if dict access to a record[model_field_name] is needed or not.

@lnielsen  @slint  Closing since it is not needed. However, for completeness I can make it pass tests so it is merged.


Fix #1275 layouts inconsistency in updateAnimated:completion of IGListBindingSectionController

Changes in this pull request
Issue fixed: #1275
This PR is straintforward solution as mentioned in #1275. When I finished it, I realized that it is not a problem in IGListCollectionViewLayout Partial Optimization and IGListBindingSectionController deserved it. Maybe IGListCollectionViewLayout is rarely used because of great builtin UICollectionViewFlowLayout or cells in section rarely need changing their sizes in practices.
Despite it is not IGListCollectionViewLayout's fault, I think it can be more optimized against invalidateLayoutWithContext. I would like to make an another PR to optimize it.
Due to this PR just giving a proposed solution, it lacks of adding tests and changing log. When this solution is accepted, I would like to complete these todos.
Checklist

[x] All tests pass. Demo project builds and runs.
[x] I added tests, an experiment, or detailed why my change isn't tested.
[x] I added an entry to the CHANGELOG.md for any breaking changes, enhancements, or bug fixes.
[x] I have reviewed the contributing guide

@lorixx It seems that you are active maintainer now. Can you review this PR and import it?

Can this PR ready to be merged or any else problem exists?

Looks good overall, the only main feedback I have is around that IGTestDiffingSectionController is doing too much thing and should not be a one-for-all kind of class, would you mind creating a different SectionController and DataSource for the IGLayoutTestItem dependent use case?

@lorixx I have accepted your suggestion and implemented it. Please review it again and give some comments if any.


Dataset customized[ASK]

Description

Other Comments
How can I put my own dataset , if I dont want to use the one provided by reco
thanks

solved


is there an option to run the docker container with a debug log level?

We are currently using the container to generate c4 model pngs in our project.
When I run it locally it takes extremely long - mainly because I am using an M1 macbook (linux/arm64/v8).
Sometimes it takes so long that I am wondering whats actually happening in the container. Therefor it would be cool to have a param or env setting to run the container with a more verbose log level

The support will be added through the log-level argument. Valid vales are the Logback levels (ERROR, WARN, INFO, DEBUG, TRACE).
./dsl2.png --log-level=TRACE


Request: targeting a VM?

Hello Alexey.
I have developed a low level VM with intent to use C as a scripting language.
Can I commission you to have SmallerC be able to output assembly code for my VM?
I have already created a custom assembler for the VM as well so there's no need to output direct bytecode unless you prefer to do that.
Sincerely, Assyrianic.

I'm not interested in doing that.

alright, thank you for your reply.


nheko: 0.6.1 -> 0.6.2

bump!

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Fits CONTRIBUTING.md.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: nheko
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Unexpected error: command failed with exit code 1 on aarch64-linux (full log)
Attempted: nheko
Partial log (click to expand)
copying path '/nix/store/hlfl7qqxnvcin596mzfpz7dw88vmljhk-harfbuzz-icu-1.8.8-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/8s2s1hjdvdnhl51hvxxanaskdk09hm5x-gst-plugins-base-1.14.2-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/4s2z5fzlf10j0s1aq73zz1l9gb9wy0a0-boost-1.67_0-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/i3gscvqdnxg0nj9xkln03yicw3fxyh3r-mtxclient-0.2.0' from 'https://cache.nixos.org'...
warning: unable to download 'https://cache.nixos.org/nar/0z8a3jsw0n3s414rvj8qrmpvykcgv31w2z8kscsgr5xsc8qwpqbw.nar.xz': HTTP error 503; retrying in 313 ms
error 7 while decompressing xz file
warning: unable to download 'https://cache.nixos.org/nar/0cdyds6xjj9rggf11cx36mw3ydi80i8css9czy2w6llyfj9lvyc4.nar.xz': HTTP error 503; retrying in 261 ms
error 7 while decompressing xz file
cannot build derivation '/nix/store/wmhkygv3kiarnj64f4ch90asjr0n5f14-nheko-0.6.2.drv': 4 dependencies couldn't be built
error: build of '/nix/store/wmhkygv3kiarnj64f4ch90asjr0n5f14-nheko-0.6.2.drv' failed

Success on x86_64-linux (full log)
Attempted: nheko
Partial log (click to expand)
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/hsmck87vnscmkabl31cj1gi468ldwl10-nheko-0.6.2
shrinking /nix/store/hsmck87vnscmkabl31cj1gi468ldwl10-nheko-0.6.2/bin/nheko
strip is /nix/store/dxf1m7dhc4qb655bdljc1fsd74v1nag3-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/hsmck87vnscmkabl31cj1gi468ldwl10-nheko-0.6.2/bin
patching script interpreter paths in /nix/store/hsmck87vnscmkabl31cj1gi468ldwl10-nheko-0.6.2
checking for references to /build in /nix/store/hsmck87vnscmkabl31cj1gi468ldwl10-nheko-0.6.2...
postPatchMkspecs
/nix/store/hsmck87vnscmkabl31cj1gi468ldwl10-nheko-0.6.2


Replace broken links in Exceptions section

Replace broken links in Exceptions section
In numerous cases, .NET for Windows Store apps and Portable Class Libraries throw a different exception than apps and libraries that target the .NET Framework.  In most of those cases, the exception information appears in Markdown format. However, since the <exception> section is included in IntelliSense, it must be in XML format and should be enclosed in a <see href=...> element.
Typically, the offending text appears as follows:

In the [.NET for Windows Store apps](http://go.microsoft.com/fwlink/?LinkID=247912) or the [Portable Class Library](~/docs/standard/cross-platform/cross-platform-development-with-the-portable-class-library.md), catch...

It should instead appear as:

In <see href="https://msdn.microsoft.com/library/windows/apps/br230232(v=vs.110).aspx">.NET for Windows Store apps</see> or the <see href="~/docs/standard/cross-platform/cross-platform-development-with-the-portable-class-library.md">Portable Class Library</see>, catch...

@rpetrusha - out of interest, is this the same issue in the namespace summary's - like seen on pages like this one?
https://docs.microsoft.com/en-us/dotnet/api/?view=netframework-4.7.2
I'd assumed it was just a bug on docs.microsoft.com - didn't realise they needed to be that way for Intellisense. 🙂

Yes @ChrisMaddock. But I'll close this as a dupe of #1931. I'm not sure if the href works with the .md links though. I usually transform them into absolute paths. I'll need to test this.

Duplicate of #1931

You can see an example of where I did this fix here for one summary: https://github.com/dotnet/dotnet-api-docs/pull/14/files


Added new message count onto icon

Monitors webpage title change and uses that as an indication of unread chats. This mimics the behaviour of Facebook.com's messages notifications as well.

Uhm, this notifies the new messages since last time I checked, it doesn't assert that I have read them though.

It removes them based on the current Facebook messenger behaviour - ie. on Facebook.com, the red number will disappear if you click on your messages, but in your inbox, it will still appear as bolded and unread.
And like you said, this is just a stepping stone to get to the point where an actual notification system is implemented.

Nice work!


"take every key" and "take every value" are broken

For example:
const someMap = {
  key1: "hello",
  key2: "there",
};

Saying "take every key one" or "take every value one" results in error Require at least one target with this action

I believe the issue is with https://github.com/pokey/cursorless-vscode/blob/38857a0ab912be0cd0e950222200debad6f3e0c6/src/languages/index.ts#L68
We could prob support a ^ in our pattern DSL that indicates which node's children should be iterated for "every".  So in the case of "key", it would be object^.pair[key].  Then I believe we'd also need to modify our matcher code to be able to match when it is given a node in the middle of the chain, rather than only supporting top or bottom-most node.  When the ^ is left out, we could just return the parent of the node.  WDYT @AndreasArvidsson ?


Location selector

I've implemented some stories as well, those can help in review?

This is still messed up somehow in that updateLocation is returning stale stuff :(

This is still messed up somehow in that updateLocation is returning stale stuff

What do you mean?

I'm still working on this, I'll re-request review when I'm ready for another round

This is still messed up somehow in that updateLocation is returning stale stuff

What do you mean?

If I go to storybook and look at the console, the results will be messed up depending on if I type into the display address box, move the circle or change the radius. Like for one, it'll show the result from last time. If I type "New York" into the box, the last update returns "New Yor" without k

I see. Try using update for both functions in commit

Ready for re-review

Bug: the edit profile page is currently messed up: the form won't submit if you don't edit the location

Closes #950

Oh no, I screwed it up, didn't remember develop was broken


Moving resources from subfolders to parent

I'm writing a telegraf plugin to pull data from ADC's and SDX's, and I'm experiences som difficulty importing the different resources, as they are in subdirs, but named the same as their config partner. So if I import both the config and the stat resource, then I have to rename one of them.
import (
        lbconfig "github.com/citrix/adc-nitro-go/resource/config/lb"
        lbstats "github.com/citrix/adc-nitro-go/resource/stat/lb"
        networkconfig "github.com/citrix/adc-nitro-go/resource/config/network"
        networkstats "github.com/citrix/adc-nitro-go/resource/stat/network"

)

var lbs lbstats.Lbvserverstats
var lbc lbconfig.Lbvserver
var rnats networkstats.Rnatstats
var rnatc networkconfig.Rnat

What I would suggest is
import (
        res_stat "github.com/citrix/adc-nitro-go/resource/config"
        res_conf "github.com/citrix/adc-nitro-go/resource/stat"
)

var lbs res_stat.Lbvserverstats
var lbc res_conf.Lbvserver
var rnats res_stat.Rnatstats
var rnatc res_conf.Rnat

It's easy to see how this expands very exponential with imports and aliases.
I've tried to implement the move at https://github.com/KalleDK/adc-nitro-go/commits/main
Also I'm cleaning up the resource structs by tagging the json with string, where the json value is a string encoded int

Hi KalleDK is there a way I can get in touch with you on this?

Ofcourse - I send you a mail with contact info


Make Prisma client work without a Prisma server

Right now, using the Prisma client always requires the deployment of an extra Prisma server that sits between the database and API server (or another program that needs to access the database). Prisma servers can be deployed with Docker.

While the Prisma server is great for scalability and performance, it is a notable barrier for people that are just getting started with Prisma. Developers that want to get to know Prisma as an ORM replacement are often surprised because other ORMs don't require this additional setup step.
In an effort of making Prisma more approachable, Prisma servers should be considered optional complexity and there should be a way to use the Prisma client without them.

Closing as duplicate of #2992


add (markdown) emphasis escaping

This feature adds the possibility of escaping italic on markdown
Before the only way to write a word with underscore (e.g. SYS_ADMIN) was placing a space after the underscore, now it's possible to escape it with a backslash so it won't trigger the italic feature.
Change
The change consist on combining the actual negative look-ahead for a space with a negative look-behind for a backslash

Sadly, we cannot use negative look behind yet as that feature is still too newly added to Safari.

Actually, hold that thought, perhaps it's time to change this policy: https://github.com/highlightjs/highlight.js/issues/3890

@joshgoebel great so if my change can be added that would be great to me because I have it patched in one of my projects and is not the best way to have it

No lookbehind for now as it would be a breaking change... why not just have a single (separate) rule that matches \* but then chooses not to highlight it?  Would that hurt anything else?

Ping.


Automator: update istio/api@master dependency in istio/istio@master

Generated by Automator - 2020-06-02T02:06:42+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
2c6c914600caee60b6c281453e575d8d21ab7d3b
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@istio-testing: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
2c6c914600caee60b6c281453e575d8d21ab7d3b
link
/test unit-tests_istio

integ-distroless-k8s-tests_istio
2c6c914600caee60b6c281453e575d8d21ab7d3b
link
/test integ-distroless-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Implementing AST properties using singleton edge lists

This PR implements AST edges for all AST properties. Because of a bug (or rather "known limitation") of Neo4J OGM we need to be a bit tricky in assigning AST edges to single-node properties. Specifically, we need to wrap them into a singleton list of one element.
The major advantage is two-fold:

We can now easily set the astParent in the constructor of the AstEdge (and probably do other things as well)
We can now traverse all edges in the graph since all properties are now edges and not just those in a list

There is a new allEdges / edges function as well as a new shortcut astEdges to retrieve all AST edges starting from a certain node.

I want to do another small clean up pass before merging.


"most common" Aggregator with Dask

Hi! Thanks for developing Flox, it has been quite useful in our workflows.
We have been working on a method to reduce categorical data using flox, using a "most common" strategy.
    most_common = Aggregation(
        name="most_common",
        numpy=_custom_grouped_reduction,
        chunk=None,
        combine=None,
    )

    result = flox.xarray.xarray_reduce(
        data,
        *coords,
        func=most_common,
        expected_groups=bounds,
    )

This works very well for in-memory datasets, however, not on Dask data (and I have not been able to get it to work with Dask data).
If the input is dask data, the following error is returned
NotImplementedError: Aggregation 'most_common' is only implemented for dask arrays when method='blockwise'.

If method="blockwise" is set, then the following error is returned.
InvalidIndexError: Reindexing only valid with uniquely valued Index objects

I have tried to implement chunk= and combine=, but I don't think that is possible with this method.
Memory limit of xarray_reduce
Additionally, because of the size of our input dataset the group_idx array does not fit in memory. This is due to the size being dim1 * dim2 * ... * size(np.int64).
Our current workaround is to apply the reduction on subset slices of the source data and target data, and then merging them. This works OK but is quite ugly and slow.
Is there another way to avoid this limit?

Most common implementation
def _most_common_label(neighbors: np.ndarray) -> int:
    """Find the most common label in a neighborhood.

    Note that if more than one labels have the same frequency which is the highest,
    then the first label in the list will be picked.
    """
    unique_labels, counts = np.unique(neighbors, return_counts=True)
    return unique_labels[np.argmax(counts)]

def _custom_grouped_reduction(
    group_idx: np.ndarray,
    array: np.ndarray,
    *,
    axis: int = -1,
    size: int | None = None,
    fill_value=None,
    dtype=None
) -> np.ndarray:
    """Custom grouped reduction for flox.Aggregation to get most common label.

    Args:
        group_idx : integer codes for group labels (1D)
        array : values to reduce (nD)
        axis : axis of array along which to reduce.
            Requires array.shape[axis] == len(group_idx)
        size : expected number of groups. If none,
            output.shape[-1] == number of uniques in group_idx
        fill_value : fill_value for when number groups in group_idx is less than size
        dtype : dtype of output

    Returns:
        np.ndarray with array.shape[-1] == size, containing a single value per group
    """
    return npg.aggregate_numpy.aggregate(
        group_idx,
        array,
        func=_most_common_label,
        axis=axis,
        size=size,
        fill_value=fill_value,
        dtype=dtype,
    )

Ah, now this is interesting!
The general solution is hard and approximate (I think). You'll need to implement something like count min sketch (I'm sure there are others)
The easier way: You'll have to decompose the problem and compute each unique item, and associated count in chunk, merge those intermediates together in combine and then pick the top-most in finalize.
Are there useful properties for the field you're collapsing or the field you're grouping by (example plots of both these fields would be useful)? For examlke

are there only a small number of unique labels?
method="blockwise" only works

Hi Deepak, thank you for your reply.

The easier way: You'll have to decompose the problem and compute each unique item and associated count in chunk (like you already do), merge those intermediates together in combine and then pick the top-most in finalize.

I attempted to do this, however I am unsure of how to exactly make this work. Below is the code, but I am not sure of how to wrap the individual functions in such a way that they're compatible with Aggregation.
Code
def unique_labels(a: np.ndarray) -> np.ndarray:
    labels = np.unique(a)
    return labels

def unique_counts(a: np.ndarray) -> np.ndarray:
    _, counts = np.unique(a, return_counts=True)
    return counts

def most_common_chunked(multi_values: np.ndarray, multi_counts: np.ndarray, **kwargs):
    all_values, index = np.unique(multi_values, return_inverse=True)
    all_counts = np.zeros(all_values.size, np.int64)
    np.add.at(all_counts, index, multi_counts.ravel())  # inplace
    return all_values[all_counts.argmax()]

most_common = Aggregation(
    name="most_common",
    numpy=_custom_grouped_reduction,
    chunk=(unique_labels, unique_counts),  # first compute blockwise
    combine=(wrap_stack, wrap_stack),  # stack these intermediate results
    finalize=most_common_chunked,  # get most common value from the combined result
    fill_value=0,
)

Unfortunately, this will need some major thinking. You'll have to handle the unique and count intermediates together, similar to how the argreduction is run. This is not trivial.
Is it possible to rechunk so that a blockwise solution works? Can you describe your problem precisely? A reproducible example would be best...

Due to the nature of our dataset it will have a limited number of unique labels.

OK that's good for the exact nature of this solution.

Also np.unique flattens the array. Is that OK for your purposes?

Hi Deepak, thank you for your replies and the modifications to the code. I am currently too busy with other projects to focus on this, but I will get back to it and try it out as soon as I can make some time!

Ah I've been dense. We can simply compute the histogram and then get the mode:
import flox
import numpy as np
import pandas as pd

array = np.array([[0, 0, 0, 1, 1, 5, 4, 5, 5, 3], [0, 0, 0, 1, 1, 3, 4, 5, 5, 3]])
array[1, :] += 4
by = np.array([0, 0, 0, 1, 1, 2, 2, 2, 3, 3])

The default only works with numpy and problems that can be executed "blockwise"
flox.groupby_reduce(array, by, func="mode", axis=-1)

(array([[0, 1, 5, 3],
        [4, 5, 7, 7]]),
 array([0, 1, 2, 3]))

This is the more general implementation that should always work regardless of chunking
result, uniques, groups = flox.groupby_reduce(np.ones_like(array), array, by, func="count", fill_value=0, axis=-1)
uniques[result.argmax(axis=-2)], groups

(array([[0, 1, 5, 3],
        [4, 5, 7, 7]]),
 array([0, 1, 2, 3]))

Responding to https://github.com/xarray-contrib/flox/pull/391#issuecomment-2333640949
Here, estimate the histogram manually and find the mode. You'll need to tell it what category values you expect here np.arange(0,6))
import pandas as pd

histogram = flox.xarray.xarray_reduce(
    xr.ones_like(data, dtype=bool),
    data.astype(int),  # important, needs to be int
    *coords,
    func="count",
    expected_groups=(pd.Index(np.arange(0, 6)), *bounds),
    fill_value=-1,
)
mode = histogram.idxmax("soil_type")

Here, estimate the histogram manually and find the mode.

Nice, this works and seems to have not too many issues being computed lazily!
The memory usage of the delayed objects does seem to be quite high though. In the ARCO ERA5 example, a time dim size of 700k makes Python use 4GB of ram (2GB for the output of flox.xarray.xarray_reduce, another 2GB for the object produced by histogram.idxmax("soil_type")).
It scales strongly wit the size of the time dimension, but not at all with the size of the latitude & longitude bins :thinking:

In theory, we could enable passing just the number of expected categories, and discover the values at compute time (https://github.com/xarray-contrib/flox/issues/15), but that's a bit more complexity.

I think it's fine to have users specify the expected categories. That's no problem for this use-case.

Ah I made a mistake: you'll need to specify dim=("latitude", "longitude"), to preserve the time dimension.

Ah I made a mistake: you'll need to specify dim=("latitude", "longitude"), to preserve the time dimension. Right now it will reduce across time too which would explain your memory usage comment

I did modify your suggestion by adding the dim kwarg! There still seems to be a high memory use depending on the size of the time dimension though.
I'll share a new gist so you can try it out. Need me to open an issue at flox?

This was fixed in xarray-regrid. And I think xarray can expose the same thing as histogram


added README documentation

I do not understand why the tests are failing here? They pass with npm test. I'm just honestly so confused.


Move constants to a separate module

Would be good to investigate if dotenv can work with gh-pages.

Apparently dotenv only works on the server, and the way to replicate its behaviour on the browser is using dotenv-webpack, which of course requires webpack. I am not sure it's worth it to add webpack to the project just for this, so I may factor out those constants by putting them in a config.json file.


Services: first build: abs added

{package} {version} {:snowflake:}
Destination channel: {Snowflake | defaults}
Links

{ticket_number}
Upstream repository
Upstream changelog/diff
Relevant dependency PRs:

...

Explanation of changes:

...

Linter check found the following problems:
ERROR conda.cli.main_run:execute(125): `conda run conda-lint /tmp/abs_8f4bjldzmj/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:39: missing_description: The recipe is missing a description
clone/recipe/meta.yaml:16: avoid_noarch: noarch: python packages should be avoided

===== ERRORS =====

clone/recipe/meta.yaml:39: missing_documentation: The recipe is missing a doc_url or doc_source_url
clone/recipe/meta.yaml:39: missing_license_family: The recipe is missing the about/license_family key.
clone/recipe/meta.yaml:39: missing_dev_url: The recipe is missing a dev_url
clone/recipe/meta.yaml:21: missing_wheel: For pypi packages, wheel should be present in the host section
===== Final Report: =====
4 Errors and 2 Warnings were found

Linter check found the following problems:
ERROR conda.cli.main_run:execute(125): `conda run conda-lint /tmp/abs_1dszwar01p/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:16: avoid_noarch: noarch: python packages should be avoided
clone/recipe/meta.yaml:42: missing_description: The recipe is missing a description

===== ERRORS =====

clone/recipe/meta.yaml:42: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
1 Error and 2 Warnings were found

Linter check found the following problems:
ERROR conda.cli.main_run:execute(125): `conda run conda-lint /tmp/abs_88i6bjy6ze/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:16: avoid_noarch: noarch: python packages should be avoided
clone/recipe/meta.yaml:43: missing_description: The recipe is missing a description

===== ERRORS =====

clone/recipe/meta.yaml:43: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
1 Error and 2 Warnings were found

Linter check found the following problems:
ERROR conda.cli.main_run:execute(125): `conda run conda-lint /tmp/abs_03mbmxwc44/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:43: missing_description: The recipe is missing a description
clone/recipe/meta.yaml:16: avoid_noarch: noarch: python packages should be avoided

===== ERRORS =====

clone/recipe/meta.yaml:43: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
1 Error and 2 Warnings were found


[Bug]: Doesn't work on VSCode 1.87.0

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
VSCode updated to 1.87.0 and now Vibrancy doesn't seem to work.
Expected Behavior
No response
Steps To Reproduce
No response
Environment
- OS: MacOS (Ventura 13.0.1)
- VSCode: 1.87.0 (commit 019f4d1419fbc8219a181fab7892ebccf7ee29a2)
- Extension: 1.1.27
- Theme: Default Dark

Anything else?
No response

I'm also on 1.87.0 and Vibrancy is working fine for me. Can you describe the issue in more detail, troubleshooting steps you've taken, etc? You can also try to reinstall VSCode as that seems to work for many people encountering issues after updates.

Thanks for the quick response.
I did already try reinstalling VSCode and reinstalling the extension but neither fixed the issue. Following the update to 1.87.0 when I restarted VSCode this morning, the vibrancy effect no longer works - modifying the opacity setting has no effect (the window is fully opaque at all times).

Can you check the value for the setting "Vibrancy: Type" (vscode_vibrancy.type) and post it here? Also, make sure that you don't have transparency disabled globally through accessibility settings.

Hey, i have to same problem. My env:

Version: 1.87.1 (user setup)
Commit: 1e790d77f81672c49be070e04474901747115651
Date: 2024-03-06T00:21:36.258Z
Electron: 27.3.2
ElectronBuildId: 26836302
Chromium: 118.0.5993.159
Node.js: 18.17.1
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22631

Also my screen get a little bit weird like this:

There is constant forgy. Finally i can't find "vscode_vibrancy.type" in settings.json. Just added this : "vscode_vibrancy.imports": ["/path/to/file"]

@sahinyakici please check pinned issues: https://github.com/illixion/vscode-vibrancy-continued/issues/122

@illixion Thank you. but I guess I'll wait for the extension to work on 1.87.1.

Doesn't seem to be fixed in 1.87.1 or 1.87.2 😢

Doesn't seem to be fixed in 1.87.1 or 1.87.2 😢

Are you referring to the macOS or Windows issue?

Doesn't seem to be fixed in 1.87.1 or 1.87.2 😢

Are you referring to the macOS or Windows issue?

Windows. Wrong issue, whoops!

Can you check the value for the setting "Vibrancy: Type" (vscode_vibrancy.type) and post it here? Also, make sure that you don't have transparency disabled globally through accessibility settings.

For me, the vscode_vibrancy.type setting is auto.

Closing as stale, if this is still an issue for you, please make sure that you don't have "Reduce transparency" enabled in macOS Accessibility settings and reopen.


Error in notebook

@git-add-akshay There is an error in "y_pred = model.predict(test_symptoms)"
Any idea how to fix this?

Thanks @git-add-akshay


[admin] drive navigation no longer detected

Regression from #388

:tada: This issue has been resolved in version 6.19.2 :tada:
The release is available on:

GitHub release
v6.19.2

Your semantic-release bot :package::rocket:


Support JS SDK Testing

Remove the need to stand up server in order to test js-sdk
Part of #11

JS SDK tests pass in CI:
https://github.com/protectai/rebuff/actions/runs/6780407793/job/18428997316
I then copied the python PR target strategy now that it's fixed:
https://github.com/protectai/rebuff/pull/73


juliuslib with c++

Hi, I'm doing a project on a SBC with julius on Ubuntu 20.04, I installed julius via apt, then ran it with a modified version on Sample.jconf from Quickstart 2018 and it's not a burden on the little computer and runs well.
I'm a relatively new programmer but I'd like to use it with a simple AI I've written in C++ but can't seem to find any information on the topic, just a ton of files so I know there has to be a header file and list of functions.
Would someone be kind enough to point me in the right direction?
Thanks in advance, Tom

I wrote a function to compare the log against an earlier version and pull the last line if it has. It works but it's buggy as nothing stops both programs from interacting with the log at once.
Any thoughts would be appreciated.

Still looking for a C++ API for Julius, my clumsy logfile method works but there is no way to do more sophistivated things likw changing grammar without killing the process and restarting it.


Are there plans for @storybook/html and @storybook/addon-info to play nicely?

Currently using @storybook/html and @storybook/addon-info breaks storybook. The combo gets an error Expecting an HTML snippet or DOM node from the story.
I would like to use @storybook/html to build HTML/CSS components and have the copy code to the clipboard feature from @storybook/addon-info. The other options in addon-info would be great too, but copy to the clipboard is my primary use case.
There are 2 closed issues that reference my needs #3985 + #2187. The latter mentions using vue, I attempted this but found it adds undesired complexity.
I appreciate all the work that has gone into storybook. Thank you to all the maintainers for their effort. I am unsure if I am able to work on this feature, I'm just curious if this has any priority.
Here is an example of how I'd like to use @storybook/html + @storybook/addon-info in a story...
import { storiesOf } from '@storybook/html';
import { withInfo } from '@storybook/addon-info';

storiesOf('My little pony', module)
  .addDecorator(withInfo)
  .add('Twilight Sparkle', () => `<div>Twilight Sparkle</div>`)

// OR

storiesOf('Unicorns', module)
  .add('Twilight Sparkle', withInfo('Twilight Sparkle')(() => `<div>Twilight Sparkle</div>`))

...and the desired result

Thank you.

@Pickra Not in its current form. We're working on a much better approach to solving the problems addon-info addresses that will work with all frameworks. Unfortunately we don't have a timeline for that yet.

Cool @shilman, thanks for the speedy response.

Hey @shilman, I just wanted to mention that @kgroat and I built a solution in case anyone else was looking for something.
https://github.com/Pickra/copy-code-block
Thanks again!

Will be fixed in https://github.com/storybooks/storybook/issues/1147


Stdout (and stderr) are unexpectedly deleted on a refresh operation

What happened?
We have a script we're running using the command resource that outputs data about the objects that it is creating. We then consume and parse the stdout output to pass this information into dependent resources. However when we run a refresh on the stack, the stdout and stderr output values are cleared and then because the Command resource already exists the output is never available again unless we force the resource to be recreated.
Steps to reproduce
Create a command resource that outputs a string value to stdout.
Create another resource that uses stdout from the command resource.
Run a refresh.
The dependent resource will no longer have a value from stdout.
Expected Behavior
Either the refresh should not delete stdout and stderr, or the Command resource needs to have a "Get" command to output the current value of the resource so that refresh will update stdout and stderr instead of deleting them.
Actual Behavior
stdout and stderr are deleted on pulumi refresh
Versions used
The pulumi-command version used was 0.3.2, I don't have access to the project at the moment to get the other versions. Let me know if I need to provide them and I'll update when I can.
Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

@lukehoban Is there any chance that this is by design, e.g. potentially to keep secrets out of the state file?


Possible evtx-sigma-checker Issue

Hello,
I've noticed that evtx-sigma-checker is not reading the value of ProcessId and AccessMask as strings, meanwhile Aurora detects the following example rule as string and works perfectly:
Sigma working with Aurora
EventID: 4663
ObjectType: 'File'
ObjectServer: 'Security'
ProcessId: '0x4'
AccessMask: '0x1'
Sigma working with evtx-sigma-checker
EventID: 4663
ObjectType: 'File'
ObjectServer: 'Security'
ProcessId: 0x4
AccessMask: 0x1
It could be an issue from my end or from the compiled evtx-sigma-checker
thanks in advance

Hi,
This sounds interesting. Maybe there's a difference in formatting from different sources (ETW versus EVTX files)? Could you send me an EVTX file with the event you described?

Access-Password-File-TruePositive.zip
Thanks for your response.
EventID: 4663
ObjectType: 'File'
ObjectServer: 'Security'
ObjectName: 'C:\Windows\SYSVOL\domain\Password\password.txt'
**ProcessId: '0x4'
AccessMask: '0x1'**

exclude:
SubjectUserName|endswith: '$'
condition:
- selection and not exclude

Thanks for the file!
From the EVTX file, I think the "different formatting" hypothesis holds up. The raw event in the evtx-checker looks like:
...  ObjectServer: Security  ObjectType: File  ObjectName: <redacted>  HandleId: 8020  AccessList: %%4416  AccessMask: 1  ProcessId: 4  ProcessName:   ResourceAttributes: S:AI  Provider_Name: Microsoft-Windows-Security-Auditing  ...
So access mask and process ID both get formatted as decimal numbers here, not as hex numbers. Probably the formatting information gets lost in the EVTX parser.
I'll need to talk to some colleagues about this. This is probably not trivial to fix; but the current evtx-sigma-checker behaviour results in differences between recorded baseline and live behaviour, which is also less than ideal.

@secDre4mer Thank you, appreciate your great support!

New release that contains the fix is now available.


Metadata wrong for https://data.datacite.org/10.14469/hpc/1248

The metadata downloads a table.css file instead of displaying metadata

Still current as of 15/03/17.

That needs to be reported to datacide..

Matt,
This entry was “hand crafted”  and is therefore different from the ones injected by eg the  HPC portal.  I am just checking that the DataCite scripts for this specific entry do not contain an error somewhere.  It seems that the  “wrong” file was in some sense sent, and perhaps this is  a file index issue at this  end?
I use this example for my public talks, so it is of some interest to get it to work correctly  for the metadata!

On 16 Mar 2017, at 13:57, M J Harvey<EMAIL_ADDRESS>wrote:
That needs to be reported to datacite..
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

PS If it is  Datacite that is the issue, I have no information about the injection used.  I guess its you who have to contact them?
Henry
Henry Rzepa, http://orcid.org/0000-0002-8635-8390

On 16 Mar 2017, at 13:57, M J Harvey<EMAIL_ADDRESS>wrote:
That needs to be reported to datacite..


Collect comparison benchmarks from other systems

We want to be able to compare how IREE is doing vs relevant alternative systems. For now that likely means TVM and TFLite. We need a way to periodically collect this information.
Considerations

Right now, getting a TFLite benchmark means finding someone with a phone and then running a series of steps. This should at least be turned into a script.
It would be nice to not require someone have access to matching hardware for doing the benchmarking
It would be nice to have this in a similar graph format as our own benchmarks
We don't want to get regression notifications for these benchmarks
TFLite uses XNNPack by default now, but that makes quite a few constraining assumptions about matmuls. Given that we're not yet aiming for that level of specialization, it may be more useful to measure vs TFLite with XNNPack turned off. These numbers shouldn't be taken out of context to be about TFLite performance. It may or may not be useful to have TFLite benchmarks with and without XNNPack
We need to figure out how to benchmark a TVM model. I assume this would be for desktop only. Again, we want to be able to benchmark this on the same hardware.
If we put these in Dana, we should make sure they're clearly separated and labeled.
We don't want to pollute our build infrastructure with configuration of other projects. This should probably all be done with totally separate scripting.
We need to make sure we have a way to get the same model source for all benchmarks, even if they may take them in a slightly different format.
It may be useful to compare the results of the model runs to ensure we really are testing the same model.

I think it should be possible to upload these benchmarks to Dana, but either turn off regression monitoring or effectively do so by setting the threshold very high. We probably don't want to run these every time, so we could set up a different pipeline to do this (or make the existing pipeline check some throttle to do it, which would be a bit hacky, but might enable additional reuse).
For now though, the first step would be scripts to do this manually, but repeatably.
FYI @antiagainst @julianwa

This issue looks obsolete. There are some comparative benchmarks in https://github.com/openxla/openxla-benchmark/tree/main/comparative_benchmark now. I think there are also several other out of tree / ad-hoc comparative benchmark efforts. Not sure how useful it would be to keep tracking those projects in this repo itself.


Feat: add sniffing for TUN

We're currently processing your upload.  This comment will be updated when the results are available.