Unable to make TCP DNS queries on one cluster, not others, until DestinationRule recreated

Hi,
We have three clusters, dev, preprod and prod, which are all built identically (same Infra as Code etc).  On one of them (preprod), applications that are behind Istio are unable to make TCP DNS queries:
[atcloud@airflow-web-5d9984d64-2pqpm app]$ dig +tcp www.google.com
;; communications error to <IP_ADDRESS>#53: connection reset

Looking at istio debug logs, there's nothing particularly telling:
[2019-02-11 08:07:38.552][28][debug][filter] external/envoy/source/extensions/filters/listener/original_dst/original_dst.cc:18] original_dst: New connection accepted
[2019-02-11 08:07:38.552][28][debug][filter] src/envoy/tcp/mixer/filter.cc:28] Called tcp filter: Filter
[2019-02-11 08:07:38.552][28][debug][filter] src/envoy/tcp/mixer/filter.cc:38] Called tcp filter: initializeReadFilterCallbacks
[2019-02-11 08:07:38.552][28][debug][filter] external/envoy/source/common/tcp_proxy/tcp_proxy.cc:168] [C423] new tcp proxy session
[2019-02-11 08:07:38.552][28][debug][filter] src/envoy/tcp/mixer/filter.cc:98] [C423] Called tcp filter onNewConnection: remote <IP_ADDRESS>:49261, local <IP_ADDRESS>:53
[2019-02-11 08:07:38.552][28][debug][filter] external/envoy/source/common/tcp_proxy/tcp_proxy.cc:305] [C423] Creating connection to cluster outbound|53||kube-dns.kube-system.svc.cluster.local
[2019-02-11 08:07:38.552][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:98] [C423] closing data_to_write=0 type=1
[2019-02-11 08:07:38.552][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:133] [C423] closing socket: 1
[2019-02-11 08:07:38.552][28][debug][filter] src/envoy/tcp/mixer/filter.cc:134] Called tcp filter onEvent: 1
[2019-02-11 08:07:38.552][28][debug][main] external/envoy/source/server/connection_handler_impl.cc:218] [C423] new connection
[2019-02-11 08:07:38.552][28][debug][filter] src/envoy/tcp/mixer/filter.cc:33] Called tcp filter : ~Filter
[2019-02-11 08:07:39.746][23][debug][main] external/envoy/source/server/server.cc:126] flushing stats

Pods in Dev and Prod are absolutely fine.
We have mTLS enabled on our cluster, and have done since day one.  As a result we have had the following destinationrule in place for almost 3 months:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  creationTimestamp: 2018-11-05T17:35:35Z
  generation: 1
  labels:
    app: istio-crd
    chart: istio-crd-1.0.5
    heritage: Tiller
    release: istio-crd
  name: disable-mtls-to-k8s-dns
  namespace: istio-system
  resourceVersion: "80583893"
  selfLink: /apis/networking.istio.io/v1alpha3/namespaces/istio-system/destinationrules/disable-mtls-to-k8s-dns
  uid: 339f9c78-e121-11e8-a96c-42010aa40060
spec:
  host: kube-dns.kube-system.svc.cluster.local
  trafficPolicy:
    tls:
      mode: DISABLE

Because I enjoy shooting in the dark, and because I'd seen TCP dns queries failed in a similar fashion months ago before we put this destination rule in, I decided to delete and recreate the DestinationRule, and this fixed the situation:
[atcloud@airflow-web-5d9984d64-2pqpm app]$ dig +tcp www.google.com

; <<>> DiG 9.9.4-RedHat-9.9.4-73.el7_6 <<>> +tcp www.google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19108
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.google.com.                        IN      A

;; ANSWER SECTION:
www.google.com.         299     IN      A       <IP_ADDRESS>
www.google.com.         299     IN      A       <IP_ADDRESS>

[2019-02-11 08:15:18.731][28][debug][filter] external/envoy/source/extensions/filters/listener/original_dst/original_dst.cc:18] original_dst: New connection accepted
[2019-02-11 08:15:18.731][28][debug][filter] src/envoy/tcp/mixer/filter.cc:28] Called tcp filter: Filter
[2019-02-11 08:15:18.731][28][debug][filter] src/envoy/tcp/mixer/filter.cc:38] Called tcp filter: initializeReadFilterCallbacks
[2019-02-11 08:15:18.731][28][debug][filter] external/envoy/source/common/tcp_proxy/tcp_proxy.cc:168] [C436] new tcp proxy session
[2019-02-11 08:15:18.731][28][debug][filter] src/envoy/tcp/mixer/filter.cc:98] [C436] Called tcp filter onNewConnection: remote <IP_ADDRESS>:57143, local <IP_ADDRESS>:53
[2019-02-11 08:15:18.731][28][debug][filter] external/envoy/source/common/tcp_proxy/tcp_proxy.cc:305] [C436] Creating connection to cluster outbound|53||kube-dns.kube-system.svc.cluster.local
[2019-02-11 08:15:18.731][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:572] [C437] connecting to <IP_ADDRESS>:53
[2019-02-11 08:15:18.731][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:581] [C437] connection in progress
[2019-02-11 08:15:18.731][28][debug][main] external/envoy/source/server/connection_handler_impl.cc:218] [C436] new connection
[2019-02-11 08:15:18.732][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:466] [C437] connected
[2019-02-11 08:15:18.733][28][debug][filter] src/envoy/tcp/mixer/filter.cc:105] Called tcp filter completeCheck: OK
[2019-02-11 08:15:18.733][28][debug][filter] src/envoy/tcp/mixer/filter.cc:78] [C436] Called tcp filter onRead bytes: 45
[2019-02-11 08:15:18.737][28][debug][filter] src/envoy/tcp/mixer/filter.cc:88] [C436] Called tcp filter onWrite bytes: 141
[2019-02-11 08:15:18.737][28][debug][filter] src/envoy/tcp/mixer/filter.cc:78] [C436] Called tcp filter onRead bytes: 0
[2019-02-11 08:15:18.738][28][debug][filter] src/envoy/tcp/mixer/filter.cc:88] [C436] Called tcp filter onWrite bytes: 0
[2019-02-11 08:15:18.738][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:451] [C437] remote close
[2019-02-11 08:15:18.738][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:133] [C437] closing socket: 0
[2019-02-11 08:15:18.738][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:98] [C436] closing data_to_write=0 type=0
[2019-02-11 08:15:18.738][28][debug][connection] external/envoy/source/common/network/connection_impl.cc:133] [C436] closing socket: 1
[2019-02-11 08:15:18.738][28][debug][filter] src/envoy/tcp/mixer/filter.cc:132] Called tcp filter onEvent: 1 upstream <IP_ADDRESS>:53
[2019-02-11 08:15:18.738][28][debug][main] external/envoy/source/server/connection_handler_impl.cc:51] [C436] adding to cleanup list
[2019-02-11 08:15:18.738][28][debug][filter] src/envoy/tcp/mixer/filter.cc:33] Called tcp filter : ~Filter

Really worried about this, that a DestinationRule appears to have been randomly ignored?  It's worth noting that all other DestinationRules appear to be being honoured.

OK
This came back only moments later.  This time we tried restarting pilot and again this has (probably temporarily) fixed the problem.
I have captured cluster debug logs from both before and after - please can someone from istio reach out to me directly on Slack so I can provide them securely.
Thanks
Karl

It has returned after an istio-pilot restart too.

OK - latest changes, I set PILOT_DIRECT_EDS=0 to disable the incremental EDS updates because the only difference between these clusters is that preprod has more pods than the other clusters.
Setting PILOT_DIRECT_EDS=0 seems to have resolved the problem and our DNS queries are now working fine and dandy again.
cc @costinm

@costinm @rshriram we turned PILOT_DIRECT_EDS back on today after upgrading to 1.0.6 and this error still persists.
It appears that istio-proxies do not have any endpoints on the kubedns cluster so TCP dns requests fail:
[atcloud@sauron-web-5b5df8f647-2v6q7 app]$ curl -s http://<IP_ADDRESS>:15000/clusters | grep kube-dns
outbound|53||kube-dns.kube-system.svc.cluster.local::default_priority::max_connections::1024
outbound|53||kube-dns.kube-system.svc.cluster.local::default_priority::max_pending_requests::1024
outbound|53||kube-dns.kube-system.svc.cluster.local::default_priority::max_requests::1024
outbound|53||kube-dns.kube-system.svc.cluster.local::default_priority::max_retries::3
outbound|53||kube-dns.kube-system.svc.cluster.local::high_priority::max_connections::1024
outbound|53||kube-dns.kube-system.svc.cluster.local::high_priority::max_pending_requests::1024
outbound|53||kube-dns.kube-system.svc.cluster.local::high_priority::max_requests::1024
outbound|53||kube-dns.kube-system.svc.cluster.local::high_priority::max_retries::3
outbound|53||kube-dns.kube-system.svc.cluster.local::added_via_api::true

The service definitely has endpoints:
❯ kks get ep kube-dns -o yaml
apiVersion: v1
kind: Endpoints
metadata:
  creationTimestamp: 2018-05-11T16:02:05Z
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: KubeDNS
  name: kube-dns
  namespace: kube-system
  resourceVersion: "113798791"
  selfLink: /api/v1/namespaces/kube-system/endpoints/kube-dns
  uid: a6b930b9-5534-11e8-9910-42010aa400bf
subsets:
- addresses:
  - ip: <IP_ADDRESS>
    nodeName: gke-delivery-platfor-durable-20190114-d2a699bd-zd3k
    targetRef:
      kind: Pod
      name: kube-dns-7549f99fcc-jg2vw
      namespace: kube-system
      resourceVersion: "113798336"
      uid: 11d04e44-345a-11e9-9430-42010aa40128
  - ip: <IP_ADDRESS>
    nodeName: gke-delivery-platfor-durable-20190114-d2a699bd-zd3k
    targetRef:
      kind: Pod
      name: kube-dns-7549f99fcc-q4mrt
      namespace: kube-system
      resourceVersion: "113798544"
      uid: 17c2d9fd-345a-11e9-9430-42010aa40128
  - ip: <IP_ADDRESS>
    nodeName: gke-delivery-platfor-durable-20190114-ea1efef2-n4w4
    targetRef:
      kind: Pod
      name: kube-dns-7549f99fcc-fq2pk
      namespace: kube-system
      resourceVersion: "113798740"
      uid: 1f535b05-345a-11e9-9430-42010aa40128
  - ip: <IP_ADDRESS>
    nodeName: gke-delivery-platfor-durable-20190114-4d15f0a5-s7bz
    targetRef:
      kind: Pod
      name: kube-dns-7549f99fcc-hnrls
      namespace: kube-system
      resourceVersion: "113798782"
      uid: 27229fd5-345a-11e9-9430-42010aa40128
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP

And pilot is aware of them (my colleague is going to post the pilot config in a second).
The only happens on our largest cluster, which is leading me to believe that it is potentially something to do with the number of endpoints/services or something similar but there is nothing telling in the logs, other than the pod detection:
❯ kis logs istio-pilot-5868cc6688-g2d25 discovery | grep kube-dns
2019-02-20T05:00:14.146913Z     info    Handling event add for pod kube-dns-7549f99fcc-fq2pk in namespace kube-system -> <IP_ADDRESS>
2019-02-20T05:00:14.149150Z     info    Handling event add for pod kube-dns-autoscaler-67c97c87fb-l2g22 in namespace kube-system -> <IP_ADDRESS>
2019-02-20T05:00:14.149737Z     info    Handling event add for pod kube-dns-7549f99fcc-hnrls in namespace kube-system -> <IP_ADDRESS>
2019-02-20T05:00:14.151760Z     info    Handling event add for pod kube-dns-7549f99fcc-q4mrt in namespace kube-system -> <IP_ADDRESS>
2019-02-20T05:00:14.151907Z     info    Handling event add for pod kube-dns-7549f99fcc-jg2vw in namespace kube-system -> <IP_ADDRESS>

So for whatever reason, that config is not getting to the side cars.

{"svc": "kube-dns.kube-system.svc.cluster.local:dns", "ep": [
{
"endpoint": {
"Family": 0,
"Address": "10.x.x.x",
"Port": 53,
"ServicePort": {
"name": "dns",
"port": 53,
"protocol": "UDP"
},
"UID": "kubernetes://kube-dns-7549f99fcc-jg2vw.kube-system"
},
"service": {
"hostname": "kube-dns.kube-system.svc.cluster.local",
"address": "10.x.x.x",
"ports": [
{
"name": "dns",
"port": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "TCP"
}
],
"external": "",
"MeshExternal": false,
"Resolution": 0,
"creationTime": "2018-05-11T16:02:05Z",
"Attributes": {
"Name": "kube-dns",
"Namespace": "kube-system",
"UID": "istio://kube-system/services/kube-dns"
}
},
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "3105955977"
},
"az": "europe-west4/europe-west4-a",
"serviceaccount": "spiffe://cluster.local/ns/kube-system/sa/kube-dns"
},
{
"endpoint": {
"Family": 0,
"Address": "10.x.x.x",
"Port": 53,
"ServicePort": {
"name": "dns",
"port": 53,
"protocol": "UDP"
},
"UID": "kubernetes://kube-dns-7549f99fcc-q4mrt.kube-system"
},
"service": {
"hostname": "kube-dns.kube-system.svc.cluster.local",
"address": "10.x.x.x",
"ports": [
{
"name": "dns",
"port": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "TCP"
}
],
"external": "",
"MeshExternal": false,
"Resolution": 0,
"creationTime": "2018-05-11T16:02:05Z",
"Attributes": {
"Name": "kube-dns",
"Namespace": "kube-system",
"UID": "istio://kube-system/services/kube-dns"
}
},
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "3105955977"
},
"az": "europe-west4/europe-west4-a",
"serviceaccount": "spiffe://cluster.local/ns/kube-system/sa/kube-dns"
},
{
"endpoint": {
"Family": 0,
"Address": "10.x.x.x",
"Port": 53,
"ServicePort": {
"name": "dns",
"port": 53,
"protocol": "UDP"
},
"UID": "kubernetes://kube-dns-7549f99fcc-fq2pk.kube-system"
},
"service": {
"hostname": "kube-dns.kube-system.svc.cluster.local",
"address": "10.x.x.x",
"ports": [
{
"name": "dns",
"port": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "TCP"
}
],
"external": "",
"MeshExternal": false,
"Resolution": 0,
"creationTime": "2018-05-11T16:02:05Z",
"Attributes": {
"Name": "kube-dns",
"Namespace": "kube-system",
"UID": "istio://kube-system/services/kube-dns"
}
},
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "3105955977"
},
"az": "europe-west4/europe-west4-b",
"serviceaccount": "spiffe://cluster.local/ns/kube-system/sa/kube-dns"
},
{
"endpoint": {
"Family": 0,
"Address": "10.x.x.x",
"Port": 53,
"ServicePort": {
"name": "dns",
"port": 53,
"protocol": "UDP"
},
"UID": "kubernetes://kube-dns-7549f99fcc-hnrls.kube-system"
},
"service": {
"hostname": "kube-dns.kube-system.svc.cluster.local",
"address": "10.x.x.x",
"ports": [
{
"name": "dns",
"port": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "TCP"
}
],
"external": "",
"MeshExternal": false,
"Resolution": 0,
"creationTime": "2018-05-11T16:02:05Z",
"Attributes": {
"Name": "kube-dns",
"Namespace": "kube-system",
"UID": "istio://kube-system/services/kube-dns"
}
},
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "3105955977"
},
"az": "europe-west4/europe-west4-c",
"serviceaccount": "spiffe://cluster.local/ns/kube-system/sa/kube-dns"
},
{}]},
{"svc": "kube-dns.kube-system.svc.cluster.local:dns-tcp", "ep": [
{
"endpoint": {
"Family": 0,
"Address": "10.x.x.x",
"Port": 53,
"ServicePort": {
"name": "dns",
"port": 53,
"protocol": "UDP"
},
"UID": "kubernetes://kube-dns-7549f99fcc-jg2vw.kube-system"
},
"service": {
"hostname": "kube-dns.kube-system.svc.cluster.local",
"address": "10.x.x.x",
"ports": [
{
"name": "dns",
"port": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "TCP"
}
],
"external": "",
"MeshExternal": false,
"Resolution": 0,
"creationTime": "2018-05-11T16:02:05Z",
"Attributes": {
"Name": "kube-dns",
"Namespace": "kube-system",
"UID": "istio://kube-system/services/kube-dns"
}
},
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "3105955977"
},
"az": "europe-west4/europe-west4-a",
"serviceaccount": "spiffe://cluster.local/ns/kube-system/sa/kube-dns"
},
{
"endpoint": {
"Family": 0,
"Address": "10.x.x.x",
"Port": 53,
"ServicePort": {
"name": "dns",
"port": 53,
"protocol": "UDP"
},
"UID": "kubernetes://kube-dns-7549f99fcc-q4mrt.kube-system"
},
"service": {
"hostname": "kube-dns.kube-system.svc.cluster.local",
"address": "10.x.x.x",
"ports": [
{
"name": "dns",
"port": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "TCP"
}
],
"external": "",
"MeshExternal": false,
"Resolution": 0,
"creationTime": "2018-05-11T16:02:05Z",
"Attributes": {
"Name": "kube-dns",
"Namespace": "kube-system",
"UID": "istio://kube-system/services/kube-dns"
}
},
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "3105955977"
},
"az": "europe-west4/europe-west4-a",
"serviceaccount": "spiffe://cluster.local/ns/kube-system/sa/kube-dns"
},
{
"endpoint": {
"Family": 0,
"Address": "10.x.x.x",
"Port": 53,
"ServicePort": {
"name": "dns",
"port": 53,
"protocol": "UDP"
},
"UID": "kubernetes://kube-dns-7549f99fcc-fq2pk.kube-system"
},
"service": {
"hostname": "kube-dns.kube-system.svc.cluster.local",
"address": "10.x.x.x",
"ports": [
{
"name": "dns",
"port": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "TCP"
}
],
"external": "",
"MeshExternal": false,
"Resolution": 0,
"creationTime": "2018-05-11T16:02:05Z",
"Attributes": {
"Name": "kube-dns",
"Namespace": "kube-system",
"UID": "istio://kube-system/services/kube-dns"
}
},
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "3105955977"
},
"az": "europe-west4/europe-west4-b",
"serviceaccount": "spiffe://cluster.local/ns/kube-system/sa/kube-dns"
},
{
"endpoint": {
"Family": 0,
"Address": "10.x.x.x",
"Port": 53,
"ServicePort": {
"name": "dns",
"port": 53,
"protocol": "UDP"
},
"UID": "kubernetes://kube-dns-7549f99fcc-hnrls.kube-system"
},
"service": {
"hostname": "kube-dns.kube-system.svc.cluster.local",
"address": "10.x.x.x",
"ports": [
{
"name": "dns",
"port": 53,
"protocol": "UDP"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "TCP"
}
],
"external": "",
"MeshExternal": false,
"Resolution": 0,
"creationTime": "2018-05-11T16:02:05Z",
"Attributes": {
"Name": "kube-dns",
"Namespace": "kube-system",
"UID": "istio://kube-system/services/kube-dns"
}
},
"labels": {
"k8s-app": "kube-dns",
"pod-template-hash": "3105955977"
},
"az": "europe-west4/europe-west4-c",
"serviceaccount": "spiffe://cluster.local/ns/kube-system/sa/kube-dns"
},

So pilot has the endpoints, a DR change (which triggers full push) fixes it. And the endpoints are stable - so somehow when the connection happened the cluster was not sent.
I don't think the 'incremental' part is the root cause - disabling incremental just makes it that any change of endpoint anywhere in the cluster is doing a full push, hiding the problem.

Current status: it seems the root problem is envoy not getting some endpoints - the CDS cluster is sent ( based on the stats ), Pilot has the endpoints in memory - and it appears that a full push will send them properly.  Both the change in DR, pilot restart or turning off incremental EDS ( which results in frequent updates ) are recovering. So a full push updates - but somehow in some cases it gets an empty list for this specific cluster.
Unfortunately the logs I got have the ADS logging turned off to warning - there are some messages showing what Pilot is sending (stats and timing of the pushes). It would help a lot to repro with this enabled.
Second info would be an envoy config_dump and full /stats from the affected sidecar, and in general additional envoy logs from that job.
Code review: so far nothing interesting, the full push is recomputing clusters, and we send the cached value. But I see no code path where the clusters will end up empty - kubedns is pretty stable.

Found the problem, kube-system is actually excluded by code.
Needs to be fixed in 1.0 and 1.1 branches - seems like a release blocker, but easy to fix (since calls to apiserver and dns can both fail in unexpected ways)

@Stono can you create a service Entry for kube-system with resolution DNS ? I think that should solve the problem

I can, but, I don't understand why this is a problem on one cluster, but not the other two, when they're identical, other than the number of services deployed.
If it was something as deterministic as what you're both saying, i'd see consistently across clusters?

@costinm are you linking a PR to this issue?
Wouldn't want this leaking into 1.1 as it'd make 1.1 unusable for us

@duderino need to create a 1.0.7 with this fix.

1.0.7:  https://github.com/istio/istio/issues/12090

@howardjohn @costinm @munrodg @Stono @utako I think the current state is we're good for 1.1 but still waiting on a fix on the release-1.0 branch before we cut 1.0.7

Yes, this should be fixed and merged in 1.1 now. Please let me know if you get similar issues in 1.1. Fix for 1.0 is not yet ready

What's the situation with a 1.0.7 release?  We've had to turn of incremental eds because of this which creates a lot more push lag for us on our cluster.

@rshriram what do you mean a ServiceEntry for kube-system as that's a namespace?
Can you please show me?

https://github.com/istio/istio/pull/12030 has the change for 1.0.7 if you want to track it

This is now a critical problem on 1.1.0, TCP DNS is broken for us and we can't set the EDS flag to fix it.

To be clear what we see here:

New pod comes up, can make TCP dns queries
Proxy receives a partial config update
TCP dns queries fail

Previously, we set PILOT_DIRECT_EDS=0 to "fix" this, but that isn't an option in 1.1.0.

If i edit the kube-dns service, and add a random annotation, the other pods TCP DNS start working for a short period.  So seems like that service being edited pushes out an update which containers kube-dns.
Then another update will come along, and remove it.

It's worth repeating that i only see this on large clusters.  This one is 553 pods with 387 services.

I'm chatting to @dwradcliffe - he has a cluster of a similar size, (but about 1/4 the number of services) and doesn't see it.  He runs galley, I don't.

@howardjohn is working on this now

Keeping this up to date with slack, I can deterministically recreate this issue.
As you can see, no dns endpoints in the kube-dns cluster:
❯ istioctl proxy-config endpoint monkeynetes-57df648c4d-2cwrp --cluster "outbound|53||kube-dns.kube-system.svc.cluster.local"
ENDPOINT     STATUS     CLUSTER

And digs fails:
[atcloud@monkeynetes-57df648c4d-2cwrp app]$ dig +tcp @<IP_ADDRESS> www.google.com
;; communications error to <IP_ADDRESS>#53: connection reset

If i create these two resources:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: dns
  namespace: kube-system
spec:
  hosts:
  - "kube-dns.kube-system.svc.cluster.local"
  ports:
  - number: 53
    name: tcp
    protocol: TCP
  resolution: NONE
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: disable-mtls-to-dns
  namespace: kube-system
spec:
  host: "kube-dns.kube-system.svc.cluster.local"
  trafficPolicy:
    tls:
      mode: DISABLE

serviceentry.networking.istio.io/dns created
destinationrule.networking.istio.io/disable-mtls-to-dns created

I get endpoints:
❯ istioctl proxy-config endpoint monkeynetes-57df648c4d-2cwrp --cluster "outbound|53||kube-dns.kube-system.svc.cluster.local"
ENDPOINT            STATUS      CLUSTER
<IP_ADDRESS>:53      HEALTHY     outbound|53||kube-dns.kube-system.svc.cluster.local
<IP_ADDRESS>:53      HEALTHY     outbound|53||kube-dns.kube-system.svc.cluster.local
<IP_ADDRESS>:53     HEALTHY     outbound|53||kube-dns.kube-system.svc.cluster.local
<IP_ADDRESS>:53      HEALTHY     outbound|53||kube-dns.kube-system.svc.cluster.local
<IP_ADDRESS>:53      HEALTHY     outbound|53||kube-dns.kube-system.svc.cluster.local

And digs work:
[atcloud@monkeynetes-57df648c4d-2cwrp app]$ dig +short +tcp @<IP_ADDRESS> www.google.com
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>

If I delete those entries:
k serviceentry.networking.istio.io "dns" deleted
destinationrule.networking.istio.io "disable-mtls-to-dns" deleted

Digs continue to work:
[atcloud@monkeynetes-57df648c4d-2cwrp app]$ dig +short +tcp @<IP_ADDRESS> www.google.com
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>

Until I do something to trigger a partial push (deleting another pod in that namespace):
❯ k delete pod monkeynetes-57df648c4d-d9jvh
pod "monkeynetes-57df648c4d-d9jvh" deleted

Then digs start to fail:
[atcloud@monkeynetes-57df648c4d-2cwrp app]$ dig +short +tcp @<IP_ADDRESS> www.google.com
;; communications error to <IP_ADDRESS>#53: connection reset

And the cluster is empty again:
❯ istioctl proxy-config endpoint monkeynetes-57df648c4d-2cwrp --cluster "outbound|53||kube-dns.kube-system.svc.cluster.local"
ENDPOINT     STATUS     CLUSTER

For anyone else experiencing this, adding a ServiceEntry and a DestinationRule if you have mTLS enabled, works around the issue:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: dns
  namespace: kube-system
spec:
  hosts:
  - "kube-dns.kube-system.svc.cluster.local"
  ports:
  - number: 53
    name: tcp
    protocol: TCP
  resolution: NONE
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: disable-mtls-to-dns
  namespace: kube-system
spec:
  host: "kube-dns.kube-system.svc.cluster.local"
  trafficPolicy:
    tls:
      mode: DISABLE

I've spent some time digging into this on Friday, sharing my findings to the date.
One thing that's special about kube-dns service is that it (being a normal DNS server) talks two protocols on the same port. I.e. its Service resource has two "service ports" (UDP and TCP) on the same physical port (53).
From reading the source code it doesn't seem like Istio supports that very well. There are several places where we perform lookups by a port number picking the first port that matches. I have setup a couple of experiments and I'm getting different behavior depending on which port occurs first in the Service resource definition. If UDP goes first (kube-dns case) I couldn't get my service to work even with ServiceEntry (kube-dns however works fine even without one). By the way ServiceEntry helps because it essentially helps to disambiguate the port. This also depends on the order of endpoints returned by K8s APIs, I'm researching whether there's any consistency to it as this might explain why this issue only affects one cluster but not the other two.
We definitely need to clean up the way we handle ports, but that isn't trivial. I'm trying to put together a straightforward/safe changeset to either add deterministic ordering or to simply skip UDP ports when we're looking up ports by a port number/iterating over them etc., still haven't got it working yet.
Regarding "incremental push breaking things after ServiceEntry is removed": with a recent change we're no longer ignoring kube-system namespace, so once ServiceEntry is gone Pilot will pick up kube-dns Service resource with conflicting ports. We need to confirm that both full push and incremental push code paths behave consistently with regards to Service vs ServiceEntry -- something may be off there.
The investigation is still ongoing but this is promising. Initially I suspected some obscure concurrency related issue that potentially may affect any service, not just kube-dns, that would've been much bigger problem, than lack of proper support of two protocols on the same port.

@philrud - thanks for the update.  I'm glad the blast radius of the issue seems to be a bit more confined to kube-dns.  It's just a shame that the blast radius of kube-dns is large - haha!
I do find it odd that I only get it only on the larger more complex clusters so definitely feels like something else is a bit off.
To be clear, we never had the ServiceEntry on 1.0.6 with incremental push turned off, and never saw this problem.  So I agree with "We need to confirm that both full push and incremental push code paths behave consistently with regards to Service vs ServiceEntry".

Not the cause, but related: https://github.com/kubernetes/kubernetes/issues/39188
Bug in kubernetes causes issues with ports overlapping, could be related. Kube-dns will always have both the TCP and UDP port, but when creating own service with both, need to be careful that both are created.

@Stono Can you tell us how to reproduce, I don't think we have found the root cause.

@hzxuzhonghu yeah, you have :-) https://github.com/istio/istio/pull/12766/files

@Stono @howardjohn I think this is ready to be closed?

Yeah we can close, the fix should be in 1.1.3. Feel free to open another issue or reopen if you see this in 1.1.3 again.


New version: Qalculate.Qalculate version 4.7.0

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.4 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/microsoft/winget-pkgs/pull/111359&drop=dogfoodAlpha

/azp run

/azp run


Remove all existing system order ambiguities in DefaultPlugins

Objective
As discussed in https://github.com/bevyengine/bevy/issues/7386, system order ambiguities within DefaultPlugins are a source of bugs in the engine and badly pollute diagnostic output for users.
We should eliminate them!
This PR is an alternative to #15027: with all external ambiguities silenced, this should be much less prone to merge conflicts and the test output should be much easier for authors to understand.
Note that system order ambiguities are still permitted in the RenderApp: these need a bit of thought in terms of how to test them, and will be fairly involved to fix. While these aren't good, they'll generally only cause graphical bugs, not logic ones.
Solution
All remaining system order ambiguities have been resolved.
Review this PR commit-by-commit to see how each of these problems were fixed.
Testing
cargo run --example ambiguity_detection passes with no panics or logging!

this seems like a good middle ground -- declare bankruptcy on existing ambiguities but enforce no net new ones going forward. we can always go back and develop better fixes for these later.

I'm actually pretty happy with most of these fixes! The rendering sets are a bit weird and implicit, but I think ultimately correct. I'd like to make text and text2d play nicer with each other / be less distinct, but that's another big project. Same with making UI animatable.
My main complaint is around handle_internal_asset_events: dropping an exclusive system in the middle of PreUpdate feels very messy to me and will definitely impact parallelism in an unpredictable way.


Add missing setting to desktop file

This is an AppImage that should AFAICS not be integrated by tools like AppImageLauncher. Therefore this key will make tools like AppImageLauncher ignore this AppImage.
See https://github.com/TheAssassin/AppImageLauncher/issues/233 for reference.

master branch is heavily outdated ATM currently I have already added 925a193d700b428cdcbea748bff63907c959d5ba these changes are in staging branch and builds from continuous are built from there.


Staging fails with "failed to create container", can't set cgroup config, (bosh-lite environment)

Issue
cf push fails for most simple apps (here: "cf-ex-php-info" example app) pushing to a fresh deployed "cloud foundry" running on a local "bosh-lite" installation built with the docker CPI.
Problem seems to be, that the cf runner(?) tries to configure control group properties that can't be set when already running in a container.
Context
I created for testing a local, docker-based bosh-lite environment.  I used initially the "bucc" instructions from here: https://github.com/starkandwayne/bucc with the "docker" CPI. (The VirtualBox CPI is currently broken, see here: https://github.com/cppforlife/bosh-virtualbox-cpi-release/issues/18.)  I then followed the instruction here: "https://github.com/cloudfoundry/cf-deployment/blob/master/texts/deployment-guide.md" to deploy the cloud foundry from "https://github.com/cloudfoundry/cf-deployment".
I retried with a plain bosh-lite from "https://github.com/cloudfoundry/bosh-deployment".
Both deployments went without problems I know of.
Steps to Reproduce

Create a local bosh-lite environment using the "docker" CPI

either using the repository from here https://github.com/starkandwayne/bucc
or using the instruction here https://bosh.io/docs/bosh-lite/
I tried both ways with identical result

Deploy cf against the director from step 1
Configure an initial org and space and login
Run git clone https://github.com/cloudfoundry-samples/cf-ex-php-info and enter the work tree
Execute cf push

I created a gist with my deployment steps, you can run that script to exactly redo the steps I used: https://gist.github.com/ipa-bak/732310132381c65b6636f9ce8f79ac54#file-make-bosh-cf-simple-sh . Save as "make-bosh-cf-simple.sh" and then adapt to your local system at the top of the script, then run with ./make-bosh-cf-simple.sh bosh
I attached below the manifest used to create the director and the one used to deploy cf:

Director manifest: director-manifest.yaml.txt
cf-deployment manifest: cf-manifest.yaml.txt

Expected result
The app gets staged and eventually started.
Current result
The cf push returns with error StagingError - Staging error: staging failed
It produces the following error messages:
Pushing app cf-ex-php-info to org cf-ex / space cf-ex-space as admin...
Applying manifest file /home/cf-test/workspace_2020-10-03-223039/cf-ex-php-info/manifest.yml...
Manifest applied
Packaging files to upload...
Uploading files...
 3.21 KiB / 3.21 KiB [==============================================================================================] 100.00% 1s

Waiting for API to complete processing files...

Staging app and tracing logs...
   Downloading php_buildpack...
   Downloaded php_buildpack
   Cell b150fae6-354c-4217-b1fb-8b91c63f8d34 creating container for instance afec48b7-5b88-49f4-88d5-9fe66bb29e04
   Cell b150fae6-354c-4217-b1fb-8b91c63f8d34 failed to create container for instance afec48b7-5b88-49f4-88d5-9fe66bb29e04:
      OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:459: container init
      caused: process_linux.go:422: setting cgroup config for procHooks process caused: failed to write "1073741824" to
      "/sys/fs/cgroup/memory/docker/4e78cd5bc062e646907456357396a0f6c9715e46fa8bca62b0159f765d77f736/
      19de8c8c-3b71-448c-7b56-de1dc5baef61/garden/afec48b7-5b88-49f4-88d5-9fe66bb29e04/memory.memsw.limit_in_bytes":
      open /sys/fs/cgroup/memory/docker/4e78cd5bc062e646907456357396a0f6c9715e46fa8bca62b0159f765d77f736/
      19de8c8c-3b71-448c-7b56-de1dc5baef61/garden/afec48b7-5b88-49f4-88d5-9fe66bb29e04/memory.memsw.limit_in_bytes:
      permission denied: unknown
   Cell b150fae6-354c-4217-b1fb-8b91c63f8d34 destroying container for instance afec48b7-5b88-49f4-88d5-9fe66bb29e04
   Cell b150fae6-354c-4217-b1fb-8b91c63f8d34 successfully destroyed container for instance afec48b7-5b88-49f4-88d5-9fe66bb29e04

The lines from Downloading php_buildpack... to successfully destroyed container repeat four times.
I am not aware of other relevant logs, please let me know if I should look for a specific log file.
Possible Fix
No specific suggestion, but a concourse installation (coming with the "bucc" installation method) running in the same director environment does work and can run its build containers inside the docker container.

Hello,
It looks like you don't have swap enabled on the diego-cell (which is probably expected with the docker cpi). Because of this garden can't write the swap memory limit in the cgroups file. We have a property in Garden for exactly this case here. You can enable it with the following ops file, that you can pass when deploying cf:
---
- type: replace
  path: /instance_groups/name=diego-cell/jobs/name=garden/properties/garden/disable_swap_limit?
  value: true

After redeploying it should work.
@cloudfoundry/cf-garden cc

Hello @mnitchev , thanks for the suggestion.  I can confirm it now works for me.
It would be nice if links to such documentation could be included in the main ReadMe.md (here or in cf-deployment).
Anyway, fell free to close this issue as solved.


The docs and preview web link is broken

@okanji  I found that this link still works: https://bettertyped.github.io/react-zoom-pan-pinch/?path=/story/docs-introduction--page


chore: remove faker.random. deprecation warnings

Using this module produces quite a lot of console logs because of deprecations in faker.
This just replaces those deprecations for new API.
 console.log
    Deprecation Warning: faker.random.number is now located in faker.datatype.number

      at Random.number (../node_modules/faker/lib/random.js:37:13)
          at Array.map (<anonymous>)

Codecov Report

Merging #97 (cfcee5a) into main (7517967) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #97   +/-   ##
=======================================
  Coverage   99.63%   99.63%
=======================================
  Files          14       14
  Lines         272      272
  Branches       78       78
=======================================
  Hits          271      271
  Misses          1        1

Impacted Files
Coverage Δ

src/config.ts
100.00% <100.00%> (ø)

src/fake.ts
100.00% <100.00%> (ø)

src/fakers/array.ts
100.00% <100.00%> (ø)

src/fakers/base.ts
98.50% <100.00%> (ø)

src/fakers/boolean.ts
100.00% <100.00%> (ø)

src/fakers/mixed.ts
100.00% <100.00%> (ø)

src/fakers/number.ts
100.00% <100.00%> (ø)

src/fakers/object.ts
100.00% <100.00%> (ø)

src/fakers/string.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7517967...cfcee5a. Read the comment docs.


Problems encountered

Hello, I have encountered some small problems when using SwinV2 these days. I would like to get your answers here.

when my input size is small, such as 96*96, window_szie=8, will appear
https://github.com/ChristophReich1996/Swin-Transformer-V2/blob/cff3824f5d75dfd93553867efcf53562c24dd555/swin_transformer_v2/model_parts.py#L289
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

In this paper, sequential self-attention calculation is used to save GPU memory, but in large-size image application, I set Sequential_self_attention =True, which will result in OOM. Set Sequential_self_attention =False does not.

when i update the code https://github.com/ChristophReich1996/Swin-Transformer-V2/blob/cff3824f5d75dfd93553867efcf53562c24dd555/swin_transformer_v2/model_parts.py#L207,GPU memory usage also soared further.

I am currently experimenting with the application of swinV2, an efficient and memory saving network, to 3D data, therefore pay more attention to the occupation problem on the display memory.

Hi @834799106,
Thanks for your interest in my reimplementation, and thanks for pointing out these issues. Feel free to leave a star there if you liked the code :)

I have been able to reproduce the bug. It appears to be that for some (not all) model configurations and input shapes the input tensor to the window attention is not contiguous in memory (see for more information), as also stated by the error message. Why this is, especially only for some shapes, is currently not clear to me but simply replacing the view with a reshaping solved the issue.
As mentioned in the README this is a very experimental implementation, and so the sequential self-attention implementation. The paper only states that they are employing a sequential self-attention but nothing more (see section 3.4, paragraph sequential self-attention computation of the paper). One could imagine more possible implementation, for example, sequential iterating over the number of attention heads. For me, the most obvious implementation was to iterate over the query tokens. I don't think that the sequential self-attention itself saves the memory but the combination with clever checkpointing. This is because in the current sequential self-attention probably all tensors are stored for the backward pass. But I haven't had time to look deeper into it. Maybe you find a way to make the sequential implementation to be more memory efficient, probably in combination with checkpointing. Feel free to open a pull request if you found a solution!
This change refers to issue #1. As mentioned there every entry of the attention matrix gets normalized now, resulting in a bigger tensor. This also leads to a bigger memory footprint.

Cheers
Christoph


About repetitive help requests.

There are at least 5-6 requests every day in python discord, asking how to install python, which version of python they want to install, why "pip" is not working and someone directly pinging the help channel without doing any research for their errors. While I don't think we can deal with people trying to get spoonfed but at least, we do something about the "pip", python versions, and how to install python and IDEs.

We could create tags for how to install Python, how to install packages... See the issue #810.

So what is your proposed solution to this? Is it to create tags as AtieP suggested?

Writing tag will be good for this bcz tag main purpose is for info only

Stale issue, closing for now. Creating a tag for this would be perfectly fine.


(#72) Comments after string literals raises an error

Ran into issue while implementing #62 where I could not add a comment after a string literal when doing a %bind.
This change will chop of all comments for any line
Close #72

@derpandaz looks good to me! :+1: Thank you for your contribution!

@derpandaz looks good to me! :+1: Thank you for your contribution!


Update data-storage-privacy.md

Added details about advanced hunting data

@JohanFreelancer9 Please copy review

Issue#https://github.com/MicrosoftDocs/microsoft-365-docs/issues/4540

@mjcaparas Please merge this PR. Thanks


perf: use more performant data structures

#519  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @RebeccaStevens and the rest of your teammates on  Graphite

Merge activity

Oct 7, 11:11 PM EDT: @RebeccaStevens merged this pull request with Graphite.


[Layer] Expose LayersManager children and host styling

Feature description
Problem
I recently removed all z-indexing from my app and converted to the Layer API which is pretty neat!
However, there was one styling issue that caused some problems for me. The fact that the LayersManager throws all of the Layers into a div that cannot be styled:
https://github.com/uber-web/baseui/blob/master/src/layer/layers-manager.js#L29
This caused many issues for me, but one specific one is using the LayersManager component in a flex container: https://codesandbox.io/s/layer-basic-usage-re30i
One other issue I recently ran into is not being able to set overflow: scroll for any descendants of LayersManager because I cannot set the height of the LayersManager div: https://codesandbox.io/s/layer-basic-usage-o1y78
Proposed Solutions

Expose styling for the host div and also the div that holds children
Don't put the children in a div, expose styling to host div

Hey @austintackaberry 👋
Could you potentially achieve what you want by not using a LayersManager and using an overridden Block? Then you could use that Block as a custom mountNode for Layer?
That might be a quick work around.

Hey @jh3y!
I was able to find workarounds for my usecase. I only added the problems I faced and the codesandboxes in order to bolster my opinion that this feature is useful 🙂

@austintackaberry @jh3y should we close this?

It doesn't matter to me, but shouldn't it stay open until the feature is implemented?

Thanks for jumping on this! Is there a way for me to see which version will include this feature?


Support for IntelliJ 2022.2

The plugins currently fails with "Incompatible: requires IDE build 221.* or earlier" on Intellij 2022.2.

I removed the upper version limit, which should reduce the number of times I need to update the plugin.
The new version should be available in a day or two on the Jetbrains marketplace. This same version is also available in the releases section of this project:
https://github.com/AVeenstra/RunAnything/releases/tag/1.11


Calculate changes week over week

Using the list of week objects, need to implement code which calculates the change in weight, body_fat, and lean_body_mass week over week.

Closed by commit 12ab2b6599c06c8e89454511d0568a192c8d44ef.


The new plugin does not support the old version of Joplin

Hello, my joplin software version is 2.11.11, when I intalled 2.10.2 of freehand drawing plugin, it prompted me to upgrade Joplin software. But I don't want upgrade Joplin software as My computer is win7 system.  Can you give me old version freehand drawing(jpl file) ? I can not generate jpl from the code you pubilished.

I've built a copy of the plugin from this branch, which disables the features that require newer Joplin versions. I'm attaching the built plugin below:
io.github.personalizedrefrigerator.js-draw.jpl.zip

I'm closing this as resolved. Please let me know if something else needs to be done.


CP-11224: Missing pages in the user manual wiki

From unknown CodePlex user on Friday, 12 September 2008 06:08:28
Some links on the user manual wiki take you a page displaying this message:
The wiki page  does not exist

e.g.
User Manual -> Observer Concepts -> VertexPredecessorRecorderObserver
User Manual  -> Graph Concepts  ->  Traversal Concepts -> IBidirectionalGraph

Form unknown CodePlex user on Monday, 15 September 2008 07:53:43
in progress

Form unknown CodePlex user on Sunday, 29 March 2009 23:09:46

Form unknown CodePlex user on Sunday, 29 March 2009 23:10:02

Form unknown CodePlex user on Sunday, 29 March 2009 23:14:54

Form unknown CodePlex user on Sunday, 29 March 2009 23:21:27

Form unknown CodePlex user on Thursday, 21 February 2013 17:44:57

Form unknown CodePlex user on Thursday, 16 May 2013 04:59:29


fix: ignore npm install result as semantic-release-gh-pages-plugin ex…

…pects package.json to exist but cargo publish doesn't like it not being in git.

:tada: This PR is included in version 1.1.5 :tada:
The release is available on:

v1.1.5
GitHub release
v1.1.5

Your semantic-release bot :package::rocket:


Fix segfault in ArrayVec<Vec>::into_vec

Fixes #599

The new asserts are enough for this to be covered by existing tests. The error was also detected by miri, and this is now fixed.

Out of curiosity, I notice that after the fix the code is still constructing a prefix of a slice using from_raw_parts_mut in DerefMut/Deref, and only checking bounds in debug builds. What performance benefit did you measure from skipping the bounds checks in release builds, rather than using bounds-checked prefixing (&mut slice[..len]), which would turn this class of mistake into a panic rather than a segfault?
(Just finished tracking down an intermittent crash in our debugger due to this.)

I didn't check for this PR, but when the ArrayVec implementation was first introduced it had a measurable difference in the existing benchmarks.


FR: MISP export

Export to MISP the selected row
Export to MISP all bookmarked (#42) rows
Export to MISP the timeline adding relationship

1 done. Selecting "Export MISP" under actions on a dumped file you can export it to misp:


Suggestion: move bio-oil-3 out of coal-mk02 into either bioprocessing or coal-mk03

Mod source
PyAE Beta
Which mod are you having an issue with?

[X] pyalienlife
[ ] pyalternativeenergy
[X] pycoalprocessing
[ ] pyfusionenergy
[ ] pyhightech
[ ] pyindustry
[ ] pypetroleumhandling
[X] pypostprocessing
[ ] pyrawores

Operating system

=Windows 10

What kind of issue is this?

[ ] Compatibility
[ ] Locale (names, descriptions, unknown keys)
[ ] Graphical
[ ] Crash
[X] Progression
[X] Balance
[ ] Pypostprocessing failure
[ ] Other

What is the problem?
Raw Coal - Stage 2 (coal-mk02) contains the coarse-coal-to-coal, coarse-coal, and bio-oil-3 recipes
bio-oil-3 one triggers a dependency on bioprocessing, which pushes it some 4-5000 logistics packs deeper in the tree than, say, the various other early stage 2 ore processing techs (lead, copper, iron, nickel, tin) which also makes it an 800 pack tech on its own
would suggest moving bio-oil-3 into either bioprocessing, or coal-mk03
Steps to reproduce
load in, observe tree
Additional context
n/a
Log file
n/a

seaweed-mk02 has a similar issue, forcing the molybdenum dependency onto the seaweed-2 recipe due to the seaweed-mk02 recipe being in the same tech

Fixed. Thanks for the report.


ERROR in running examples/benchmark.py

The error traceback is :
Traceback (most recent call last):
File "examples/benchmark.py", line 10, in
from habitat.sims.habitat_simulator.actions import HabitatSimActions
File "/data1/fuyu/habitat-lab/habitat/sims/habitat_simulator/actions.py", line 12, in
import habitat_sim
File "/data1/fuyu/.conda/envs/Object_Goal_Navigation/lib/python3.8/site-packages/habitat_sim-0.1.5-py3.8-linux-x86_64.egg/habitat_sim/init.py", line 14, in
from habitat_sim.nav import *
File "/data1/fuyu/.conda/envs/Object_Goal_Navigation/lib/python3.8/site-packages/habitat_sim-0.1.5-py3.8-linux-x86_64.egg/habitat_sim/nav/init.py", line 1, in
from habitat_sim._ext.habitat_sim_bindings import (
ImportError: /data1/fuyu/.conda/envs/Object_Goal_Navigation/lib/python3.8/site-packages/habitat_sim-0.1.5-py3.8-linux-x86_64.egg/habitat_sim/_ext/habitat_sim_bindings.cpython-38-x86_64-linux-gnu.so: undefined symbol: __cpu_model
I don't know how to fix it and can't find any solution in other issue.
I'm using this repository: https://github.com/devendrachaplot/Object-Goal-Navigation
with habitat_sim-0.1.5, pytorch==1.6.0, torchvision==0.7.0, cudatoolkit==10.2.

@keeper1314 this looks like a compatibility issue with gcc compiler. Did you install habitat-sim-0.1.5 from conda or build from source? What gcc version is installed on your system?

I build habitat-sim-0.1.5 from source. gcc version is 9.1.0.


feat(sklplots): add confusion matrix normalized option

Adds an option to normalize confusion matrix like this:
live.log_sklearn_plot(
        "confusion_matrix",
        [0, 0, 1, 1],
        [0, 1, 1, 0],
        name="confusion_matrix_normalized",
        normalized=True,
    )

Quite handy to see a better heat map in case of classes being highly misbalanced.

[X] ❗ I have followed the Contributing to DVCLive
guide.

[X] 📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in
dvc.org and linked it here.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

Docs PR is here https://github.com/iterative/dvc.org/pull/4486


Create own helpers

Hi Virk,
I would like to separate tools or business logic from data models, using some kind of Helper Classes, just like Yii2 Helpers where you can create helpers for generic tasks.
I saw that Adonis has helpers already but didn't find a way to define custom ones, what is you recommendation on this matter? I don't want to bloat my controllers/models with code that can be abstracted and encapsulated outside them.
I know that one possible approach is to create a Provider but that doesn't work with business specific logic.
Thanks.

@lordpixel Any specific reason why providers does not work with business specific logic?
Providers are nothing but a way to tie your own classes/modules to the AdonisJs app life cycle, apart from that you can write a provider for anything. To keep them clean, do not hardcode anything to your provider and instead make use of DI to interact with other service providers.

Closing as there is no activity.

@thetutlage can you please refer to http://stackoverflow.com/questions/28290332/best-practices-for-custom-helpers-on-laravel-5 I want to create a helper class which I can call anywhere including view, how to create such helper?

found it - http://www.adonisjs.com/docs/3.0/views#defining-globals

found it - http://www.adonisjs.com/docs/3.0/views#defining-globals

It seems the link is broken.


FR: Showing alias of notes when generating local index

I often have more meaningful names in alias. It would be nice if there is an option to include alias as well, when we generating local index or global index!

Thanks for the request :) I was busy working on the index when you posted this, so I thought I'd give it a try!
I think it's working in 0.9.19, will you give it a go and let me know? You have to enable it as a setting under Create Index

Fantastic! It works smoothly :) Thanks so much for the quick update :)


Rails app templates - Error when generating 02-pretty

I get the following error when generating 02-pretty:
    generate    kaminari:views
Could not find generator 'kaminari:views'. Maybe you meant 'migration' or 'controller' or 'css:assets'
Run `rails generate --help` for more options.
        gsub    app/views/kaminari/_paginator.html.erb

It appears the root cause is that the kaminari gem is never actually added to my Gemfile.

Thanks for the fix, merged!


sync fails on Juju 3.5.3

Using:
channels:
  latest/stable:    <IP_ADDRESS> 2024-05-15 (306) 114MB -
  latest/candidate: <IP_ADDRESS> 2024-06-14 (365) 114MB -
  latest/beta:      <IP_ADDRESS> 2024-06-14 (365) 114MB -
  latest/edge:      <IP_ADDRESS> 2024-07-25 (379) 115MB -
installed:          <IP_ADDRESS>            (379) 115MB -

I am getting an odd error when I've ran juju version 3.5.3:
jhack sync -s ./src -s ./lib/charms/opensearch/ opensearch returns a KeyError: "applications" on: https://pastebin.ubuntu.com/p/GdnB37rK3Y/
The model is set as my default and the opensearch application does exist.
Rolling back to juju version 3.4.5 resolves the issue.
Also using Juju 3.4.5, I also created an empty model and ran the command above (so I could trigger the same KeyError as above). I can see the error message is more complete: https://pastebin.ubuntu.com/p/6rrGhk8qyq/.
It does find more information about the model:
           status = {
                    │   'model': {
                    │   │   'name': 'test-scale-to-one-and-back-canv',
                    │   │   'type': 'iaas',
                    │   │   'controller': 'localhost-localhost',
                    │   │   'cloud': 'localhost',
                    │   │   'region': 'localhost',
                    │   │   'version': '3.4.5',
                    │   │   'model-status': {
                    │   │   │   'current': 'available',
                    │   │   │   'since': '07 Aug 2024 18:30:44+02:00'
                    │   │   },
                    │   │   'sla': 'unsupported'
                    │   },
                    │   'machines': {},
                    │   'applications': {},
                    │   'storage': {},
                    │   'controller': {
                    │   │   'timestamp': '18:31:19+02:00'
                    │   }
                    }

uhm that's odd, indeed. It looks like it has issues running juju status. Was the model still being set up by any chance?

can you share the output of juju status --format json?

I suspect it may have been an issue of the command being run too soon after model creation or another similar glitch. Without a reproducer and more data I can't help.
Can you also try from edge?

Hi @PietroPasotti I can reproduce with the following steps:
sudo snap install juju --channel=3.5/stable
juju bootstrap localhost

juju add-model test2
juju deploy ubuntu

This is the juju status:
$ juju status
Model  Controller           Cloud/Region         Version  SLA          Timestamp
test2  localhost-localhost  localhost/localhost  3.5.4    unsupported  19:11:21Z

App     Version  Status  Scale  Charm   Channel        Rev  Exposed  Message
ubuntu  24.04    active      1  ubuntu  latest/stable   25  no

Unit       Workload  Agent  Machine  Public address  Ports  Message
ubuntu/0*  active    idle   0        <IP_ADDRESS>

Machine  State    Address       Inst id        Base          AZ  Message
0        started  <IP_ADDRESS>  juju-2cb202-0<EMAIL_ADDRESS>    Running

Now, if I run jhack, here is the output error I get: https://pastebin.ubuntu.com/p/D9tf9kMBJd/

I am running that test in an AWS instance with Ubuntu 22.04

Refreshed to edge  (<IP_ADDRESS>.2) and still seeing issues.


New Poetry Python Project Scaffold

Uses poetry for dependencies management
Added Bot workspace for discord bot
Added FastApi server to handle bot api requests
Added MKDocs for documentation for discord bot commands
Removed some Docker and other files for simplicity now will add later with proper files and configuration

This Closes #2

Will share further details in the discussion tab about the architecture?


Horizontal scrolling broken when used with django-admin-interface

I have an issue open over at django-admin-interface because I'm not sure which side to report this to.
When django-import-export is paired with django-admin-interface, the ability to scroll horizontally on the before/after preview screen doesn't work.

Can you retry with v3.  There were some fixes to help with integration with other packages.  That may help.

https://stackoverflow.com/questions/67432186/how-to-enable-horizontal-scrolling-in-django-admin-list-view

https://stackoverflow.com/questions/67432186/how-to-enable-horizontal-scrolling-in-django-admin-list-view

we updated to 3.2 and our team is still reporting the same issue.

Closing - seems like this was fixed in django-admin-interface


Error: EPERM: operation not permitted, open 'C:\Users(Username)\AppData\Local\Temp\tileset_0.png'

So, here is my bug: when I try to build a game, it will say this:
Error: EPERM: operation not permitted, open 'C:\Users\VRFAN\AppData\Local\Temp\tileset_0.png'
When I opened it, it looked like a corrupted image. Is there a solution?

In the first line, I forgot to put: when I try to build and run a game

@BugTellingFairy Please follow the submissions template which includes version and os, it keeps bug reports clean and organized.
Your issue is most likely a lack of admin privilege, try to run gbstudio as admin, or reboot your computer.

I rebooted my pc and tried to run it, and used admin, but still won't work. btw I'm on a Windows 8.1 Build 9600

Case dismissed. I deleted tileset_0.png and reinstalled the program.


EmitterOpRandomMinMaxConfig type correction

Added the description of the "int" boolean property that was missing

Shouldn't it be

If true, only integers are selected from the range

or something similar?

Shouldn't it be

If true, only integers are selected from the range

or something similar?

I took "EmitterOpRandomStartEndConfig.js" description of random as reference.

But that's completely different. Here int: true means force the random value to be an integer.

But that's completely different. Here int: true means force the random value to be an integer.

Oh, yeah, I misread the code. I'm gonna change it.

👍


Seems like it can't handle .d.ts files

I have in my tsconfig.json:
"typeRoots": ["node_modules/@types", "src/types"],

and have a file src/types.d.ts which contains global types like:
declare global {
	interface Window {
		customProp: string;
	}
}

and when I use  tsu-node ./index.ts it works well before I import the file which trying to use that Window['customProp'] as type, then I get errors:
error TS2339: Property 'customProp' does not exist on type 'Window'.
error TS2339: Property 'customProp' does not exist on type 'Window & typeof globalThis'

Hello! Thanks for the contribution :)
Unfortunately,  handling global types is an issue from ts-node library. For optimization it only loads files which are referenced from the file it has to execute in the first place (index.ts in this case) or it's imports.
To fix the issue I added an loadAllFiles option for each project (node edition). With it enabled all project files are preloaded according to include, exclude and files options in tsconfig.json, thus no global types are lost.
Secondly, there's an issue with your definition: add an export {}; line in the end. And if you declare variables, just use declare var myVariable: type instead of declare global { ... }.


Apply the DRY principle and add better formatting

2 big objects
enToFaDictionary and faToEnDictionary are 2 big objects. With their keys and values being the exact reversed version of each other! So by reversing one (in this case enToFaDictionary) and assigning it to the other, we still get the same result only with much fewer lines of code (63 to be exact).
Side Note: This could be accomplished using Array.reduce method and more advance ES6 features as well. A for loop is the simplest one.
using quotes for object keys
unless you want to use a key that’s not a valid JavaScript identifier, wrapping object keys in quotes is not necessary and it doesn't make a difference.
Side Note: the JSON data exchange format does require double quotes around identifiers (and does not allow single quotes).
semicolons
Some lines were missing it, my prettier automatically added it to them.

Hi @JalalHamed and thanks for contributing❤
You have made amazing changes and improved the performance of the code🚀
For 2 big objects part:
I decided to save two objects because I thought that "for loop" will cost me more than storing them in memory. but for now, I think it would work fine and not get performance issues. and of course, it follows the DRY principle.
Please review this thread:
https://github.com/hadizz/persian-typo-mistake-extension/pull/1#discussion_r785133590
For the prettier part:
Could you please add your prettier configs to the project? so that we have some convention between contributors.

For sure. I'm on it.

Thanks for helping🙏🏻 I checked the pull request and I think it's ready to be merged.


Worker pool and message distribution questions

Hi,
I am trying to figure out a few things to understand whether I am using hutch wrong, I am not understanding hutch's opinions, not understanding rabbitmq, or thinking about features that don't exist but could/should (happy to work on those).
1. More than 1 consumer running at a time
I am noticing that when I run hutch, there doesn't seem to be much parallelization in terms of message consumption. I may be wrong, but I think 1 message is received and processed at a time.
I see that Bunny has a whole implementation of woker pools. and it doesn't seem like Hutch is taking advantage of it.
From my quick experimentation, it shouldn't be much more complicated than passing (nil, 10) to open_channel on this line.  Of course 10 should come from Hutch::Config, but what it represents is the consumer_pool_size
2. Balanced distribution from all the channels
This one I am having a hard time understanding, but here is what I see:
I have two queues, A and B with many jobs in each of them. When Hutch is running, it FIRST consumes messages from queue A, before starting consuming from queue B.
My understanding is that queue A and B should be publishing messages to its subscribed consumers at a relatively balanced rate. Not one after the other.
I don't know taht much about rabbitmq/bunny/hutch so I am having a hard time figuring out where this is coming form or if I am even expecting the right thing.
The only thing I could find about this was:

When doing so, it is important to understand that, in AMQP 0-9-1, messages are load balanced between consumers and not between queues.

This was in the direct exchange section. I think it applies here too since we passed the exchange step at this point...
I would really appreciate any comment about this. Happy to work on whatever the repo maintainers think could make its way into the codebase, but I'd need a little direction.
Thanks!

Bunny uses consumer thread pool of size 1 by default. Why? Because with concurrent processing you get race conditions and most Ruby developers are not prepared for that. Hutch can expose a setting to configure this.
The 2nd thing makes no sense. Hutch uses a single channel without any prefetch limit by default (configurable with :channel_prefetch) and there is no guarantee of ordering of deliveries for different consumers.
Messages are load balanced between consumers. Consumers consume from queues. This is after routing so exchange type makes no difference. However messages got to the queue, if there's more than one consumer and prefetch allows for delivering more messages, deliveries will round robin.

@michaelklishin thanks for your response it helps.
1
I understand the race condition concerns. And another approach could be to run two (or more) times hutch. However, when I tried that, I noticed that only my first hutch was consuming messages, and the other ones were just idling. If I killed the first hutch, then the second instance would start getting messages.
Is that the expected behavior?
2
I understand that the way I phrased my comment was wrong. But what I was trying to get at, was that I expected deliveries to round robin (eg, 1 from queue1, 1 from queue2, 1 from queue1, ...), but that is not what I observe.
I had 14k messages in two different queues. My consumer for the second queue only started receiving messages after the 14k messages in the first one were delivered.
I am trying to understand why this happens, as this is not what I expected to see happen.

If 2 consumers share a queue, then both should consume messages. In fact, this is a common test scenario for me.
If you consume from N queues, expect deliveries in a random order (per-queue ordering is guaranteed, of course).

@midu I saw this exact scenario when I was creating a PoC for hutch.
Here is the scenario that duplicated the behavior:

Start Hutch worker that's consuming 'a.b.c'.  Add a 2 second delay in the process so you can see it happen.
Publish 100 messages.
Watch worker start consuming messages from 'a.b.c'.
Start another hutch worker that's also consuming from 'a.b.c'.  Notice, it's not consuming messages.
Publish another 100 messages.
2nd worker starts consuming messages.

I was able to fix this behavior in my test code by adding the following configuration item:
# Prefetch is important.  It guarantees that if you add
# multiple consumers to a single topic queue, then they
# will share work.  Without this setting, there was
# an issue where consumers started *after* the message
# was published were not sharing work with consumers
# that were stated *before*.
#
Hutch::Config.set(:channel_prefetch, 1)

Disclaimer:  I'm not a hutch or Rabbitmq expert.  I'm barely a novice.  But I noticed that setting this configuration option, fixed the issue for me.  It makes sense, though.  Hutch goes with a default for 0 on the prefetch configuration.    See https://github.com/gocardless/hutch/blob/master/lib/hutch/config.rb#L37.  My guess is the Bunny library is making some optimizations here for consuming worker threads.  Dunno.


tensorman: init at 0.1.0

Motivation for this change
Easy management of Tensorflow containers with Docker as the backend.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
@Lassulus
@mmahut
opening a new PR was just easier than rebasing. I hope you don't mind.
cc @

@GrahamcOfBorg build tensorman

@GrahamcOfBorg build tensorman

can you squash the last 2 commits and make the commit message say: tensorman: init at 0.1.0

the commit message needs to change. right now it says: add tensorman but it should say: tensorman: init at 0.1.0

fixed


Sometimes Wheel don't react to onChange

I've used code like this:
const [hsva, setHsva] = useState({ h: 214, s: 43, v: 90, a: 1 })

const onChangeColorPicker = (event: any) => {
    setHsva({ ...hsva, ...event.hsva })
  }

return (
 <Wheel className={"wheel"} color={hsva} onChange={(event) => onChangeColorPicker(event)} />
)

At times, when I simply click (not click, hold, and move) on the wheel, the color doesn't update immediately. I find that I have to click on the same spot again to trigger the onChange event.
Do I need to use another approach?

@spl1nt3r99 I can't reproduce the problem you describe.
You can give me an example that can be reproduced


Rotate Password

Need to rotate the password on the accounts found here to comply with new MS requirements.
Release Note Category

[ ] Feature changes/additions
[ ] Bug fixes
[x] Internal Infrastructure Improvements

Release Note Description
No release notes.

Should we use secret-manager instead of directing to coreidentity? Only owners will be able to see the list of non people account, should that list be explicitly stated instead of a link that only certain folks will be able to access?

@ilyas1974, as the list of non-personal accounts at https://coreidentity.microsoft.com/manage/service is user-specific, how do we ensure we have all accounts covered? I assume you have ownership of (almost) every non-personal account we use, but how can we make it certain? Cheers!

Thanks Ilya, I confirm the account I see in coreidentity is captured on that page you reffered.

Reopening to cover one additional account that was originally missed.

Rotated and confirmed it's listed in a secret-manager manifest.


[BugFix] Fix RoboHiveEnv tests

Description

Make RoboHiveEnv tests actually run correctly when executed
Fixed a bug in testing of envs without cameras
Refactor and simplify test case

Motivation and Context
Currently, the tests for RoboHiveEnv aren't actually executing correctly.  When the env is instantiated, the from_pixels key isn't actually used, and none of the envs are tested with from_pixels=True.
After the test code is fixed, I get 9 failures for the RoboHive tests, all of which seem to be caused by an issue with DKitty environments.
Short Test log:
================================================= short test summary info =================================================
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientFixed-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientRandom-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientRandom_v2d-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandFixed-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandRandom-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandRandom_v2d-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkFixed-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkRandom-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkRandom_v2d-v0-True] - gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed ...

Full traceback:
-------------------------------------------------- Captured stdout call ---------------------------------------------------
Registered a new env-variant: A:head-A:left-A:right-A:tail-A:tracki-A:tracki-DKittyWalkRandom-v0
________________________________ TestRoboHive.test_robohive[DKittyWalkRandom_v2d-v0-True] _________________________________
Traceback (most recent call last):
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/gym/envs/registration.py", line 120, in spec
    importlib.import_module(mod_name)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'A'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/runner.py", line 342, in from_call
    result: Optional[TResult] = func()
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/runner.py", line 263, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 138, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 121, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/threadexception.py", line 87, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/threadexception.py", line 63, in thread_exception_runtest_hook
    yield
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 121, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/unraisableexception.py", line 90, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/unraisableexception.py", line 65, in unraisable_exception_runtest_hook
    yield
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 121, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/logging.py", line 839, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/logging.py", line 822, in _runtest_for
    yield
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 121, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/capture.py", line 882, in pytest_runtest_call
    return (yield)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 121, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/skipping.py", line 256, in pytest_runtest_call
    return (yield)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/runner.py", line 178, in pytest_runtest_call
    raise e
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/runner.py", line 170, in pytest_runtest_call
    item.runtest()
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/python.py", line 1831, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 138, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/_pytest/python.py", line 194, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/sriramsk/desktop/torchrl/test/test_libs.py", line 3355, in test_robohive
    env = RoboHiveEnv(envname, from_pixels=from_pixels)
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/libs/robohive.py", line 54, in __call__
    instance: RoboHiveEnv = super().__call__(*args, **kwargs)
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/libs/gym.py", line 543, in __call__
    instance: GymWrapper = super().__call__(*args, **kwargs)
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/common.py", line 166, in __call__
    instance: EnvBase = super().__call__(*args, **kwargs)
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/libs/gym.py", line 1277, in __init__
    super().__init__(**kwargs)
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/libs/gym.py", line 731, in __init__
    super().__init__(**kwargs)
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/common.py", line 2993, in __init__
    self._env = self._build_env(**kwargs)  # writes the self._env attribute
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/libs/robohive.py", line 201, in _build_env
    env = self.lib.make(
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/gym/envs/registration.py", line 156, in make
    return registry.make(id, **kwargs)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/gym/envs/registration.py", line 100, in make
    spec = self.spec(path)
  File "/home/sriramsk/miniconda3/envs/torch_rl/lib/python3.9/site-packages/gym/envs/registration.py", line 123, in spec
    raise error.Error('A module ({}) was specified for the environment but was not found, make sure the package is installed with `pip install` before calling `gym.make()`'.format(mod_name))
gym.error.Error: A module (A) was specified for the environment but was not found, make sure the package is installed with `pip install` before calling `gym.make()`

LGTM
Let's wait until I fix the CI with this
https://github.com/pytorch/rl/pull/2063

The tests are still failing because of missing envs, e.g. DKittyOrientFixed-v0
@vikashplus here is the complete list
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientFixed-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyOrientFixed-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientFixed-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyOrientFixed-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientRandom-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyOrientRandom-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientRandom-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyOrientRandom-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientRandom_v2d-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyOrientRandom_v2d-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyOrientRandom_v2d-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyOrientRandom_v2d-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandFixed-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyStandFixed-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandFixed-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyStandFixed-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandRandom-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyStandRandom-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandRandom-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyStandRandom-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandRandom_v2d-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyStandRandom_v2d-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyStandRandom_v2d-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyStandRandom_v2d-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkFixed-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyWalkFixed-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkFixed-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyWalkFixed-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkRandom-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyWalkRandom-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkRandom-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyWalkRandom-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkRandom_v2d-v0-True] - gym.error.UnregisteredEnv: No registered env with id: DKittyWalkRandom_v2d-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[DKittyWalkRandom_v2d-v0-False] - gym.error.UnregisteredEnv: No registered env with id: DKittyWalkRandom_v2d-v0
FAILED test/test_libs.py::TestRoboHive::test_robohive[FK1_CloseAllFixed-v4-True] - gym.error.UnregisteredEnv: No registered env with id: FK1_CloseAllFixed-v4
FAILED test/test_libs.py::TestRoboHive::test_robohive[FK1_CloseAllFixed-v4-False] - gym.error.UnregisteredEnv: No registered env with id: FK1_CloseAllFixed-v4

Found a fix for the issue! The camera names for the DKitty env is causing a problem.
['A:trackingZ', 'A:trackingY', 'A:headCam', 'A:tailCam', 'A:leftCam', 'A:rightCam']

The colon in the camera name is causing some issue, I'm not sure what that is. But adding cams = [i.replace("A:", "A_") for i in cams] fixes the tests. All RoboHive tests are now passing on my machine.

Should we merge this PR for now and remove the workaround after the camera names are updated in robohive in the next release? That seems like a reasonable approach.

I'd vote that we go ahead with the proposed workaround. I'll make sure that we land this change in the next RoboHive release.

I pulled the latest changes, upgraded robohive to 0.7, and replaced gym with gymnasium. The older failures are fixed, but a new set of tests are now failing.
================================================================================================================ short test summary info =================================================================================================================
FAILED test/test_libs.py::TestRoboHive::test_robohive[door-v1-False] - ValueError: too many values to unpack (expected 2)
FAILED test/test_libs.py::TestRoboHive::test_robohive[door-v1-True] - ValueError: too many values to unpack (expected 2)
FAILED test/test_libs.py::TestRoboHive::test_robohive[hammer-v1-False] - ValueError: too many values to unpack (expected 2)
FAILED test/test_libs.py::TestRoboHive::test_robohive[hammer-v1-True] - ValueError: too many values to unpack (expected 2)
FAILED test/test_libs.py::TestRoboHive::test_robohive[pen-v1-False] - ValueError: too many values to unpack (expected 2)
FAILED test/test_libs.py::TestRoboHive::test_robohive[pen-v1-True] - ValueError: too many values to unpack (expected 2)
FAILED test/test_libs.py::TestRoboHive::test_robohive[relocate-v1-False] - ValueError: too many values to unpack (expected 2)
FAILED test/test_libs.py::TestRoboHive::test_robohive[relocate-v1-True] - ValueError: too many values to unpack (expected 2)
=============================================================================================== 8 failed, 1034 passed, 11432 warnings in 921.68s (0:15:21) ===============================================================================================

Full traceback:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/libs/robohive.py", line 62, in __call__
    instance._refine_specs()
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/libs/robohive.py", line 303, in _refine_specs
    rollout = self.rollout(2, return_contiguous=False).get("next")
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/common.py", line 2528, in rollout
    tensordict = self.reset()
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/common.py", line 2105, in reset
    tensordict_reset = self._reset(tensordict, **kwargs)
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/libs/gym.py", line 1165, in _reset
    return super()._reset(tensordict, **kwargs)
  File "/home/sriramsk/desktop/torchrl/torchrl/envs/gym_like.py", line 357, in _reset
    obs, info = self._reset_output_transform(self._env.reset(**kwargs))
ValueError: too many values to unpack (expected 2)

Nope it seems that in this case robohive is returning only one (at least torchrl cannot see two things coming out of reset)
The envs are hammer, door, relocate and pen.
This commit solves it, and as you can see it does so by providing an empty dict if there is only one output

this line kinda does the same thing directly in the base class that uses @implement_for("gymnasium") to ensure compatibility

for the  hammer, door, relocate and pen envs (v1s), I'm overwriting the reset function. Does the @implement_for("gymnasium") functionality carry over?
Let's say it doesn't,  do you know why it's not affecting other envs (e.g. 1) where we use the same trick for gym compatibility?

Here's my understanding of what is happening: the reset you pointed in env_base does indeed return 2 values but as you were asking just above the new method doesn't and @implement_for doesn't catch that
The way we deal with this in RL is to have one _process_reset_data method or similar that we tune to the gym version and never touch. Example
class EnvBase:
    @implement_for("gymnasium")
    def _process_reset_data(self, reset_data):
        if not isinstance(reset_data, tuple) or not (len(reset_data) == 2 and isinstance(reset_data[1], dict)):
            return reset_data, {}
        return reset_data
    @implement_for("gym", None, 0.25) # or whatever other version
    def _process_reset_data(self, reset_data):
        if isinstance(reset_data, tuple) and len(reset_data) == 2 and isinstance(reset_data[1], dict):
            # throw the dict away
            return reset_data[0]
        return reset_data
	def reset(self):
        # your code here
        return self._process_reset_data(reset_data)
class OtherEnv(EnvBase):
	def reset(self):
        # your other code here
        return self._process_reset_data(reset_data)

Does that help?

Thanks @vmoens for digging in. This indeed confirms the theory.
I was also surprised by why it's not affecting other envs (e.g. 1. And there is a clever answer hidden there. Instead of returning the values, these other envs rely on the super().reset() for return for which implement_for  works perfectly.
In essence, using super().reset() does the same thing as you suggested. I feel it's a bit more elegant, I'll document it and also add a test for it. I'll change that 4 envs that didn't follow this convention and then RoboHive will hopefully take another step toward torchRL compatibility.

super() can defo do the job!
We don't use it in RL because that would make the entire Env API dependent on some (nasty) gym stuff + that would make the whole API more complex (people that don't use gym or different versions of gym should not care about that).
But in Robohive's case I guess super() is the most sensible option

Perfect. Thanks for pairing with me to better understand this issue.


POST Upload or Ingest new package

API Endpoints for update package.
(1) Upload, (2) update [with new version], (3) check rating of, and (4) download packages represented as zipped files.
o	The “rate” option should return the net score and sub-scores from Part 1, to inform prospective users of the quality of the module. It should also include two new metrics:
o	The fraction of dependencies that are pinned to at least a specific major+minor version, e.g., version 2.3.X of the dependency. (If a package has zero dependencies, it should receive a 1.0 rating. Now, suppose a package has two dependencies, one that is constrained to a particular major+minor version and another that is not. In this case, one of the two dependencies satisfies the requisite level of pinning, and so the fraction of dependencies is 1 in 2 or a score of ½ = 0.5.
o	The fraction of project code that was introduced through pull requests with a code review.
o	To reduce storage costs, the “upload” and “update” features should support a “debloat” option that removes unnecessary “bloat” from the package. You may wish to read about tree shaking, minification, etc.
o	Although ACME Corporation may upload its own internal modules to this registry, any such modules will be organized in the style of an npm package.

duplicate issue


Format '0,0[.]00 $' works only with Dollar symbol.

I used from the Format Examples on the Website this Format '0,0[.]00 $' and it works well except that I need '0,0[.]00 € the EUR Symbol but when I replace $ with € I get two zeros extra instead of €. Any idea why and how I can fix this?

look at documentations/locals


Delivery and Management API specific options

This is to allow to pass different configuration values of the same option to Delivery and Management clients.
It's needed if we want to say configure the Delivery API to fail quickly as we don't want to hog page rendering, and on the other hand let Management API take as long as needed in the same app, as those queries are not run by visitors and don't have to be fast.
config.options = {
    # Optional:
    # Use `delivery_api` and `management_api` keys to limit to what API the settings
    # will apply. Useful because Delivery API is usually visitor facing, while Management
    # is used in background tasks that can run much longer. For example:
    delivery_api: {
      timeout_read: 6
    },
    management_api: {
      timeout_read: 100
    }
}

Also in this PR webmock was updated to latest version, because the previous version was causing VCR fails when running test suite locally or on Travis. This bug: https://www.bountysource.com/issues/54892151-twitter-gem-vcr-request-initialized-with-an-invalid-body

The CI is finally green.


Issues with nested files #dev branch

I am trying to run a nested file directv/dev/popup gaston run in directv i get a strange error in

in the file server/wacther , added some error handeling but cant seem to find blessify.jsFiles

update
seems to build the files once , but also fails to add the gaston client side code

needed an update of blessify


MemorySizeOptions never is ignored on jumps

const int exampleCodeBitness = 64;
const ulong exampleCodeRIP = 0x7FF69E248C14;
static readonly byte[] exampleCode = { 0xE9,0x82,0xFE,0xFF,0xFF };
[...]
var formatter = new MasmFormatter();
formatter.Options.MemorySizeOptions = MemorySizeOptions.Never;
[...]

returns
jmp near ptr 00007FF69E248A9Bh
expected
jmp 00007FF69E248A9Bh
I think it should also be the case on MemorySizeOptions.Minimum because on jumps a human knows if its a near or far jump. Because of the bug I cannot test if this would be already the case.

MemorySizeOptions is for memory operands. There's an option to disable showing 'short' and 'near' if it's a branch, see ShowBranchSize


Cannot assign value of type 'AppDelegate' to type '(any FUIAuthDelegate)?'

xcode tip: Cannot assign value of type 'AppDelegate' to type '(any FUIAuthDelegate)?'
code
class AppDelegate: NSObject, UIApplicationDelegate {   func application(_ application: UIApplication,                    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey : Any]? = nil) -> Bool {     FirebaseApp.configure()     let authUI = FUIAuth.defaultAuthUI()     // You need to adopt a FUIAuthDelegate protocol to receive callback       authUI?.delegate = self     return true   } }

xcode version : 15.3
ios : 17.4
firebaseui-ios : 12.3.1
firebase : 10.23.1
could you help me ? thanks

i use spm

Add FUIAuthDelegate to the class declaration like:
class AppDelegate: NSObject, UIApplicationDelegate, FUIAuthDelegate


[MP] Fix division by zero crash

While this indeed fixes https://github.com/godotengine/godot/issues/96354, I still have no idea what was a root cause of this issue in a first place.
There is this part in SceneReplicationInterface::_send_sync:
const List<NodePath> props = sync->get_replication_config_ptr()->get_sync_properties(); // This always returns no props

Reason why in my project I was getting these divisions by zero were because this like always returns no props at all, and yet code underneath this keeps executing. It gets a resulting size of 0, submits it to the profiler and this crash would occur.
Even with this fix it feels like there is an overarching issue, since now object is still somehow synced, but the reported size is always 0 and never gets updated.

I managed to make a MRP, no need for this bandaid fix


Cutter Produces Unreadable Portable Files

Describe the bug
Cutter produces portable cut files that cannot be opened.  The maps.json file embedded into the .glm file is a 404 HTTP error message instead of valid JSON.
To Reproduce

Build and install OpenGEE.
Use Fusion to create a map (or globe).
Publish the map.
Enable Cutter
Open the OpenGEE admin page and launch Cutter.
Select the map, define a region to cut, and click "Cut Map"
Download the .glm file and try to load into Portable Server
See the glm file fail to load

Expected behavior
The map should load into Portable Server
Desktop

Server OS: Centos, but all versions affected
Client OS: Centos, but all versions affected
Browser: Firefox, Chrome, any
Version: 5.3.2

Tested on Centos 6 and Centos 7.  Fix works.
Approved.
Great job.

After this is included in the release notes, please close this ticket

Issue fixed now.

After this is included in the release notes, please close this ticket

Sorry @tst-ccamp just saw your comment.  This issue was introduced this release (actually <IP_ADDRESS> but we don't have release notes for point releases).  Do we still need to update the release notes for 5.3.2?  The issue did not exist in 5.3.1


Add dismissible CTA bar to current HRA Portal

Add dismissible CTA to current HRA Portal
Please prioritize implementation of this approved feature request design from Katy:

Figma: CTA Bar
Register Now button route: https://iu.co1.qualtrics.com/jfe/form/SV_bpaBhIr8XfdiNRH
Eventually, I will redesign this visually and make a version for the HRA Design System.
We will call out the most important feature/priority in this CTA bar. Right now, signing up for the HRA Working Group is the highest priority.

Behavior
This design was inspired by CELLxGENE's scrolling dismissible CTA bar. It scrolls with the page and users can dismiss it during their visit. This behavior is perfect:

CELLxGENE Landing Page: https://cellxgene.cziscience.com/

Responsive

@axdanbol & @bhushankhope - Do we have an update on this going live? I reviewed and approved here 3 weeks ago. Let me know if there's anything I can do to support!


AD Enhancements in Version 2.15

Description
Starting from version 2.15, we have introduced several enhancements:

Custom Index Management:

Added support for custom index management. For more details, watch the video on https://github.com/opensearch-project/anomaly-detection-dashboards-plugin/pull/770.
Custom result indices are now managed as aliases. Consequently, additional security permissions are required for management.

New JVM Heap Usage Threshold Setting:

Introduced a new setting, plugins.anomaly_detection.jvm_heap_usage_threshold, to manage the memory circuit breaker threshold.

Documentation Improvements:

Added examples for DSL filters to enhance the documentation.

Ruby Version Update:

Updated Ruby version in CONTRIBUTING.md from 3.2 to 3.2.4 as version 3.2 is not available.

Testing done:

built and viewed the changed website locally

Issues Resolved
Resolves https://github.com/opensearch-project/documentation-website/issues/7387
Version
2.15
Checklist

[X] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@vagimeli Left 3 comments and signed off the rest. Please help with those 3.

@vagimeli Left 3 comments and signed off the rest. Please help with those 3.

@kaituo I addressed those comments. I'm moving this to editorial. Thanks!

@natebower @vagimeli answered a few questions. Other than that, I am fine with the changes.

@natebower @vagimeli answered a few questions. Other than that, I am fine with the changes.

@natebower I accepted your changes and revised per comments. Will you approve for merging? Thank you :)