Make the timestamp system in Activity closer to that of the Discord SDK, and more user friendly

Checklist

[X] I have read the Contributor Guide
[X] I have read and agree to the Code of Conduct
[X] I have added a description of my changes and why I'd like them included in the section below

Description of Changes
After working with this crate in a recent project, the largest headache I encountered was the lack of ability to set just a start timestamp on activity (to get the (xx:xx elapsed) message in my status).
This PR adds two new methods to ActivityBuilder and slightly changes IntoTimestamp and Timestamps to add this ability.
I have added ActivityBuilder::start_timestamp and ActivityBuilder::end_timestamp which both set only their respective timestamp values when called. I have also deprecated ActivityBuilder::timestamps (although this can be undone), and changed its code to just call start_timestamp and end_timestamp. (Important note, calling both methods together will set a time range, like timestamps does)
Both the start_timestamp and end_timestamp now take a generic T: IntoTimestamp instead of an impl IntoTimestamp since the Rust compiler seems to dislike the old way, throwing errors when the method is called.
I have added an IntoTimestamp impl for i64, so crate users can easily enter a unix timestamp as an integer (more useful for testing than anything).
Finally, to make setting only one timestamp at a time possible, I have added serde annotations to Timestamps to skip serialization of empty fields.
I'll be keeping an eye on this PR for any requested changes, or comments.
Related Issues
None

I don't think there is any need to deprecate timestamps, there are situations where where having a single method can be more convenient compared to 2 separate builder methods. Everything else looks good though!

Alright. Both of those should be addressed now.


Fix cci-export usage in GHA

Description
cci-export will not work properly in GHA. This PR replaces is with proper way to propagate env variables between steps.
Testing Performed
CI
Refs
https://github.com/stackrox/stackrox/pull/2916#discussion_r1040868165

/retest


W-036717 - Update execute_install_apex Task

Update execute_install_apex Task to be consistent with install script.
Critical Changes
Changes
Issues Closed
New Metadata
Deleted Metadata
Testing Notes

Passes testing. Added notes on the WI.

@dgamboni  @xinnanlisf Not sure about the process here. I approved the PR and added testing notes to WI. Is anyone of you going to merge it? lmk. thanks.

Since this is only a config change that multiple people have reviewed and approved I'll go ahead and merge it. Thanks @skosgi1.


chore: add yapf config to pyproject.toml

This PR aims to drop the .style.yapf file in favour of pyproject.toml in order to have a more minimal project structure.
Reference: Following https://github.com/google/yapf/issues/708, pyproject.toml support was added.

Codecov Report

Merging #230 (b08c1a2) into main (058ce10) will decrease coverage by 7.37%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #230      +/-   ##
==========================================
- Coverage   89.21%   81.84%   -7.38%
==========================================
  Files          52       52
  Lines        5315     5315
  Branches      549      549
==========================================
- Hits         4742     4350     -392
- Misses        438      810     +372
- Partials      135      155      +20

Impacted Files
Coverage Δ

src/ott/solvers/linear/discrete_barycenter.py
20.73% <0.00%> (-65.86%)
:arrow_down:

src/ott/tools/segment_sinkhorn.py
43.75% <0.00%> (-56.25%)
:arrow_down:

src/ott/solvers/quadratic/gw_barycenter.py
30.47% <0.00%> (-53.34%)
:arrow_down:

src/ott/problems/quadratic/gw_barycenter.py
28.18% <0.00%> (-44.55%)
:arrow_down:

src/ott/geometry/segment.py
62.22% <0.00%> (-37.78%)
:arrow_down:

src/ott/tools/sinkhorn_divergence.py
63.15% <0.00%> (-35.09%)
:arrow_down:

src/ott/tools/soft_sort.py
69.90% <0.00%> (-25.25%)
:arrow_down:

src/ott/geometry/pointcloud.py
71.09% <0.00%> (-14.46%)
:arrow_down:

src/ott/geometry/graph.py
77.50% <0.00%> (-14.38%)
:arrow_down:

src/ott/problems/linear/potentials.py
79.34% <0.00%> (-9.79%)
:arrow_down:

... and 21 more


Widescreen responsiveness

Any reason why there's no support for screen (widths) greater than 960px in responsive grids? Nowadays wider screens with greater resolutions are a common place.
I ask because my app must support IE 10+ and flexbox is still too buggy on IE.
I know that I can customize Spectre (which I do), but it would be nice no have this by default since it would not be opnionated (just more breakpoints above 960px).
Thanks for the great work!! Really like the design.

@alexandremjacques Thanks, I am adding more layout widths.

I needed this as well and just raised https://github.com/picturepan2/spectre/pull/183 to add 1200 grid support :)


Prometheus Integration "istio-certs" volume not found

Issue Description:
When following the guide @ https://istio.io/latest/docs/ops/integrations/prometheus/, (The TLS Settings area is my reference.)
I have istio running under TLS STRICT mode and thus need certs for Prometheus to scrape with TLS.
See my deployment yaml below for the additions:

Added the annotations to the container spec
Added the volume mounts to both containers in the deployment
Added TLS context to the three scrape jobs that originally had a TLS context

Expected behavior:
Deployment should yield a successful Prometheus integration in an Istio environment that uses STRICT TLS.
Actual behavior:
Istio-certs volume cannot be found:
The Deployment "prometheus" is invalid:
* spec.template.spec.containers[0].volumeMounts[1].name: Not found: "istio-certs"

Version:
Istio v1.8.1
Prometheus Deployment Yaml:
---
# Source: prometheus/templates/server/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    component: "server"
    app: prometheus
    release: prometheus
    chart: prometheus-11.16.2
    heritage: Helm
  name: prometheus
  namespace: istio-system
  annotations:
    {}
---
# Source: prometheus/templates/server/cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    component: "server"
    app: prometheus
    release: prometheus
    chart: prometheus-11.16.2
    heritage: Helm
  name: prometheus
  namespace: istio-system
data:
  alerting_rules.yml: |
    {}
  alerts: |
    {}
  prometheus.yml: |
    global:
      evaluation_interval: 1m
      scrape_interval: 15s
      scrape_timeout: 10s
    rule_files:
    - /etc/config/recording_rules.yml
    - /etc/config/alerting_rules.yml
    - /etc/config/rules
    - /etc/config/alerts
    scrape_configs:
    - job_name: prometheus
      static_configs:
      - targets:
        - localhost:9090
    - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      job_name: kubernetes-apiservers
      kubernetes_sd_configs:
      - role: endpoints
      relabel_configs:
      - action: keep
        regex: default;kubernetes;https
        source_labels:
        - __meta_kubernetes_namespace
        - __meta_kubernetes_service_name
        - __meta_kubernetes_endpoint_port_name
      scheme: https
      tls_config:
        ca_file: /etc/prom-certs/root-cert.pem
        cert_file: /etc/prom-certs/cert-chain.pem
        key_file: /etc/prom-certs/key.pem
        insecure_skip_verify: true  # Prometheus does not support Istio security naming, thus skip verifying target pod ceritifcate
    - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      job_name: kubernetes-nodes
      kubernetes_sd_configs:
      - role: node
      relabel_configs:
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
      - replacement: kubernetes.default.svc:443
        target_label: __address__
      - regex: (.+)
        replacement: /api/v1/nodes/$1/proxy/metrics
        source_labels:
        - __meta_kubernetes_node_name
        target_label: __metrics_path__
      scheme: https
      tls_config:
        ca_file: /etc/prom-certs/root-cert.pem
        cert_file: /etc/prom-certs/cert-chain.pem
        key_file: /etc/prom-certs/key.pem
        insecure_skip_verify: true  # Prometheus does not support Istio security naming, thus skip verifying target pod ceritifcate
    - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      job_name: kubernetes-nodes-cadvisor
      kubernetes_sd_configs:
      - role: node
      relabel_configs:
      - action: labelmap
        regex: __meta_kubernetes_node_label_(.+)
      - replacement: kubernetes.default.svc:443
        target_label: __address__
      - regex: (.+)
        replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
        source_labels:
        - __meta_kubernetes_node_name
        target_label: __metrics_path__
      scheme: https
      tls_config:
        ca_file: /etc/prom-certs/root-cert.pem
        cert_file: /etc/prom-certs/cert-chain.pem
        key_file: /etc/prom-certs/key.pem
        insecure_skip_verify: true  # Prometheus does not support Istio security naming, thus skip verifying target pod ceritifcate
    - job_name: kubernetes-service-endpoints
      kubernetes_sd_configs:
      - role: endpoints
      relabel_configs:
      - action: keep
        regex: true
        source_labels:
        - __meta_kubernetes_service_annotation_prometheus_io_scrape
      - action: replace
        regex: (https?)
        source_labels:
        - __meta_kubernetes_service_annotation_prometheus_io_scheme
        target_label: __scheme__
      - action: replace
        regex: (.+)
        source_labels:
        - __meta_kubernetes_service_annotation_prometheus_io_path
        target_label: __metrics_path__
      - action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        source_labels:
        - __address__
        - __meta_kubernetes_service_annotation_prometheus_io_port
        target_label: __address__
      - action: labelmap
        regex: __meta_kubernetes_service_label_(.+)
      - action: replace
        source_labels:
        - __meta_kubernetes_namespace
        target_label: kubernetes_namespace
      - action: replace
        source_labels:
        - __meta_kubernetes_service_name
        target_label: kubernetes_name
      - action: replace
        source_labels:
        - __meta_kubernetes_pod_node_name
        target_label: kubernetes_node
    - job_name: kubernetes-service-endpoints-slow
      kubernetes_sd_configs:
      - role: endpoints
      relabel_configs:
      - action: keep
        regex: true
        source_labels:
        - __meta_kubernetes_service_annotation_prometheus_io_scrape_slow
      - action: replace
        regex: (https?)
        source_labels:
        - __meta_kubernetes_service_annotation_prometheus_io_scheme
        target_label: __scheme__
      - action: replace
        regex: (.+)
        source_labels:
        - __meta_kubernetes_service_annotation_prometheus_io_path
        target_label: __metrics_path__
      - action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        source_labels:
        - __address__
        - __meta_kubernetes_service_annotation_prometheus_io_port
        target_label: __address__
      - action: labelmap
        regex: __meta_kubernetes_service_label_(.+)
      - action: replace
        source_labels:
        - __meta_kubernetes_namespace
        target_label: kubernetes_namespace
      - action: replace
        source_labels:
        - __meta_kubernetes_service_name
        target_label: kubernetes_name
      - action: replace
        source_labels:
        - __meta_kubernetes_pod_node_name
        target_label: kubernetes_node
      scrape_interval: 5m
      scrape_timeout: 30s
    - honor_labels: true
      job_name: prometheus-pushgateway
      kubernetes_sd_configs:
      - role: service
      relabel_configs:
      - action: keep
        regex: pushgateway
        source_labels:
        - __meta_kubernetes_service_annotation_prometheus_io_probe
    - job_name: kubernetes-services
      kubernetes_sd_configs:
      - role: service
      metrics_path: /probe
      params:
        module:
        - http_2xx
      relabel_configs:
      - action: keep
        regex: true
        source_labels:
        - __meta_kubernetes_service_annotation_prometheus_io_probe
      - source_labels:
        - __address__
        target_label: __param_target
      - replacement: blackbox
        target_label: __address__
      - source_labels:
        - __param_target
        target_label: instance
      - action: labelmap
        regex: __meta_kubernetes_service_label_(.+)
      - source_labels:
        - __meta_kubernetes_namespace
        target_label: kubernetes_namespace
      - source_labels:
        - __meta_kubernetes_service_name
        target_label: kubernetes_name
    - job_name: kubernetes-pods
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - action: keep
        regex: true
        source_labels:
        - __meta_kubernetes_pod_annotation_prometheus_io_scrape
      - action: replace
        regex: (.+)
        source_labels:
        - __meta_kubernetes_pod_annotation_prometheus_io_path
        target_label: __metrics_path__
      - action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        source_labels:
        - __address__
        - __meta_kubernetes_pod_annotation_prometheus_io_port
        target_label: __address__
      - action: labelmap
        regex: __meta_kubernetes_pod_label_(.+)
      - action: replace
        source_labels:
        - __meta_kubernetes_namespace
        target_label: kubernetes_namespace
      - action: replace
        source_labels:
        - __meta_kubernetes_pod_name
        target_label: kubernetes_pod_name
      - action: drop
        regex: Pending|Succeeded|Failed
        source_labels:
        - __meta_kubernetes_pod_phase
    - job_name: kubernetes-pods-slow
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - action: keep
        regex: true
        source_labels:
        - __meta_kubernetes_pod_annotation_prometheus_io_scrape_slow
      - action: replace
        regex: (.+)
        source_labels:
        - __meta_kubernetes_pod_annotation_prometheus_io_path
        target_label: __metrics_path__
      - action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        source_labels:
        - __address__
        - __meta_kubernetes_pod_annotation_prometheus_io_port
        target_label: __address__
      - action: labelmap
        regex: __meta_kubernetes_pod_label_(.+)
      - action: replace
        source_labels:
        - __meta_kubernetes_namespace
        target_label: kubernetes_namespace
      - action: replace
        source_labels:
        - __meta_kubernetes_pod_name
        target_label: kubernetes_pod_name
      - action: drop
        regex: Pending|Succeeded|Failed
        source_labels:
        - __meta_kubernetes_pod_phase
      scrape_interval: 5m
      scrape_timeout: 30s
  recording_rules.yml: |
    {}
  rules: |
    {}
---
# Source: prometheus/templates/server/clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    component: "server"
    app: prometheus
    release: prometheus
    chart: prometheus-11.16.2
    heritage: Helm
  name: prometheus
rules:
  - apiGroups:
      - ""
    resources:
      - nodes
      - nodes/proxy
      - nodes/metrics
      - services
      - endpoints
      - pods
      - ingresses
      - configmaps
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
      - ingresses
    verbs:
      - get
      - list
      - watch
  - nonResourceURLs:
      - "/metrics"
    verbs:
      - get
---
# Source: prometheus/templates/server/clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    component: "server"
    app: prometheus
    release: prometheus
    chart: prometheus-11.16.2
    heritage: Helm
  name: prometheus
subjects:
  - kind: ServiceAccount
    name: prometheus
    namespace: istio-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: prometheus
---
# Source: prometheus/templates/server/service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    component: "server"
    app: prometheus
    release: prometheus
    chart: prometheus-11.16.2
    heritage: Helm
  name: prometheus
  namespace: istio-system
spec:
  ports:
    - name: http
      port: 9090
      protocol: TCP
      targetPort: 9090
  selector:
    component: "server"
    app: prometheus
    release: prometheus
  sessionAffinity: None
  type: "ClusterIP"
---
# Source: prometheus/templates/server/deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    component: "server"
    app: prometheus
    release: prometheus
    chart: prometheus-11.16.2
    heritage: Helm
  name: prometheus
  namespace: istio-system
spec:
  selector:
    matchLabels:
      component: "server"
      app: prometheus
      release: prometheus
  replicas: 1
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "true"
        traffic.sidecar.istio.io/includeInboundPorts: ""   # do not intercept any inbound ports
        traffic.sidecar.istio.io/includeOutboundIPRanges: ""  # do not intercept any outbound traffic
        proxy.istio.io/config: |  # configure an env variable `OUTPUT_CERTS` to write certificates to the given folder
           proxyMetadata:
              OUTPUT_CERTS: /etc/istio-output-certs
        sidecar.istio.io/userVolume: '[{"name": "istio-certs", "emptyDir": {"medium":"Memory"}}]'  # mount the shared volume
        sidecar.istio.io/userVolumeMount: '[{"name": "istio-certs", "mountPath": "/etc/istio-output-certs"}]'
      labels:
        component: "server"
        app: prometheus
        release: prometheus
        chart: prometheus-11.16.2
        heritage: Helm
    spec:
      serviceAccountName: prometheus
      containers:
        - name: prometheus-server-configmap-reload
          image: "jimmidyson/configmap-reload:v0.4.0"
          imagePullPolicy: "IfNotPresent"
          args:
            - --volume-dir=/etc/config
            - --webhook-url=http://<IP_ADDRESS>:9090/-/reload
          resources:
            {}
          volumeMounts:
            - name: config-volume
              mountPath: /etc/config
              readOnly: true
            - name: istio-certs
              mountPath: /etc/prom-certs/

        - name: prometheus-server
          image: "prom/prometheus:v2.21.0"
          imagePullPolicy: "IfNotPresent"
          args:
            - --storage.tsdb.retention.time=15d
            - --config.file=/etc/config/prometheus.yml
            - --storage.tsdb.path=/data
            - --web.console.libraries=/etc/prometheus/console_libraries
            - --web.console.templates=/etc/prometheus/consoles
            - --web.enable-lifecycle
          ports:
            - containerPort: 9090
          readinessProbe:
            httpGet:
              path: /-/ready
              port: 9090
            initialDelaySeconds: 0
            periodSeconds: 5
            timeoutSeconds: 30
            failureThreshold: 3
            successThreshold: 1
          livenessProbe:
            httpGet:
              path: /-/healthy
              port: 9090
            initialDelaySeconds: 30
            periodSeconds: 15
            timeoutSeconds: 30
            failureThreshold: 3
            successThreshold: 1
          resources:
            {}
          volumeMounts:
            - name: config-volume
              mountPath: /etc/config
            - name: storage-volume
              mountPath: /data
              subPath: ""
            - name: istio-certs
              mountPath: /etc/prom-certs/
      securityContext:
        fsGroup: 65534
        runAsGroup: 65534
        runAsNonRoot: true
        runAsUser: 65534
      terminationGracePeriodSeconds: 300
      volumes:
        - name: config-volume
          configMap:
            name: prometheus
        - name: storage-volume
          emptyDir:
            {}

@carolynhu and @lei-tang can you take a look at ensuring our prom guidance is up to date re: certs. This may involve updates to the https://istio.io/latest/blog/2020/proxy-cert/ blog post.

@carolynhu and @lei-tang can you take a look at ensuring our prom guidance is up to date re: certs. This may involve updates to the https://istio.io/latest/blog/2020/proxy-cert/ blog post.

The configuration (e.g., provisionPrometheusCert and the example deployment file) described in  https://istio.io/latest/blog/2020/proxy-cert/ are removed from 1.8.1 branch. https://istio.io/latest/blog/2020/proxy-cert/ has included a warning "This blog post was written assuming Istio 1.5, so some of this content may now be outdated."

The configuration (e.g., provisionPrometheusCert and the example deployment file) described in  https://istio.io/latest/blog/2020/proxy-cert/ are removed from 1.8.1 branch. https://istio.io/latest/blog/2020/proxy-cert/ has included a warning "This blog post was written assuming Istio 1.5, so some of this content may now be outdated."

@lei-tang right. This is why i'm suggesting new advice, in the form of blog post updates or similar, be drafted based on current recommendations.

@lei-tang right. This is why i'm suggesting new advice, in the form of blog post updates or similar, be drafted based on current recommendations.

I was able to get the deployment to deploy by adding the istio-certs volume to the volumes section at the bottom of the file. However, now the error states:
err="error creating HTTP client: unable to load specified CA cert /etc/prom-certs/root-cert.pem: open /etc/prom-certs/root-cert.pem: no such file or directory" scrape_pool=kubernetes-nodes-cadvisor
Any ideas why?

I was able to get the deployment to deploy by adding the istio-certs volume to the volumes section at the bottom of the file. However, now the error states:
err="error creating HTTP client: unable to load specified CA cert /etc/prom-certs/root-cert.pem: open /etc/prom-certs/root-cert.pem: no such file or directory" scrape_pool=kubernetes-nodes-cadvisor
Any ideas why?

@lei-tang right. This is why i'm suggesting new advice, in the form of blog post updates or similar, be drafted based on current recommendations.

Is there anywhere that the current recommendations are published?  Running into the same issue..

@lei-tang right. This is why i'm suggesting new advice, in the form of blog post updates or similar, be drafted based on current recommendations.

Is there anywhere that the current recommendations are published?  Running into the same issue..

I have the same problem, in both istio 1.8.1 and istio 1.8.3 and I too, followed the instruction here https://istio.io/latest/docs/ops/integrations/prometheus/#tls-settings
@darkdatter if I were to guess, you have your last error because normally the volume is mounted by the side car or injected by the AdmissionController into the yaml.  When you add yourself the volume to your yaml, I would guess it prohibits the standard behaviour of istio, since the volumes already exists.  This is only a guess.

@darkdatter @simonpie is the sidecar injected to your prometheus pod?

Hopefully https://github.com/istio/istio.io/pull/8962 could clarify the doc a bit.

@darkdatter @simonpie is the sidecar injected to your prometheus pod?

It certainly is and the istio-certs volume is the pod, mounted exactly where the annotations told it to mount in the side car container.

@simonpie Can you provide a dump of prometheus pod and proxy log? Thanks!

So I if follow https://github.com/istio/istio.io/commit/b8369ea605fd1d6889284a748cdeee1bca98dcd1 correctly, @darkdatter was on the right track.  We have to create the empty volume "by hand" and then through annotation, tell the side car to mount it too ?

So I if follow istio/istio.io@b8369ea correctly, @darkdatter was on the right track. We have to create the empty volume "by hand" and then through annotation, tell the side car to mount it too ?

Yes.

So I if follow istio/istio.io@b8369ea correctly, @darkdatter was on the right track. We have to create the empty volume "by hand" and then through annotation, tell the side car to mount it too ?

Yes. Sorry for the inconvenience caused by the confusing doc.

Any idea why it did not work for him then ?

@simonpie Can you provide a dump of prometheus pod spec (k get pod promethues -oyaml) and proxy log? Thanks!

Here you go, I removed those awesome managed fields.
You have to note that I tried to cheat and have prometheus mount without the volume specified.  When I add the volumeMounts in prometheus, then I have the error justifying this issue.  But as you can see, the annotations are there, and the sidecar and the volumes.
apiVersion: v1
kind: Pod
metadata:
  annotations:
    prometheus.io/path: /stats/prometheus
    prometheus.io/port: "15020"
    prometheus.io/scrape: "true"
    proxy.istio.io/config: |
      proxyMetadata:
        OUTPUT_CERTS: /etc/istio-output-certs
    sidecar.istio.io/inject: "true"
    sidecar.istio.io/status: '{"version":"e2cb9d4837cda9584fd272bfa1f348525bcaacfadb7e9b9efbd21a3bb44ad7a1","initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":["istio-envoy","istio-data","istio-podinfo","istiod-ca-cert","istio-certs"],"imagePullSecrets":null}'
    sidecar.istio.io/userVolume: '[{"name": "istio-certs", "emptyDir": {"medium":"Memory"}}]'
    sidecar.istio.io/userVolumeMount: '[{"name": "istio-certs", "mountPath": "/etc/istio-output-certs"}]'
    traffic.sidecar.istio.io/includeInboundPorts: ""
    traffic.sidecar.istio.io/includeOutboundIPRanges: ""
  creationTimestamp: "2021-02-11T19:59:07Z"
  generateName: prom2-prometheus-server-65f6458cf9-
  labels:
    app: prometheus
    chart: prometheus-11.16.9
    component: server
    heritage: Helm
    istio.io/rev: default
    pod-template-hash: 65f6458cf9
    release: prom2
    security.istio.io/tlsMode: istio
    service.istio.io/canonical-name: prometheus
    service.istio.io/canonical-revision: latest
      manager: kubelet
    operation: Update
    time: "2021-02-11T19:59:49Z"
  name: prom2-prometheus-server-65f6458cf9-ngg9k
  namespace: infra-prometheus
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: prom2-prometheus-server-65f6458cf9
    uid: 87235399-4471-4604-8fff-60c089dbfc9d
  resourceVersion: "322574111"
  selfLink: /api/v1/namespaces/infra-prometheus/pods/prom2-prometheus-server-65f6458cf9-ngg9k
  uid: f69b3e68-f388-4f9a-959c-6000bd5524d3
spec:
  containers:
  - args:
    - --volume-dir=/etc/config
    - --webhook-url=http://<IP_ADDRESS>:9090/-/reload
    image: jimmidyson/configmap-reload:v0.4.0
    imagePullPolicy: IfNotPresent
    name: prometheus-server-configmap-reload
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /etc/config
      name: config-volume
      readOnly: true
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: prom2-prometheus-server-token-shgqs
      readOnly: true
  - args:
    - --storage.tsdb.retention.time=30d
    - --config.file=/etc/config/prometheus.yml
    - --storage.tsdb.path=/data
    - --web.console.libraries=/etc/prometheus/console_libraries
    - --web.console.templates=/etc/prometheus/consoles
    - --web.enable-lifecycle
    - --no-storage.tsdb.wal-compression
    - --web.external-url=https://prometheus.lab.interne.montreal.ca
    - --storage.tsdb.retention.size=250GB
    image: prom/prometheus:v2.21.0
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 3
      httpGet:
        path: /app-health/prometheus-server/livez
        port: 15020
        scheme: HTTP
      initialDelaySeconds: 30
      periodSeconds: 15
      successThreshold: 1
      timeoutSeconds: 30
    name: prometheus-server
    ports:
    - containerPort: 9090
      protocol: TCP
    readinessProbe:
      failureThreshold: 3
      httpGet:
        path: /app-health/prometheus-server/readyz
        port: 15020
        scheme: HTTP
      initialDelaySeconds: 30
      periodSeconds: 5
      successThreshold: 1
      timeoutSeconds: 30
    resources:
      limits:
        cpu: "2"
        memory: 8Gi
      requests:
        cpu: "2"
        memory: 8Gi
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /etc/config
      name: config-volume
    - mountPath: /data
      name: storage-volume
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: prom2-prometheus-server-token-shgqs
      readOnly: true
  - args:
    - proxy
    - sidecar
    - --domain
    - $(POD_NAMESPACE).svc.cluster.local
    - --serviceCluster
    - prometheus.$(POD_NAMESPACE)
    - --proxyLogLevel=warning
    - --proxyComponentLogLevel=misc:error
    - --concurrency
    - "2"
    env:
    - name: JWT_POLICY
      value: first-party-jwt
    - name: PILOT_CERT_PROVIDER
      value: istiod
    - name: CA_ADDR
      value: istiod.istio-system.svc:15012
    - name: POD_NAME
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.name
    - name: POD_NAMESPACE
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.namespace
    - name: INSTANCE_IP
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: status.podIP
    - name: SERVICE_ACCOUNT
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: spec.serviceAccountName
    - name: HOST_IP
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: status.hostIP
    - name: CANONICAL_SERVICE
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.labels['service.istio.io/canonical-name']
    - name: CANONICAL_REVISION
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.labels['service.istio.io/canonical-revision']
    - name: PROXY_CONFIG
      value: |
        {"proxyMetadata":{"OUTPUT_CERTS":"/etc/istio-output-certs"}}
    - name: ISTIO_META_POD_PORTS
      value: |-
        [
            {"containerPort":9090,"protocol":"TCP"}
        ]
    - name: ISTIO_META_APP_CONTAINERS
      value: prometheus-server-configmap-reload,prometheus-server
    - name: ISTIO_META_CLUSTER_ID
      value: Kubernetes
    - name: ISTIO_META_INTERCEPTION_MODE
      value: REDIRECT
    - name: ISTIO_METAJSON_ANNOTATIONS
      value: |
        {"proxy.istio.io/config":"proxyMetadata:\n  OUTPUT_CERTS: /etc/istio-output-certs\n","sidecar.istio.io/inject":"true","sidecar.istio.io/userVolume":"[{\"name\": \"istio-certs\", \"emptyDir\": {\"medium\":\"Memory\"}}]","sidecar.istio.io/userVolumeMount":"[{\"name\": \"istio-certs\", \"mountPath\": \"/etc/istio-output-certs\"}]","traffic.sidecar.istio.io/includeInboundPorts":"","traffic.sidecar.istio.io/includeOutboundIPRanges":""}
    - name: ISTIO_META_WORKLOAD_NAME
      value: prom2-prometheus-server
    - name: ISTIO_META_OWNER
      value: kubernetes://apis/apps/v1/namespaces/infra-prometheus/deployments/prom2-prometheus-server
    - name: ISTIO_META_MESH_ID
      value: cluster.local
    - name: TRUST_DOMAIN
      value: cluster.local
    - name: OUTPUT_CERTS
      value: /etc/istio-output-certs
    - name: ISTIO_KUBE_APP_PROBERS
      value: '{"/app-health/prometheus-server/livez":{"httpGet":{"path":"/-/healthy","port":9090,"scheme":"HTTP"},"timeoutSeconds":30},"/app-health/prometheus-server/readyz":{"httpGet":{"path":"/-/ready","port":9090,"scheme":"HTTP"},"timeoutSeconds":30}}'
    image: docker.io/istio/proxyv2:1.8.3
    imagePullPolicy: Always
    name: istio-proxy
    ports:
    - containerPort: 15090
      name: http-envoy-prom
      protocol: TCP
    readinessProbe:
      failureThreshold: 30
      httpGet:
        path: /healthz/ready
        port: 15021
        scheme: HTTP
      initialDelaySeconds: 1
      periodSeconds: 2
      successThreshold: 1
      timeoutSeconds: 3
    resources:
      limits:
        cpu: "2"
        memory: 1Gi
      requests:
        cpu: 100m
        memory: 128Mi
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop:
        - ALL
      privileged: false
      readOnlyRootFilesystem: true
      runAsGroup: 1337
      runAsNonRoot: true
      runAsUser: 1337
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/istio
      name: istiod-ca-cert
    - mountPath: /var/lib/istio/data
      name: istio-data
    - mountPath: /etc/istio/proxy
      name: istio-envoy
    - mountPath: /etc/istio/pod
      name: istio-podinfo
    - mountPath: /etc/istio-output-certs
      name: istio-certs
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: prom2-prometheus-server-token-shgqs
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  initContainers:
  - args:
    - istio-iptables
    - -p
    - "15001"
    - -z
    - "15006"
    - -u
    - "1337"
    - -m
    - REDIRECT
    - -i
    - ""
    - -x
    - ""
    - -b
    - ""
    - -d
    - 15090,15021,15020
    env:
    - name: OUTPUT_CERTS
      value: /etc/istio-output-certs
    image: docker.io/istio/proxyv2:1.8.3
    imagePullPolicy: Always
    name: istio-init
    resources:
      limits:
        cpu: "2"
        memory: 1Gi
      requests:
        cpu: 100m
        memory: 128Mi
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        add:
        - NET_ADMIN
        - NET_RAW
        drop:
        - ALL
      privileged: false
      readOnlyRootFilesystem: false
      runAsGroup: 0
      runAsNonRoot: false
      runAsUser: 0
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: prom2-prometheus-server-token-shgqs
      readOnly: true
  nodeName: lablkbw12b
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext:
    fsGroup: 65534
    runAsGroup: 65534
    runAsNonRoot: true
    runAsUser: 65534
  serviceAccount: prom2-prometheus-server
  serviceAccountName: prom2-prometheus-server
  terminationGracePeriodSeconds: 0
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 90
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 90
  volumes:
  - configMap:
      defaultMode: 420
      name: prom2-prometheus-server
    name: config-volume
  - name: storage-volume
    persistentVolumeClaim:
      claimName: prom2-prometheus-server
  - name: prom2-prometheus-server-token-shgqs
    secret:
      defaultMode: 420
      secretName: prom2-prometheus-server-token-shgqs
  - emptyDir:
      medium: Memory
    name: istio-envoy
  - emptyDir: {}
    name: istio-data
  - downwardAPI:
      defaultMode: 420
      items:
      - fieldRef:
          apiVersion: v1
          fieldPath: metadata.labels
        path: labels
      - fieldRef:
          apiVersion: v1
          fieldPath: metadata.annotations
        path: annotations
    name: istio-podinfo
  - configMap:
      defaultMode: 420
      name: istio-ca-root-cert
    name: istiod-ca-cert
  - emptyDir:
      medium: Memory
    name: istio-certs
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2021-02-11T19:59:09Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2021-02-11T19:59:49Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2021-02-11T19:59:49Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2021-02-11T19:59:07Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: cri-o://fce91c9102de5f158898f6afe79a3c69be530991487b89b66e296cb7e84d5640
    image: docker.io/istio/proxyv2:1.8.3
    imageID: docker.io/istio/proxyv2@sha256:5cfde7ffd5b921cf805f4cf18013d3f1b825f41fe1bd1d977d805c45ca955d5a
    lastState: {}
    name: istio-proxy
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2021-02-11T19:59:10Z"
  - containerID: cri-o://587874fef65073f4339d473da5bdcee46a8ee918473775131df2e751af52e3b4
    image: docker.io/prom/prometheus:v2.21.0
    imageID: docker.io/prom/prometheus@sha256:d43417c260e516508eed1f1d59c10c49d96bbea93eafb4955b0df3aea5908971
    lastState: {}
    name: prometheus-server
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2021-02-11T19:59:10Z"
  - containerID: cri-o://903bd3c74eb1700f35a19edeba773831ecbd194085c6ac38f69eca2e8e7985f4
    image: docker.io/jimmidyson/configmap-reload:v0.4.0
    imageID: docker.io/jimmidyson/configmap-reload@sha256:17d34fd73f9e8a78ba7da269d96822ce8972391c2838e08d92a990136adb8e4a
    lastState: {}
    name: prometheus-server-configmap-reload
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2021-02-11T19:59:09Z"
  hostIP: <IP_ADDRESS>
  initContainerStatuses:
  - containerID: cri-o://73fa58b1bf26bf2a4638ba5d3320e1e6c72c04087ef79c82537643dc033ae9a5
    image: docker.io/istio/proxyv2:1.8.3
    imageID: docker.io/istio/proxyv2@sha256:5cfde7ffd5b921cf805f4cf18013d3f1b825f41fe1bd1d977d805c45ca955d5a
    lastState: {}
    name: istio-init
    ready: true
    restartCount: 0
    state:
      terminated:
        containerID: cri-o://73fa58b1bf26bf2a4638ba5d3320e1e6c72c04087ef79c82537643dc033ae9a5
        exitCode: 0
        finishedAt: "2021-02-11T19:59:09Z"
        reason: Completed
        startedAt: "2021-02-11T19:59:09Z"
  phase: Running
  podIP: <IP_ADDRESS>
  podIPs:
  - ip: <IP_ADDRESS>
  qosClass: Burstable
  startTime: "2021-02-11T19:59:07Z"

The volume is not mounted at prometheus-server container:
    name: prometheus-server
    ports:
    - containerPort: 9090
      protocol: TCP
    readinessProbe:
      failureThreshold: 3
      httpGet:
        path: /app-health/prometheus-server/readyz
        port: 15020
        scheme: HTTP
      initialDelaySeconds: 30
      periodSeconds: 5
      successThreshold: 1
      timeoutSeconds: 30
    resources:
      limits:
        cpu: "2"
        memory: 8Gi
      requests:
        cpu: "2"
        memory: 8Gi
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:  <------ Miss istio certs volume here.
    - mountPath: /etc/config
      name: config-volume
    - mountPath: /data
      name: storage-volume
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: prom2-prometheus-server-token-shgqs
      readOnly: true

No, this is what I said.  I cannot add the instructions to the prom-server container
    - name: istio-certs
      mountPath: /etc/prom-certs/

since the kube apply will not work and return the error :
"spec.template.spec.containers[1].volumeMounts[2].name: Not found: "istio-certs" "
But if I remove the mount, I dont get the error, the pods works more or less, but or course certs are not available to the prometheus server container.

@simonpie that is what the linked doc pr trying to fix. Can you try this new guide with that pending PR: https://deploy-preview-8962--preliminary-istio.netlify.app/latest/docs/ops/integrations/prometheus/#tls-settings

@simonpie that is what the linked doc pr trying to fix. Can you try this new guide with that pending PR: https://deploy-preview-8962--preliminary-istio.netlify.app/latest/docs/ops/integrations/prometheus/#tls-settings

Hello,
Almost work I had to change the istio-certs, otherwise I get deployment message in the replicaset :
 kc get events -n infra-prometheus
LAST SEEN   TYPE      REASON              OBJECT                                          MESSAGE
53m         Warning   FailedCreate        replicaset/prom2-prometheus-server-55487458d7   (combined from similar events): Error creating: Pod "prom2-prometheus-server-55487458d7-pb6b8" is invalid: [spec.containers[1].volumeMounts[2].name: Not found: "istio-certs", spec.containers[2].volumeMounts[4].name: Not found: "istio-certs"]
43m         Warning   FailedCreate        replicaset/prom2-prometheus-server-55487458d7   Error creating: Pod "prom2-prometheus-server-55487458d7-6scvd" is invalid: [spec.containers[1].volumeMounts[2].name: Not found: "istio-certs", spec.containers[2].volumeMounts[4].name: Not found: "istio-certs"]
26m         Warning   FailedCreate        replicaset/prom2-prometheus-server-55487458d7   Error creating: Pod "prom2-prometheus-server-55487458d7-9jzbm" is invalid: [spec.containers[1].volumeMounts[2].name: Not found: "istio-certs", spec.containers[2].volumeMounts[4].name: Not found: "istio-certs"]
9m43s       Warning   FailedCreate        replicaset/prom2-prometheus-server-55487458d7   Error creating: Pod "prom2-prometheus-server-55487458d7-8zm4n" is invalid: [spec.containers[1].volumeMounts[2].name: Not found: "istio-certs", spec.containers[2].volumeMounts[4].name: Not found: "istio-certs"]

But if I change istio-certs to istio-certs-dir, it works.  I honestly cannot find where is the conflict and I have not found any logs anywhere to help me.  A git diff will confirms that these are the only changes between what works and what not.
Since we deploy using helm, here is the final code that worked :
  podAnnotations:
    sidecar.istio.io/inject: "true"
    sidecar.istio.io/rewriteAppHTTPProbers: "true"
    traffic.sidecar.istio.io/includeInboundPorts: ""   # do not intercept any inbound ports
    traffic.sidecar.istio.io/includeOutboundIPRanges: ""  # do not intercept any outbound traffic
    proxy.istio.io/config: |  # configure an env variable `OUTPUT_CERTS` to write certificates to the given folder
      proxyMetadata:
        OUTPUT_CERTS: /etc/istio-output-certs
    sidecar.istio.io/userVolumeMount: '[{"name": "istio-certs-dir", "mountPath": "/etc/istio-output-certs"}]' # mount the shared volume at sidecar proxy

  extraVolumeMounts:
    - name: istio-certs-dir
      mountPath: /etc/prom-certs/

  extraVolumes:
    - name: istio-certs-dir
      emptyDir:
        medium: Memory

Also, the instruction are also wrong in the automatic injection aspect.  One has to label the namespace for this to work.  The side car will not be injected if only the annotations and policies are set to true: see this https://github.com/istio/istio/issues/6476
Thank you @bianpengyuan, hope this helps

[spec.containers[1].volumeMounts[2].name: Not found: "istio-certs", spec.containers[2].volumeMounts[4].name: Not found: "istio-certs"]

This does ot make sense to me if you are sure that both volume and volume mounts are using name istio-certs. It will be a k8s bug..

Also, the instruction are also wrong in the automatic injection aspect. One has to label the namespace for this to work. The side car will not be injected if only the annotations and policies are set to true: see this #6476

This was addressed in the pending PR as well. The injection label is removed, and sidecar injection doc  is referenced instead.

This does ot make sense to me if you are sure that both volume and volume mounts are using name istio-certs. It will be a k8s bug..

I agree, I found that by doing kubectl edit, but I would not be ready just yet to file a bug with the kubernetes project.

This was addressed in the pending PR as well. The injection annotation is removed, and sidecar injection doc is referenced instead.

Yes, its probably not needed, but I added it back in order to try to find a solution.  I must point out that, unless I'm mistaken,
https://deploy-preview-8962--preliminary-istio.netlify.app/latest/docs/setup/additional-setup/sidecar-injection/#controlling-the-injection-policy
which is for istio1.10, which is referenced in the PR you mentioned, still lets me believe that pod annotation would be able to trigger sidecar injection.

in istio 1.9 exported certificate has no read access for group and others
-rw-------    1 1337     1337          2302 Mar  5 17:00 cert-chain.pem
-rw-------    1 1337     1337          1679 Mar  5 17:00 key.pem
-rw-------    1 1337     1337          1094 Mar  5 17:00 root-cert.pem
This will result the certificate cannot be ready even though it is mounted with fsGroup 1337 (as result of injection)

@c0ffeec0der what do you mean by certificate cannot be ready?

@bianpengyuan sorry typo.. i mean the certificates cannot be read by prometheus container because the group permission is not defined. Shouldn't the output file have permission rwxr-xr-x instead of rw-------?

@c0ffeec0der Thanks for finding this! Indeed there is behavior change at Istio 1.9, which I think actually makes perfect sense: https://github.com/istio/istio/pull/28929. Would it be possible to run your prometheus as user 1337?

cc @douglas-reid @lei-tang @liminw on this behavior change ^^ We'd need to update the doc about limitations..

@c0ffeec0der Thanks for finding this! Indeed there is behavior change at Istio 1.9, which I think actually makes perfect sense: https://github.com/istio/istio/pull/28929. Would it be possible to run your prometheus as user 1337?

@bianpengyuan yes i can, but i am not aware of the side effects running as 1337 for prometheus. I read https://github.com/istio/istio/pull/28929 but it doesn't explain the reason of permission change. It seems odd that if istio export the certs but does not allow the group and others to read it. Can we chmod it to 644 rw-r--r-- instead?

I read #28929 but it doesn't explain the reason of permission change. It seems odd that if istio export the certs but does not allow the group and others to read it. Can we chmod it to 644 rw-r--r-- instead?

I think it is generally right to restrict the permission of cert files. I am not sure what is the other use case of OUTPUT_CERTS and how the certs are used in those cases. IIRC it is used in VM integration. @liminw @lei-tang @stevenctl any idea about this? and do you have any recommendation re permission of the cert file?

We discussed about this at today's security WG meeting. The agreement is to change the default permission of output cert to 644 when the proxy runs on k8s, and leave it 600 when running on other env such as VM.

We discussed about this at today's security WG meeting. The agreement is to change the default permission of output cert to 644 when the proxy runs on k8s, and leave it 600 when running on other env such as VM.

@bianpengyuan fyi, Just did simulation with 644 using chmod from busybox, works ok


test_audit_plugin.TestAuditPlugin.test_audit_plugin test faliure

iRODS version: 4.3.2 release candidate (irods/irods@5ba6de8627802ef994f9b6fa600ef1103bfc4ea7)
Plugin version: 4.3.2 release candidate (151e1737432307b028dba0354fb07b88bac281c3)
Externals: irods/externals@1b220bcd013678008c34a9268c15becfed35acb6
Running in the testing environment with the ubuntu-22.04-mysql-8 project
The test test_audit_plugin.TestAuditPlugin.test_audit_plugin fails consistently.
logs: https://gist.github.com/SwooshyCueb/a953843637583a77709ab5ae27c04d20

Just sharing my experience, which undoubtedly overlaps, saw these results:
======================================================================
ERROR [2.310s]: test_audit_plugin (irods.test.test_audit_plugin.TestAuditPlugin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/irods/scripts/irods/test/test_audit_plugin.py", line 54, in test_audit_plugin
    listener.run()
  File "/var/lib/irods/scripts/irods/test/queue_listener.py", line 42, in run
    container.run()
  File "/usr/local/lib64/python3.6/site-packages/proton/_reactor.py", line 194, in run
    while self.process():
  File "/usr/local/lib64/python3.6/site-packages/proton/_reactor.py", line 259, in process
    event.dispatch(self._global_handler)
  File "/usr/local/lib64/python3.6/site-packages/proton/_events.py", line 161, in dispatch
    _dispatch(handler, type.method, self)
  File "/usr/local/lib64/python3.6/site-packages/proton/_events.py", line 130, in _dispatch
    handler.on_unhandled(method, *args)
  File "/usr/local/lib64/python3.6/site-packages/proton/_reactor.py", line 903, in on_unhandled
    event.dispatch(self.base)
  File "/usr/local/lib64/python3.6/site-packages/proton/_events.py", line 161, in dispatch
    _dispatch(handler, type.method, self)
  File "/usr/local/lib64/python3.6/site-packages/proton/_events.py", line 128, in _dispatch
    m(*args)
  File "/usr/local/lib64/python3.6/site-packages/proton/_handlers.py", line 1159, in on_reactor_quiesced
    s.writable()
  File "/usr/local/lib64/python3.6/site-packages/proton/_handlers.py", line 1367, in writable
    sock = IO.connect(self._addrs[0])
  File "/usr/local/lib64/python3.6/site-packages/proton/_io.py", line 66, in connect
    s.connect(addr[4])
OSError: [Errno 99] Cannot assign requested address

...and...
======================================================================
FAIL [0.256s]: test_delayed_rule_with_plugin_configured (irods.test.test_audit_plugin.TestAuditPlugin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/irods/scripts/irods/test/test_audit_plugin.py", line 81, in test_delayed_rule_with_plugin_configured
    admin_session.assert_icommand(['irule', '-r', rep_name, '-F', rule_file], 'STDERR', 'SYS_NOT_SUPPORTED')
  File "/var/lib/irods/scripts/irods/test/session.py", line 164, in assert_icommand
    return assert_command(*args, **kwargs)
  File "/var/lib/irods/scripts/irods/test/command.py", line 77, in assert_command
    return _assert_helper(*args, should_fail=False, **kwargs)
  File "/var/lib/irods/scripts/irods/test/command.py", line 105, in _assert_helper
    assert result
AssertionError

Stdout:
 --- IrodsSession: icommand executed by [rods#tempZone] [iinit] ---
Assert Command: iinit
Expecting STDOUT_SINGLELINE: ['']
  stdout:
    | WARNING: Error 25 disabling echo mode. Password will be displayed in plaintext.
    | Enter your current iRODS password:
  stderr:
    |
Output found

 --- IrodsSession: icommand executed by [rods#tempZone] [irule -r irods_rule_engine_plugin-audit_amqp-instance -F test_audit_plugin_delayed_rule.r] ---
Assert Command: irule -r irods_rule_engine_plugin-audit_amqp-instance -F test_audit_plugin_delayed_rule.r
Expecting STDERR: ['SYS_NOT_SUPPORTED']
  stdout:
    |
  stderr:
    | remote addresses: <IP_ADDRESS> ERROR: rcExecMyRule error.  status = -130000 SYS_INVALID_INPUT_PARAM
Output not found

FAILED TESTING ASSERTION

The SYS_INVALID_INPUT_PARAM failure appears to occur because the REP is not configured at the time the test is run, at least according to the server logs. It is expecting SYS_NOT_SUPPORTED because the REP does not support the rule that the test is attempting to run. However, irule fails to find the targeted REP instance and fails with SYS_INVALID_INPUT_PARAM. The REP does get configured at some point in the tests (by inspection of server_config.json), but it seems it was not at the time this test was run. I will try to run this one in isolation.
The other issue is the one that will be a bit harder to figure out, I think. The qpidproton listener fails to run() and doesn't really give any good errors other than the one shown above (that I've seen). Interestingly, the logs provided by @SwooshyCueb clearly show that messages are appearing somewhere, so this test fixture just needs to be stood up properly to get over that hump.
Not that it particularly matters, but I also tried using rabbitmq instead and found that the same issues exist.

Just confirmed that test_audit_plugin.TestAuditPlugin.test_delayed_rule_with_plugin_configured passes when run in isolation.
Also noting that I'm running these tests with AlmaLinux 8 / MySQL 8.

So with a confident diagnosis of the tests themselves not yet being held correctly by the framework - and otherwise, no code changes... this doesn't feel like a blocker for release.

I'm not so sure about that. The test that is failing is the only test that actually exercises the plugin and confirms that it is working. The other tests are either dealing with configuration or literally just running the unixfilesystem suite with the REP configured to make sure it doesn't break anything by existing in normal usage.

I see.

I installed OpenJDK 1.8 and was able to get the same error as shown in OP, so we're getting closer...


Add core & sql managed rules to WAF

Who is this for?
This change is intended to block a wider array of malicious requests to wellcomecollection.org in order to reduce unnecessary alarms and increase protection to the underlaying services.
What is it doing for them?
We add AWSManagedRulesSQLiRuleSet and AWSManagedRulesCommonRuleSet, to block requests that match recognised OWASP vulnerabilities and include potential (useless) SQL injection attacks that we've seen in alerts.
How do we test this?

[ ] Deploy this change to the e2e stack, run the e2e tests to catch any obvious problems
[ ] Deploy this change with the action set to count
In order that we can deploy to production and identify potential impact with real traffic before switching to block

[!NOTE]
The AWSManagedRulesCommonRuleSet has the potential to match valid requests as it does things like match large request bodies and long urls. Unlikely to be an issue but it's something we can test for so we should probably do that.

Part of https://github.com/wellcomecollection/wellcomecollection.org/issues/10511


How to get a specific parameter inside an array?

Hi,
for my case I'm using {{movie.moviebackground | random}}. Now I want to be able to get the url parameter inside that random array/object element. How would I be able to do it with vue-filter?

@visualcookie vue-filter don't have a filter help you get a parameter inside an object right now.  I think I should add a filter named get, it can help you get a parameter inside an object.
but, at this time, you can use I map filter before random, the output of map filter is an array which contains the parameter you want, and random filter can pick one.
{{movie.moviebackground | map getAproperty | random}}
this means you have to write a help function.


problem about multi gpu

Hi, thank you for your release!
The adjacency matrix computing needs all embeddings together. However the code is using dataparallel on multi gpus. In this situation, the adjacency matix computing can only uses part of the embeddings in a single gpu. Using single gpu and multi gpus has different results.  I'm wondering the solution for such a problem.

Hi @invokerpicker, thanks for the interest in our work! We perform embedding propagation after extracting embeddings with DataParallel such that all of them are together in the same GPU.

Extract embeddings: https://github.com/ElementAI/embedding-propagation/blob/master/src/models/finetuning.py#L140
Embedding prop: https://github.com/ElementAI/embedding-propagation/blob/master/src/models/finetuning.py#L145

I hope this solves your question.

Thanks!


PEP 479 warning in test runs

            except IndexError:
                # The total count on Server changed while fetching exit gracefully
                raise StopIteration

should be
            except IndexError:
                # The total count on Server changed while fetching exit gracefully
                return

After pep 479. I've run the tests on 2.7 and 3.6 and they are fine with the change.

Done!


avoiding duplicate kubernetes resource dependencies

right now we use regular <dependency> relationships so that we can compose things together; e.g. cd-pipeline depends on gogs, jenkins etc.
However due to transitive dependencies, when the DependencyEnricher kicks in, it sees cd-pipeline (which contains gogs) and also gog so it ends up duplicating all the resources twice.
I wonder if we can help trim dependency trees; so when we create cd-pipeline jar its not really transitively dependent on gogs & jenkins etc.
Not 100% sure the ideal approach. We could use provided or test scope for these kinds of dependencies? Not totally sure which; either should hopefully limit the scope and reduce this issue.

Wonder if we could just build a set of dependencies to remove dupes before we write out the aggregated manifests?

when we make an uberjar and you consume it, you don't also transitively also depend on all the parts that made the uberjar (as you'd get duplicates) - we make a new jar with no dependencies.
I was hoping we could do the same kinda thing with how we do composition of kubernetes manifests - to avoid unnecessary duplicates. There's also the issue that by depending on, say, cd-pipeline in an Arquillian test you bring in all the maven dependency trees of all the stuff used to build those apps. It'd be nice to stop the transitive dependency trees for these packages - from a dependency management perspective, they are just stand alone jars with manifests inside.
I originally started hacking a merge goal; its just slightly scary just throwing stuff away when really its just dodgy mvn dependency trees making things look like there's duplicates. I was hoping we could define a 'build time dependency' to build a jar; then those dependencies disappear when its then used to build a different jar. e.g. if we have 2 Deployments called 'foo' - which one wins?
Maybe the cleanest option is to just put the dependencies in the maven plugin; so that there's really no maven dependencies for cd-pipeline for example?

I'm thinking of adding the <dependency> elements inside the fabric8-maven-plugin then its much cleaner and avoids unnecessary transitive dependencies.
I'll add more warnings if we find duplicates as well.


Default sort order for tools should be "recently updated"

In reviewing the tools section, @nathenharvey noticed that the tools list is sorted chronologically by creation time. The default sort should be reverse chronological by update time, at least when following the "View All Tools" link at the bottom of "Recently Updated" sidebar.

@robbkidd wrong repo


How to use Rest Endpoint to get useful information

Hi,
Thanks for your reply to my last issue.  I show you my current code according to your code sample:
private async void BetfairDogRaces1Day()
{
var credentials = new Credentials(UserName, Password, APIKey);
using var client = new BetfairApiClient(credentials);
var filter = new ApiMarketFilter()
.WithMarketTypes(MarketType.Win)
.WithCountries(Country.UnitedKingdom, Country.Ireland)
.WithEventTypes(EventType.GreyhoundRacing)
.FromMarketStart(DateTimeOffset.UtcNow)
.ToMarketStart(DateTimeOffset.UtcNow.AddDays(1));
        var query = new MarketCatalogueQuery()
            .Include(MarketProjection.Event)
            .Include(MarketProjection.MarketStartTime)
            .Include(MarketProjection.MarketDescription)
            .Include(MarketProjection.RunnerDescription);

        var marketCatalogues = await client.MarketCatalogue(filter, query);
        var top200Races = new MarketCatalogueQuery()
            .Include(MarketProjection.Event)
            .Include(MarketProjection.MarketStartTime)
            .Include(MarketProjection.MarketDescription)
            .Include(MarketProjection.RunnerDescription)
            .OrderBy(MarketSort.FirstToStart)
            .Take(200);
        Debug.Print(top200Races.ToString());
    }

However, from debug window, I can't see any useful data.
But I show you what I want to get from the Betfair API endpoint, here is my current code using HTTPS post request:
public class DataStructure
{
public class BetfairResponse
{
public string Jsonrpc { get; set; }
public List Result { get; set; }
public int Id { get; set; }
}
public class MarketCatalogue
{
    public string MarketId { get; set; }
    public string MarketName { get; set; }
    public DateTime MarketStartTime { get; set; }
    public Event Event { get; set; }
    public List<Runner> Runners { get; set; }
}

public class Event
{
    public string Id { get; set; }
    public string Name { get; set; }
    public string CountryCode { get; set; }
    public string Timezone { get; set; }
    public string Venue { get; set; }
    public DateTime OpenDate { get; set; }
}

public class Runner
{
    public long SelectionId { get; set; }
    public string RunnerName { get; set; }
    public double Handicap { get; set; }
    public int SortPriority { get; set; }
}

public class RaceDetails
{
    public string MarketId { get; set; } = string.Empty;
    public DateTime StartTime { get; set; }
    public string CountryCode { get; set; } = string.Empty;
    public string RaceCourse { get; set; } = string.Empty;
}

public class GreyhoundTracks
{
    public int ID { get; set; }
    public string TrackCode { get; set; }
    public string TrackName { get; set; }
}

public class RunnerInfo
{
    public int DogID { get; set; }
    public long SelectionID { get; set; }
    public string DogName { get; set; }
}

public class RaceFixture
{
    public string Bookie { get; set; }
    public int Category { get; set; }
    public string CountryCode { get; set; }
    public string EventID { get; set; }
    public string MarketID { get; set; }
    public string MarketName { get; set; }
    public int RaceID { get; set; }
    public int RaceMeters { get; set; }
    public string TrackCode { get; set; }
    public string TrackName { get; set; }
    public DateTime StartsGMT { get; set; }
}

public class RunnerIDSelection
{
    public string Bookie { get; set; }
    public string Category { get; set; }
    public string JsonRunners { get; set; }
    public string MarketID { get; set; }
    public int RaceID { get; set; }
    public DateTime RaceDate { get; set; }
    public string TrackCode { get; set; }
    public string TrackName { get; set; }
}

    public static void BulkSaveRaceFixtures(List<RaceFixture> fixtures)
    {
        try
        {
            using SLMDB context = new();
            context.Race_Fixture.AddRange(fixtures);
            context.SaveChanges();
        }
        catch (DbUpdateException ex)
        {
            Debug.Print("[BulkSaveRaceFixtures] EX: " + ex.Message);
        }
    }

    public static void BulkSaveRunnerIDSelections(List<RunnerIDSelection> selections)
    {
        try
        {
            using SLMDB context = new();
            context.Race_Runners.AddRange(selections);
            context.SaveChanges();
        }
        catch (DbUpdateException ex)
        {
            Debug.Print("[BulkSaveRunnerIDSelections] EX: " + ex.Message);
        }
    }

    const string BetAPIUrl = @"https://api.betfair.com/exchange/betting/json-rpc/v1";
    private BetfairClient BFClient;
    public async Task GetGreyhoundRacesAsync(string sessionToken)
    {
        DateTime now = DateTime.Now;
        DateTime tomorrow = now.AddHours(24);

        var marketFilter = new
        {
            jsonrpc = "2.0",
            method = "SportsAPING/v1.0/listMarketCatalogue",
            @params = new
            {
                filter = new
                {
                    eventTypeIds = new List<string> { "4339" },
                    marketCountries = new List<string> { "GB","IE" },
                    marketTypeCodes = new List<string> { "WIN" },
                    marketStartTime = new
                    {
                        from = DateTime.Today.ToString("yyyy-MM-ddTHH:mm:ssZ"),
                        to = tomorrow.ToString("yyyy-MM-ddTHH:mm:ssZ")
                    }
                },
                maxResults = "300",
                marketProjection = new List<string> { "MARKET_START_TIME", "RUNNER_DESCRIPTION", "EVENT" }
            },
            id = 1
        };
        string jsonRequest = JsonConvert.SerializeObject(marketFilter);
        Debug.Print(jsonRequest);
        StringContent requestContent = new(jsonRequest);
        requestContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
        client.DefaultRequestHeaders.Add("X-Authentication", sessionToken);
        client.DefaultRequestHeaders.Add("X-Application", APIKey);
        HttpResponseMessage response = await client.PostAsync(BetAPIUrl, requestContent);
        response.EnsureSuccessStatusCode();
        string responseContent = await response.Content.ReadAsStringAsync();
        Debug.Print(responseContent);
        await SaveGreyhoundRacesAsync(responseContent);
    }

    public async Task SaveGreyhoundRacesAsync(string jsonResponse)
    {
        BetfairResponse<MarketCatalogue>? betfairResponse =
        JsonConvert.DeserializeObject<BetfairResponse<MarketCatalogue>>(jsonResponse);
        List<RaceFixture> raceFixtures = new();
        List<RunnerIDSelection> runnerIDSelections = new();

        Dictionary<string, List<DateTime>> venueRaceIndex = new();

        foreach (MarketCatalogue market in betfairResponse.Result)
        {
            if (!venueRaceIndex.ContainsKey(market.Event.Venue))
            {
                venueRaceIndex[market.Event.Venue] = new List<DateTime>();
            }
            venueRaceIndex[market.Event.Venue].Add(market.MarketStartTime);
        }

        foreach (MarketCatalogue market in betfairResponse.Result)
        {
            RaceFixture raceFixture = new()
            {
                Bookie = "Betfair",
                Category = 4339,
                CountryCode = market.Event.CountryCode,
                EventID = market.Event.Id,
                MarketID = market.MarketId,
                MarketName = market.MarketName,
                RaceID = GetRaceID(market.Event.Venue, market.MarketStartTime, venueRaceIndex),
                RaceMeters = GetRaceMeters(market.MarketName),
                TrackCode = GetTrackCode(market.Event.Venue),
                TrackName = market.Event.Venue,
                StartsGMT = market.MarketStartTime
            };
            raceFixtures.Add(raceFixture);

            List<RunnerInfo> runners = new();
            foreach (Runner runner in market.Runners)
            {
                int dogID = GetDogID(runner.RunnerName, runner.SortPriority);
                string dogName = Regex.Replace(runner.RunnerName, @"^[1-6]\.\s*", ""); // Remove the pattern "1. " to "6. " followed by any number of spaces
                runners.Add(new RunnerInfo
                {
                    DogID = dogID,
                    SelectionID = runner.SelectionId,
                    DogName = dogName
                });
            }

            RunnerIDSelection runnerIDSelection = new()
            {
                Bookie = "Betfair",
                Category = "Greyhound",
                JsonRunners = JsonConvert.SerializeObject(runners),
                MarketID = market.MarketId,
                RaceID = raceFixture.RaceID,
                RaceDate = raceFixture.StartsGMT,
                TrackCode = raceFixture.TrackCode,
                TrackName = raceFixture.TrackName
            };
            runnerIDSelections.Add(runnerIDSelection);
        }

        BulkSaveRaceFixtures(raceFixtures);
        BulkSaveRunnerIDSelections(runnerIDSelections);
    }

    private static int GetRaceID(string venue, DateTime startTime, Dictionary<string, List<DateTime>> venueRaceIndex)
    {
        List<DateTime> sortedTimes = [.. venueRaceIndex[venue].OrderBy(t => t)];
        return sortedTimes.IndexOf(startTime) + 1;
    }

    private static int GetRaceMeters(string marketName)
    {
        string[] parts = marketName.Split(' ');
        foreach (string part in parts)
        {
            if (Regex.IsMatch(part, @"\d{3}[mM]$"))
            {
                int raceMeter1 = int.Parse(part.Replace("m", "").Replace("M", ""));
                return raceMeter1;
            }
        }
        throw new ArgumentException("Market name does not contain a valid distance.");
    }

    private string GetTrackCode(string venue)
    {
        if (DnameCode.TryGetValue(venue, out string? trackCode))
        {
            return trackCode;
        }
        else
        {
            Debug.Print(venue);
            return "UNKNOWN";
        }
    }

    private static int GetDogID(string runnerName, int sortPriority)
    {
        Match match = Regex.Match(runnerName, @"^([1-6])\.");
        if (match.Success)
        {
            return int.Parse(match.Groups[1].Value);
        }
        return sortPriority;
    }

=> The only thing different is that I create a dictionary to use first 3-letter as trackCode for each greyhound race course in both UK & Ireland.
The following is the json payload to request one day's greyhound race fixture:
{"jsonrpc":"2.0","method":"SportsAPING/v1.0/listMarketCatalogue","params":{"filter":{"eventTypeIds":["4339"],"marketCountries":["GB","IE"],"marketTypeCodes":["WIN"],"marketStartTime":{"from":"2024-11-22T00:00:00Z","to":"2024-11-23T07:20:10Z"}},"maxResults":"300","marketProjection":["MARKET_START_TIME","RUNNER_DESCRIPTION","EVENT"]},"id":1}
=> The response looks like this:
{"jsonrpc":"2.0","result":[{"marketId":"1.236266850","marketName":"A7 415m","marketStartTime":"2024-11-22T10:32:00.000Z","totalMatched":4.8028,"runners":[{"selectionId":55301333,"runnerName":"1. Foxwood Cardi","handicap":0.0,"sortPriority":1},{"selectionId":50748735,"runnerName":"2. Roscrea Karen","handicap":0.0,"sortPriority":2},{"selectionId":75235674,"runnerName":"3. Do It Bono","handicap":0.0,"sortPriority":3},{"selectionId":72963396,"runnerName":"4. Blazeaway Kiss","handicap":0.0,"sortPriority":4},{"selectionId":50748734,"runnerName":"5. Dunbolg Sif","handicap":0.0,"sortPriority":5},{"selectionId":42150090,"runnerName":"6. Keplar Isis","handicap":0.0,"sortPriority":6}],"event":{"id":"33805168","name":"Harlow 22nd Nov","countryCode":"GB","timezone":"Europe/London","venue":"Harlow","openDate":"2024-11-22T10:32:00.000Z"}},{"marketId":"1.236266852","marketName":"D5 238m","marketStartTime":"2024-11-22T10:48:00.000Z","totalMatched":42.64886400000001,"runners":[{"selectionId":61041122,"runnerName":"1. Chilteen Joy","handicap":0.0,"sortPriority":1},{"selectionId":43659624,"runnerName":"2. Glasmeen Tina","handicap":0.0,"sortPriority":2},{"selectionId":70869383,"runnerName":"3. Most Interesting","handicap":0.0,"sortPriority":3},{"selectionId":73964706,"runnerName":"4. Tousers Lad","handicap":0.0,"sortPriority":4},{"selectionId":62670031,"runnerName":"5. Buff Egan","handicap":0.0,"sortPriority":5},{"selectionId":43998090,"runnerName":"6. Burgess Mentor","handicap":0.0,"sortPriority":6}],"event":{"id":"33805168","name":"Harlow 22nd Nov","countryCode":"GB","timezone":"Europe/London","venue":"Harlow","openDate":"2024-11-22T10:32:00.000Z"}},{"marketId":"1.236266641","marketName":"A7 500m","marketStartTime":"2024-11-22T11:01:00.000Z","totalMatched":130.131866,"runners":[{"selectionId":52391251,"runnerName":"1. Insane Simone","handicap":0.0,"sortPriority":1},{"selectionId":57194876,"runnerName":"2. Antigua Anna","handicap":0.0,"sortPriority":2},{"selectionId":67127936,"runnerName":"3. Fatboy Stu","handicap":0.0,"sortPriority":3},{"selectionId":54380251,"runnerName":"4. Punk Rock
=> But I have to write rather complicated logic to parse the greyhound races fixture to get useful information.
If you can provide some example to use your repo/Nuget package to get necessary information, then it will be much helpful.
I show you some records I saved in SQL Server data table, but you don't have to use SQL Server database, just return a list of tuple, corresponding to the SQL Server data table is enough.
Here is the data table design code:
CREATE TABLE [dbo].[RaceFixture](
[Bookie]           nvarchar NOT NULL,
[Category]         [int]          NOT NULL,
[CountryCode]      nvarchar  NOT NULL,
[EventID]          nvarchar NOT NULL,
[MarketID]         nvarchar NOT NULL,
[MarketName]       nvarchar NOT NULL,
[RaceID]           [int]          NOT NULL,
[RaceMeters]       [int]          NOT NULL,
[TrackCode]        nvarchar  NOT NULL,
[TrackName]        nvarchar NOT NULL,
[StartsGMT]        [datetime]     NOT NULL,
CONSTRAINT [PK_RaceFixture] PRIMARY KEY CLUSTERED
(
[Bookie] ASC, [EventID] ASC, [MarketID] ASC, [RaceID] ASC, [TrackCode] ASC, [StartsGMT] ASC)
WITH (PAD_INDEX=OFF,STATISTICS_NORECOMPUTE=OFF,IGNORE_DUP_KEY=OFF,ALLOW_ROW_LOCKS=ON,ALLOW_PAGE_LOCKS=ON)
ON [PRIMARY]
)ON [PRIMARY]
GO
=> RaceID is the order for each race during the day for the same greyhound race course, so its value can be: 1, 2, 3, 4, 5, 6, 7, 8, etc.
RaceMeter is the distance of each race.
CREATE TABLE [dbo].[RunnerIDSelection](
[Bookie]           nvarchar  NOT NULL,
[Category]         nvarchar  NOT NULL,
[JsonRunners]      nvarcharNOT NULL,
[MarketID]         nvarchar  NOT NULL,
[RaceID]           [int]           NOT NULL,
[RaceDate]         [DateTime]      NOT NULL,
[TrackCode]        nvarchar   NOT NULL,
[TrackName]        nvarchar  NOT NULL,
CONSTRAINT [PK_RunnerIDSelection] PRIMARY KEY CLUSTERED
(
[Bookie] ASC, [MarketID] ASC, [RaceID] ASC, [TrackCode] ASC
)
WITH(PAD_INDEX=OFF,STATISTICS_NORECOMPUTE=OFF,IGNORE_DUP_KEY=OFF,ALLOW_ROW_LOCKS=ON,
ALLOW_PAGE_LOCKS=ON, OPTIMIZE_FOR_SEQUENTIAL_KEY=OFF)ON [PRIMARY]
)ON [PRIMARY]
GO
=> I want to use NewTonSoft.json to serialize all the dogs in one race.
I show you some records for this table:
SELECT * FROM RunnerIDSelection
Bookie	Category	JsonRunners	MarketID	RaceID	RaceDate	TrackCode	TrackName
Betfair	Greyhound	[{"DogID":1,"SelectionID":52391251,"DogName":"Insane Simone"},{"DogID":2,"SelectionID":57194876,"DogName":"Antigua Anna"},{"DogID":3,"SelectionID":67127936,"DogName":"Fatboy Stu"},{"DogID":4,"SelectionID":54380251,"DogName":"Punk Rock Zelda"},{"DogID":5,"SelectionID":70217730,"DogName":"Sullane Biscuit"},{"DogID":6,"SelectionID":72455527,"DogName":"Pocket Penny"}]	1.236266641	1	2024-11-22 11:01:00.000	GBHOV	Hove
The JsonRunners is something like this:
[{"DogID":1,"SelectionID":52391251,"DogName":"Insane Simone"},{"DogID":2,"SelectionID":57194876,"DogName":"Antigua Anna"},{"DogID":3,"SelectionID":67127936,"DogName":"Fatboy Stu"},{"DogID":4,"SelectionID":54380251,"DogName":"Punk Rock Zelda"},{"DogID":5,"SelectionID":70217730,"DogName":"Sullane Biscuit"},{"DogID":6,"SelectionID":72455527,"DogName":"Pocket Penny"}]
I show you some records from data table [RaceFixture]
SELECT * FROM RaceFixture
Bookie	Category	CountryCode	EventID	MarketID	MarketName	RaceID	RaceMeters	TrackCode	TrackName	StartsGMT
Betfair	4339	GB	33805140	1.236266641	A7 500m	1	500	GBHOV	Hove	2024-11-22 11:01:00.000
Betfair	4339	GB	33805140	1.236266643	D2 285m	2	285	GBHOV	Hove	2024-11-22 11:18:00.000
Betfair	4339	GB	33805140	1.236266645	A5 500m	3	500	GBHOV	Hove	2024-11-22 11:34:00.000
=> I got total 159 races for today, unfortunately, none of them is from Ireland, I think maybe tomorrow, I can get some races from Ireland.
So daily races count is around 140 to 160, averagely around 150, but in Weekend, if there are some Irish races, then the number could be more than 200; so I think a better to add more conditions, in Weekend, try to list UK & Irish races separately, but during weekdays (Monday to Friday), simply put them together.
Thanks,

It looks like you just need to serialize the response before outputting it to the debug window.
Something like:
Debug.Print(JsonSerializer.Serialize(top200Races);

My MarketCatalogue response object is a copy of the json structure returned from Betfair, so you should just be able to slot that into your existing code.

The information from Betfair is returned in the response from the call to await client.MarketCatalogue(filter, query);
So you should do something like this:
        var query = new MarketCatalogueQuery()
            .Include(MarketProjection.Event)
            .Include(MarketProjection.MarketStartTime)
            .Include(MarketProjection.MarketDescription)
            .Include(MarketProjection.RunnerDescription);

        var marketCatalogues = await client.MarketCatalogue(filter, query);

        Debug.Print(JsonConvert.SerializeObject(marketCatalogue));

The MarketCatalogueQuery() object is the query sent to Betfair. It does not contain any response data. It is equivalent to the MarketProject in the request you are sending to Betfair in your current code "marketProjection":["MARKET_START_TIME","RUNNER_DESCRIPTION","EVENT"].
If you do want to output that query to the debug window though, you just need to serialize it into a string object first.

Hi,
Do you have code to receive the response, like this response:
{"jsonrpc":"2.0","result":[{"marketId":"1.236266850","marketName":"A7 415m","marketStartTime":"2024-11-22T10:32:00.000Z","totalMatched":4.8028,"runners":[{"selectionId":55301333,"runnerName":"1. Foxwood Cardi","handicap":0.0,"sortPriority":1},{"selectionId":50748735,"runnerName":"2. Roscrea Karen","handicap":0.0,"sortPriority":2},{"selectionId":75235674,"runnerName":"3. Do It Bono","handicap":0.0,"sortPriority":3},{"selectionId":72963396,"runnerName":"4. Blazeaway Kiss","handicap":0.0,"sortPriority":4},
By the way, I think it is strange, today is Friday, so there were quite a number of Irish greyhound races, as I can see the from this url: https://www.grireland.ie/racing/upcoming-race-cards/
But all returned 158 races are from UK.  Ireland should have at least 90 or nearly 100 races, but the response returned no such races at all.
Can you check your code, just return some greyhound races in Ireland?
But I want you show me the code to see the response in json format, not only the request in json format.
Thanks,

Hi,
Basically, I understand how to use your repo to do the job, but still I have some question about the returned results.
Here is the json format request I made via Postman HTTP request, the payload:
{
"jsonrpc": "2.0",
"method": "SportsAPING/v1.0/listMarketCatalogue",
"params": {
"filter": {
"eventTypeIds": [
"4339"
],
"marketCountries": [
"IE", "GB"
],
"marketTypeCodes": [
"WIN"
],
"marketStartTime": {
"from": "2024-11-22T23:59:00+00:00",
"to": "2024-11-23T23:59:00+00:00"
}
},
"maxResults": "300",
"marketProjection": [
"MARKET_START_TIME",
"RUNNER_DESCRIPTION",
"EVENT"
]
},
"id": 1
}
=> From the returned results, both from your repo & Postman HTTP response, I can see there are 181 greyhound races, only 12 races are from Ireland in one greyhound race course: Shelbourne Park
However, according to this web page:
https://www.grireland.ie/racing/upcoming-race-cards/
There are the following Irish greyhound race courses have races on November 23, 2024:
23-Nov-24 - Limerick
23-Nov-24 - Tralee
23-Nov-24 - Mullingar
23-Nov-24 - Dundalk
23-Nov-24 - Shelbourne Park
23-Nov-24 - Waterford
23-Nov-24 - Galway
23-Nov-24 - Drumbo Park
23-Nov-24 - Youghal
23-Nov-24 - Lifford
23-Nov-24 - Curraheen Park
=> But API returned results contained only one greyhound race course, but actually, total 11 Irish greyhound race courses have races on 23-Nov-2024, the API return only 1, but missing 10 Irish  greyhound race courses'  races, is there any issue with the API?

The issue is that Betfair does not cover every single Irish Greyhound race. If you check the Betfair website you'll see they only list one venue on 23-Nov, Shelbourne.

OK, thanks!


Adding subsection in the outline ?

Hello everyone,
I am a new user of typst and polylux, and I have not yet explored how to create or adapt an existing theme to my needs.
Before doing so, I have a somewhat naive question: do you think it is possible to integrate subsections into the document structure and have them appear in the outline? We rarely need them for short presentations, but I often use this second level of structure in my classes.
Thank you very much! I apologize if the answer to my question seems obvious.

#outline(target: heading) includes all headings (aka. subsections) in the outline. If you want to stop at specific level, I guess you can use heading.where(level: {you desired level}). You might also want to add the indent: 1em argument to indent subsections.


run error on Example

Hi,
I just downloaded quadpy using
sudo pip install quadpu --user    (in computer A)
sudo pip3 install quadpu --user    (in computer B)
Created a file name "test.py" with the content:
import numpy
import quadpy

def f(x):
    return numpy.sin(x[0]) * numpy.sin(x[1])

triangle = numpy.array([[0.0, 0.0], [1.0, 0.0], [0.7, 0.5]])
val = quadpy.triangle.strang_fix_cowper_09().integrate(f, triangle)

running the code  using python test.py, I get the following error:
python teste.py

Traceback (most recent call last):
  File "teste.py", line 2, in <module>
    import quadpy
  File "/usr/local/lib/python2.7/dist-packages/quadpy/__init__.py", line 1, in <module>
    from . import (
  File "/usr/local/lib/python2.7/dist-packages/quadpy/disk/__init__.py", line 1, in <module>
    from ._albrecht import (
  File "/usr/local/lib/python2.7/dist-packages/quadpy/disk/_albrecht.py", line 6, in <module>
    from ._helpers import DiskScheme
  File "/usr/local/lib/python2.7/dist-packages/quadpy/disk/_helpers.py", line 7
    def __init__(self, name, weights, points, degree: int, citation=None):
                                                    ^
SyntaxError: invalid syntax

Probably some installation issue, by I`m having a hard time figuring it out.

quadpy doesn't support Python 2 anymore, and I'm not sure how you got it to install.
/usr/local/lib/python2.7

Use Python 3.


Allow to use sandbox for tasks that never ran

Before this patch, tasks that never ran (for instance a task that failed
to download its artifacts) could not be explored using sandbox page.
This was due to the fact that mesos master state had no running status
and we rely on this status to extract various container ids (necessary
to request sandbox content and launch session within container).
In the case of task that never ran, we can use the slave state to
extract the container id which is the only necessary information to
explore sandbox. We can't find the parent container id but don't need it
since it is required to launch a session only.
This patch pushed me to extend the MesosTask interface with container
info.
Change-Id: I67e57c6928d9b36107d733d24a628813e82ed251
JIRA: MESOS-4060

Note: I'm completely clueless on how to write test for this case. If you want tests for it, please guide me so I can implement something.


Speed up analytica solution by using scipy.special.hankel2

This merge speeds up the computation of the analytic solution by using the pre-compiled scipy version of the hankel2 function. @mloubout, can you please verify that the outcome is still correct?

It creates an error
TypeError: ufunc 'hankel2' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
The inverse fourier transform is also quite expensive, and this only would be run once for testing, so I don;t think speed matter that much for this one

My bad, indeed much faster, thank you @mlange05


Version 1.1.0 not on npm

Question
I was wondering if there is a reason why version 1.1.0 is not on npm. I would like to use this plugin with the hash option turned off but the latest version on npm is 1.0.2

+1, would love to have the hash option. @khalwat can we get a 1.1.0 release?

Ah sorry, I will get that rectified

Hi @khalwat ...not trying to be a pain but any updates?

Thx for the reminder... published! https://www.npmjs.com/package/save-remote-file-webpack-plugin


[BACKPORT #2189] add ChatHandlerServerModule to realtime example

To fix broken realtime example

:+1: travis needs to go fasterz

:+1:


Correction of german localization (#2649)

Per the Contributing Guidelines, please only update the files under /src.  The others are auto generated.  You need to update both /src/locale/de.js and /src/test/locale/de.js.  Thanks.

@lothargrieb any progress?

@lluchs any comments on this?
According to CLDR nominative and accusative shortened cases are different (one has dots, the other doesn't).

@lothargrieb can you please provide a regex to distinguish between standalone weekdays vs ones used in conjunciton with other stuff, like here:
https://github.com/moment/moment/pull/2741/files#diff-a03fc1ced8170ad648ebb9efeb2611a8R65
As I said in my earlier post, it looks like we need two variants (accusative / nominative or standalone / format, depending on how you want to call them).

Closing for inactivity.


added visualize py and cleaned up load data

lingering issue -- visualize.bubble_chart loads a blank chart webpage

Hello @kvanalebeek! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file py/visualize.py:

Line 29:80: E501 line too long (87 > 79 characters)
Line 30:80: E501 line too long (84 > 79 characters)
Line 33:80: E501 line too long (96 > 79 characters)
Line 42:80: E501 line too long (102 > 79 characters)
Line 55:80: E501 line too long (85 > 79 characters)
Line 60:80: E501 line too long (91 > 79 characters)
Line 90:80: E501 line too long (83 > 79 characters)
Line 119:1: E402 module level import not at top of file


F/d10 readiness

@stanbellcom, I can create a new release, but I'm a bit surprised that/if this experiment is actually used in the wild. Do you, i.e. your clients, use it for real?


Video QA with InternLM-XComposer and Fine-tuning Code Availability

1、How does InternLM-XComposer handle video QA? The examples seem to only involve images.
2、Will the fine-tuning code be open-sourced soon?

Thanks for your attention.

Video is not supported for our model now, but many ways exist to extend our model to understand videos, such as the image-band.
The open source of the fine-tuning code is on the way.


update Docker base image

We haven't checked our base image's version for about one year. For security reasons, we should update it regularly.

@langjoh do we still need that?

Not this ticket, no. Just Ruby upgrades.


Any way to disable "prone" but keep the swimming animation?

 The rest is fantastic, but this part is definitely rough.
Is there a way to disable going prone? Or, at least a potential update to add a config option for prone-on-land?

This should be doable; I'll look into it.


Personal Stats Widget

Displays some nice stats based on Pirep Model and requires vmsAcars (or any compatible acars solution) , can be improved/edited to display manual pireps too with reduced capacity.
***** Options : PersonalStats
user can be any user's id or not used at all
period can be any number of days (except 0 of course) or not used at all
disp can be full or not used at all
type can be avglanding, avgscore, avgtime, tottime, avgdistance, totdistance, avgfuel, totfuel, totflight
If no user is defined, widget will get current user's data for calculations. This may be used for dashboard or any personal pages where the viewer will be able to see his results. If stats are needed on the user profile page then 'user' option is required.
('user' => $user->id is enough to get proper results at profile page)
If no period is defined then all accepted acars reports will be used for calculations, else the results for the last n days provided
('period' => 7 will give last 7 days)
If a full card is required with the results and info text then 'disp' => full must be used while while calling the widget. It should be compatible with the default template (and any others too) but if needed personalstats.blade may be edited.
I personally use the plain text results on my dashboard and 'disp' => full in user profile pages.
Designed it for personal use, but if they are good enough we can have them in the main pack too.

Closing this pr, I think if someone is interested to build a better one can check closed requests ;)

Closing this pr, I think if someone is interested to build a better one can check closed requests ;)


moveit_resources: 2.1.0-1 in 'iron/distribution.yaml' [bloom]

Increasing version of package(s) in repository moveit_resources to 2.1.0-1:

upstream repository: https://github.com/ros-planning/moveit_resources.git
release repository: https://github.com/ros2-gbp/moveit_resources-release.git
distro file: iron/distribution.yaml
bloom version: 0.11.2
previous version for package: 2.0.6-3

dual_arm_panda_moveit_config
* Remove comments from scalars (#169 <https://github.com/ros-planning/moveit_resources/issues/169>)
* Update information about Ruckig and jerk limits (#168 <https://github.com/ros-planning/moveit_resources/issues/168>)
* Add dual arm panda config package from moveit2_tutorials (#164 <https://github.com/ros-planning/moveit_resources/issues/164>)
* Contributors: AndyZe, Sebastian Castro, Stephanie Eng

moveit_resources

No changes

moveit_resources_benchmarking
* Enable AnytimePathShortening and fix benchmarks (#167 <https://github.com/ros-planning/moveit_resources/issues/167>)
* Add benchmark data package (#162 <https://github.com/ros-planning/moveit_resources/issues/162>)
* Contributors: Sebastian Jahr

moveit_resources_fanuc_description

No changes

moveit_resources_fanuc_moveit_config
* Remove comments from scalars (#169 <https://github.com/ros-planning/moveit_resources/issues/169>)
* Enable AnytimePathShortening and fix benchmarks (#167 <https://github.com/ros-planning/moveit_resources/issues/167>)
* Update information about Ruckig and jerk limits (#168 <https://github.com/ros-planning/moveit_resources/issues/168>)
* Add execution_duration_monitoring param to the example config (#160 <https://github.com/ros-planning/moveit_resources/issues/160>)
* Contributors: AndyZe, Sebastian Jahr, Stephanie Eng

moveit_resources_panda_description

No changes

moveit_resources_panda_moveit_config
* Cherry pick TopicBased rename #163 <https://github.com/ros-planning/moveit_resources/issues/163> (#172 <https://github.com/ros-planning/moveit_resources/issues/172>)
  Co-authored-by: Jafar<EMAIL_ADDRESS>* Add STOMP planner config (#171 <https://github.com/ros-planning/moveit_resources/issues/171>)
  Co-authored-by: Henning Kayser<EMAIL_ADDRESS>* Remove comments from scalars (#169 <https://github.com/ros-planning/moveit_resources/issues/169>)
* Enable AnytimePathShortening and fix benchmarks (#167 <https://github.com/ros-planning/moveit_resources/issues/167>)
* Update information about Ruckig and jerk limits (#168 <https://github.com/ros-planning/moveit_resources/issues/168>)
* Enable acceleration limits so TOTG can work (#166 <https://github.com/ros-planning/moveit_resources/issues/166>)
* Add joint limits to Panda demo launch file + make consistent with tutorials (#165 <https://github.com/ros-planning/moveit_resources/issues/165>)
* [ROS2]: Add a new parameter to panda's xacro to select ros2_control hardware interface type (#158 <https://github.com/ros-planning/moveit_resources/issues/158>)
  * Add xacro parameter to select ros2_control hardware interface type
  * Update panda demo launch file to have ros2_control_hardware_type argument
  * Add argument to the launch description
* Add execution_duration_monitoring param to the example config (#160 <https://github.com/ros-planning/moveit_resources/issues/160>)
* Fix xacro deprecation warning: use xacro.load_yaml() (#156 <https://github.com/ros-planning/moveit_resources/issues/156>)
* Add Pilz to panda launch (#154 <https://github.com/ros-planning/moveit_resources/issues/154>)
* Add pilz_industrial_motion_planner_planning.yaml (#151 <https://github.com/ros-planning/moveit_resources/issues/151>)
* Contributors: AndyZe, Giovanni, Jafar, Marq Rasmussen, Sebastian Castro, Sebastian Jahr, Stephanie Eng

moveit_resources_pr2_description

No changes

Replaced by https://github.com/ros/rosdistro/pull/37274


O3D303 Not showing image output from the Camera on Ubuntu 18.04

So I installed ifm3d v0.14.0 using apt on my Ubuntu OS.
Following is the configuration of my laptop:
Linux srane-p5530.local 4.18.0-24-generic #25~18.04.1-Ubuntu SMP Thu Jun 20 11:13:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
I also downloaded the ifm3d-examples repository. I am able to ping my camera but whenever I try to run basic examples, I get the following error:
Timeout waiting for camera!
Also, I tried running ifm3d-pcl-viewer, I got a completely blank screen.
Whenever I run ifm3d dump  command, it successfully shows the configuration (see attachment
camera_config.txt
. Also, I am able to create and delete new applications.
Following is the firmware version of my device : 1.23.2848
Opencv version I have is 4.1.0
Can anyone please tell me where the problem might be lying?

Sorry, it was because it already had two applications preinstalled that were affecting the firmware settings. So after running the command  $ ifm3d reset it started working.

This is most probably due to the fact that the active application was set to hardware or software trigger.


Multiple errors: Usage, AwaitMessage...

Version of Komada
0.18.10, but it's present since years ago.
Are core files modified?
No
Describe the problem
1.- Usage
When a command uses the mention type in usage, and the user doesn't mention the user, it prints this message:
[2017-03-26 12:24:45] TypeError: Cannot read property '0' of null
    at validateArgs (/mnt/c/Users/manue/node_modules/komada/functions/usage.js:103:82)
    at Promise (/mnt/c/Users/manue/node_modules/komada/functions/usage.js:580:4)
    at Object.exports.run (/mnt/c/Users/manue/node_modules/komada/functions/usage.js:9:55)
    at Object.module.exports [as handleCommand] (/mnt/c/Users/manue/functions/handleCommand.js:11:45)
    at client.on (/mnt/c/Users/manue/node_modules/komada/app.js:79:18)
    at process._tickCallback (internal/process/next_tick.js:103:7)

2.- Boolean validator
Another thing in usage.js (L116-L121), it's validating if the argument is false, but not when it's true.
if (args[i] === "true") {
  args[i] = true;
} else {
  args[i] = false;
  validateArgs(++i);
}

3.- Awaitmessage
Also, now that we are here, add if (message.deletable) in L25 from awaitMessage, if you delete the message before it gets deleted, or komada couldn't send the message, it'll print this:
Uncaught Promise Error:
TypeError: message.delete is not a function
    at module.exports (/mnt/c/Users/manue/node_modules/komada/functions/awaitMessage.js:25:19)

4.- Commands
Also, I know that it's hard to catch, but if the bot has permissions to read a channel, but not to send a message into it, for example, the command ping:
[2017-03-26 12:17:55] Error: Forbidden
    at Request.callback (/mnt/c/Users/manue/node_modules/superagent/lib/node/index.js:675:11)
    at /mnt/c/Users/manue/node_modules/superagent/lib/node/index.js:883:18
    at Stream.<anonymous> (/mnt/c/Users/manue/node_modules/superagent/lib/node/parsers/json.js:16:7)
    at emitNone (events.js:86:13)
    at Stream.emit (events.js:186:7)
    at Unzip.<anonymous> (/mnt/c/Users/manue/node_modules/superagent/lib/node/unzip.js:53:12)
    at emitNone (events.js:91:20)
    at Unzip.emit (events.js:186:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)
Uncaught Promise Error:
TypeError: Cannot read property 'edit' of undefined
    at exports.run (/mnt/c/Users/manue/commands/System/ping.js:3:10)
    at process._tickCallback (internal/process/next_tick.js:103:7)

Should check if the message is editable before "trying" to edit it.
Expected Behaviour
1.- Usage
Should run this line (L111) in case of mention type. I have checked the code and might be reproduceable for member. Since in /\d+/.exec(args[i])[0], when there's no number, you're trying to read the property [0] of null.
return reject(client.funcs.newError(`${currentUsage.possibles[0].name} must be a mention or valid user id.`, 1, args));

2.- Boolean validator
It should validate if the argument is either true or false, currently it does only for false.
3.- Awaitmessage
Should verify if the message was sent and is editable, instead of trying to edit it.
4.- Commands
Previously check if the command requires SEND_MESSAGES permissions before trying to run it. In conf.botPerms, maybe?
Actual Behaviour
Everything explained above.
Steps to Reproduce.
¯\_(ツ)_/¯

/\d+/.exec(args[i])[0] We test beforehand for digits or we invalidate it. I just tested it and it works fine and as intended.

          if (/^<@!?\d+>$/.test(args[i]) || (client.users.has(/\d+/.exec(args[i])[0]) && args[i].length > 5)) {

The validateArgs needs to be moved outside of the if
I guess.
just no.

The issue with 1. is when you insert text only, with no digits, then, nothing in the provided string maths, returning null.

Issue Workflow

[ ] Usage
[ ] Boolean Validator
[ ] Await Message
[ ] Commands

Issue 4 is already a part of core Komada. Closing this as everything else is finished.


Allow non-aligned inputs to the model

This allows avoiding copies between the pytorch and TVM portions of the tests/benchmark.py script.  Otherwise, the alignment requirement prevents passing tvm.nd.from_dlpack(tokens[:, prev:current]) as an input to the model.

Depends on TVM PR#14771


[BUGFIX] Update the braille type and the translation of all items in main menu

Problem
When changing the language or type of braille, the transport and extinction elements were not updated.
Solution
Manually update the items in the translate_ui function of the internal file.
Note
The version number was changed by a previous pull request, I restored it to 3.0.4 until the next version of the repository.

Modif portée dans la version 3.1