feat: Add compatibility for ArgoCD 2.4 prefixed environment variables

Description
ArgoCD 2.4 adds a security fix where all environment variables passed to the repo server during the init and generate phases are prefixed with ARGOCD_ENV to prevent users from setting potentially sensitive environment variables. The argocd-vault-plugin binary needs to read the required values from env vars with the new prefix. This PR adds another flag to set (USE_PREFIX) to switch from reading un-prefixed to reading prefixed env vars.
Checklist
Please make sure that your PR fulfills the following requirements:

[ X] Reviewed the guidelines for contributing to this repository
[ ]X The commit message follows the Conventional Commits Guidelines.
[ X] Tests for the changes have been updated
[ ] Are you adding dependencies? If so, please run go mod tidy -compat=1.17 to ensure only the minimum is pulled in.
[ X] Docs have been added / updated
[ ] Optional. My organization is added to USERS.md.

Type of Change

[ ] Bugfix
[ X] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] New tests
[ ] Build/CI related changes
[ X] Documentation content changes
[ ] Other (please describe)

Other information
The ArgoCD 2.4 breaking change was mentioned in #352. I tried looking into ways to have the config.go module to read from both the prefixed and unprefixed env vars, but I couldn't get Viper to do that without having to rewrite how those variables read out of the config manager. This was the least intrusive way I could find to make this compatible with 2.4.
I have tested it on our own ArgoCD 2.4 instance with no issues so far.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Edward Mao seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for the PR @edjmao! I would really like to find a way that allows seamless use with 2.4 without introducing a new annotation. Will try to take a look on Monday to maybe come up with some options

One thing we could do is, in the config, specifically https://github.com/argoproj-labs/argocd-vault-plugin/blob/main/pkg/config/config.go#L271, we could find all env vars with the ARGOCD_ENV prefix and then strip the prefix and set the new var without the prefix.
@jkayani what are your thoughts on this? Or any other ideas?

I agree that the best is if a seamless transition could occur. This was just my quick attempt to get something working for my team, as we had to upgrade ArgoCD. I'm still trying to figure out how Viper works, and I must have missed that for loop at the bottom. I'm happy to close this PR if a different approach is decided upon.

@edjmao do you want to try giving this..

One thing we could do is, in the config, specifically https://github.com/argoproj-labs/argocd-vault-plugin/blob/main/pkg/config/config.go#L271, we could find all env vars with the ARGOCD_ENV prefix and then strip the prefix and set the new var without the prefix.

a shot?

Yeah, I can try this in my spare evenings this week.

Hi all, sorry for my absence on this discussion - I agree with @werne2j that it'd be best to quietly make things work, rather than having an explicit config option a user would have to set for compatibility with ArgoCD 2.4+ vs earlier releases.
The way Viper is being used right now is that it looks for env variables that match the config key on each Viper lookup (via viper.AutomaticEnv()) - I don't think we'd want to duplicate config keys for Argo CD 2.4 vs earlier and replace each v.Get(thing) with v.Get(thing) or v.Get(ARGOCD_ENV_thing) so we'd actually need to do something like this pseudo code at line 270 https://github.com/argoproj-labs/argocd-vault-plugin/blob/main/pkg/config/config.go#L270:
argocd24 := "ARGOCD_ENV_(.*)"
for k, value := range os.Getenv() {
   if k.Matches(argocd24) {
     configKey := k.Execute(argocd24)[1] // get the capturing group contents as the config key
     v.Set(configKey, value)
   }
}

for k, viperValue := range v.AllSettings() {
   ... rest of code from line 272 onward here

That way any non-AVP specific keys set at the app manifest (ARGOCD_ENV_VAULT_*) are forwarded to the backend SDK and AVP specific keys (ARGOCD_ENV_AVP_KV_VERSION) are read and honored. We can't rely on finding them in viper.AllSettings() without the above because the ARGOCD_ENV keys are only present as environment variables and aren't explicitly loaded into Viper anywhere.
@edjmao Let me know if this makes sense or not. I appreciate the effort you've made so far on this PR, hoping we can add this on and without too much trouble!

May want to make sure to mention the change in behavior. I hadn't upgraded yet but I was going to need to figure out how to deal with setting AWS_REGION for my setup since that is then used by the AWS SDK/Secrets Manager to talk to AWS.

Goal is not have any change in behavior and have things work seamlessly regardless of the version of Argo CD being used (set var X in app manifest, and AVP will use it regardless of whether it sees X or ARGOCD_ENV_X) - it's true though that we'll need to update the docs in a few places like here: https://argocd-vault-plugin.readthedocs.io/en/stable/usage/#with-helm

Found some time to revisit this PR this week. The quickest change I was able to make was to simply replace the lookup of another environment variable with the presence of ARGOCD_ENV_AVP_TYPE, since I assume that every usage of this plugin necessitates this environment variable.
_, prefixFound := os.LookupEnv(fmt.Sprintf("%s_%s", types.EnvArgoCDPrefix, types.EnvAvpType))
if prefixFound {
    v.SetEnvPrefix(types.EnvArgoCDPrefix)
}

I did end up going the route that @jkayani recommended for completeness. It's got more steps, but it's more versatile. I've update the PR with those changes.
	for _, envVar := range os.Environ() {
		if strings.HasPrefix(envVar, types.EnvArgoCDPrefix) {
			envVarPair := strings.SplitN(envVar, "=", 2)
			key := strings.TrimPrefix(envVarPair[0], types.EnvArgoCDPrefix+"_")
			val := envVarPair[1]
			v.Set(key, val)
		}
	}

@edjmao Can you rebase your branch with main? Thanks!

Codecov Report

Merging #356 (0655f9b) into main (45a4c25) will increase coverage by 0.11%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #356      +/-   ##
==========================================
+ Coverage   77.53%   77.64%   +0.11%
==========================================
  Files          22       22
  Lines        1006     1011       +5
==========================================
+ Hits          780      785       +5
  Misses        142      142
  Partials       84       84

Impacted Files
Coverage Δ

pkg/config/config.go
84.31% <100.00%> (+0.52%)
:arrow_up:

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

Rebased from main.


Probably wrong rule in "Build a Personal Portfolio Webpage"

Challenge Name
Build a Personal Portfolio Webpage
Issue Description
One of the rules is "Choose your own tribute subject". This is probably accidentally copied from the tribute project's requirements.

Thank you for report.  this issue need to fix

number 5 is wrong.

@erictleung I'll take care of this...


[manila-csi-plugin]Can't create pvc as failed to retrieve extra specs:

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:
/kind bug

The binaries affected:

[ ] openstack-cloud-controller-manager
[ ] cinder-csi-plugin
[ ] k8s-keystone-auth
[ ] client-keystone-auth
[ ] octavia-ingress-controller
[x] manila-csi-plugin
[ ] manila-provisioner
[ ] magnum-auto-healer
[ ] barbican-kms-plugin

What happened:
when create pvc, the nfs provisioner shows error
I0210 03:24:24.372247       1 reflector.go:432] sigs.k8s.io/sig-storage-lib-external-provisioner/controller/controller.go:801: Watch close - *v1.PersistentVolumeClaim total 3 items receivedI0210 03:25:42.096870       1 controller.go:1199] provision "manila/new-nfs-share-pvc" class "csi-manila-nfs": started
I0210 03:25:42.100551       1 controller.go:523] CreateVolumeRequest {Name:pvc-0749bcb7-6e92-4b09-9198-975f3c58ed5b CapacityRange:required_bytes:1073741824  VolumeCapabilities:[mount:<fs_type:"ext4" > access_mode:<mode:MULTI_NODE_MULTI_WRITER > ] Parameters:map[csi.storage.k8s.io/node-publish-secret-name:csi-manila-secrets csi.storage.k8s.io/node-publish-secret-namespace:manila csi.storage.k8s.io/node-stage-secret-name:csi-manila-secrets csi.storage.k8s.io/node-stage-secret-namespace:manila csi.storage.k8s.io/provisioner-secret-name:csi-manila-secrets csi.storage.k8s.io/provisioner-secret-namespace:manila osSecretName:csi-manila-secrets osSecretNamespace:manila osShareAccessID:809e1b6d-3fd7-4f06-a324-b5b3492c54dd shareNetworkID:3f87af64-72d9-4cd1-8b89-433f7b1eed8a type:default] Secrets:map[] VolumeContentSource:<nil> AccessibilityRequirements:<nil> XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I0210 03:25:42.100699       1 event.go:281] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"manila", Name:"new-nfs-share-pvc", UID:"0749bcb7-6e92-4b09-9198-975f3c58ed5b", APIVersion:"v1", ResourceVersion:"29828272", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "manila/new-nfs-share-pvc"
I0210 03:25:42.103463       1 connection.go:182] GRPC call: /csi.v1.Controller/CreateVolume
I0210 03:25:42.103481       1 connection.go:183] GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-0749bcb7-6e92-4b09-9198-975f3c58ed5b","parameters":{"osSecretName":"csi-manila-secrets","osSecretNamespace":"manila","osShareAccessID":"809e1b6d-3fd7-4f06-a324-b5b3492c54dd","shareNetworkID":"3f87af64-72d9-4cd1-8b89-433f7b1eed8a","type":"default"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":5}}]}
I0210 03:25:42.489844       1 connection.go:185] GRPC response: {}
I0210 03:25:42.490545       1 connection.go:186] GRPC error: rpc error: code = Internal desc = failed to get Manila capabilities for share type default: failed to retrieve extra specs: Request forbidden: [GET https://openstack/v2/54e164a854bc4fe0861ce1976337fff1/types/default/extra_specs], error message: {"forbidden": {"message": "Access was denied to this resource.", "code": 403}}

As use in openstack CLI with same credentials, I can retrieve the extra spec, and also can create a share
manila  -d  extra-specs-list
....
REQ: curl -i -X GET https://openstack/v2/54e164a854bc4fe0861ce1976337fff1/types?is_public=all -H "X-Openstack-Manila-Api-Version: 2.49" -H "X-Auth-Token: gAAAAABeQPMkZS4KuRNK_szmp45_jS2-eOJVZMmgOxnwNmw8siB9CnW8HVHcDCMNbDEuBQfuOeRLvm4roeojpF77ikm_WXs7JakB7qRg43mJIZlW7xUz30k2Wlwwuka5n_ZqbBP8a4SU2GMAyuRLQC9MDAe0MLSLtruwuW0FixkGZ0jgXBy3ow8KGx5SCdms8UjxaEefF5aWmj8tuXOrKUKQhMlCWtGExKKdQO8E0d_Q_qaI9VOovS8" -H "Accept: application/json" -H "User-Agent: python-manilaclient"

DEBUG (connectionpool:813) Starting new HTTPS connection (1): openstack:443
DEBUG (connectionpool:393) https://openstack:443 "GET /v2/54e164a854bc4fe0861ce1976337fff1/types?is_public=all HTTP/1.1" 200 None
RESP: [200] {'Content-Encoding': 'gzip', 'Transfer-Encoding': 'chunked', 'x-compute-request-id': 'req-e2abd541-5f56-44d7-8734-7a703b611484', 'Strict-Transport-Security': 'max-age=15724800; includeSubDomains', 'Vary': 'Accept-Encoding, X-OpenStack-Manila-API-Version', 'X-OpenStack-Manila-API-Version': '2.49', 'Server': 'nginx/1.15.3', 'Connection': 'keep-alive', 'Date': 'Mon, 10 Feb 2020 06:07:35 GMT', 'Content-Type': 'application/json'}
RESP BODY: {"volume_types": [{"required_extra_specs": {"driver_handles_share_servers": "1"}, "description": null, "share_type_access:is_public": true, "is_default": true, "extra_specs": {"create_share_from_snapshot_support": "True", "mount_snapshot_support": false, "revert_to_snapshot_support": "True", "snapshot_support": "1", "driver_handles_share_servers": "1"}, "id": "ad9b3eb1-b7b6-4fcf-b193-cd92e5519b86", "name": "default"}], "share_types": [{"required_extra_specs": {"driver_handles_share_servers": "1"}, "description": null, "share_type_access:is_public": true, "is_default": true, "extra_specs": {"create_share_from_snapshot_support": "True", "mount_snapshot_support": false, "revert_to_snapshot_support": "True", "snapshot_support": "1", "driver_handles_share_servers": "1"}, "id": "ad9b3eb1-b7b6-4fcf-b193-cd92e5519b86", "name": "default"}]}

DEBUG (httpclient:208) RESP: [200] {'Content-Encoding': 'gzip', 'Transfer-Encoding': 'chunked', 'x-compute-request-id': 'req-e2abd541-5f56-44d7-8734-7a703b611484', 'Strict-Transport-Security': 'max-age=15724800; includeSubDomains', 'Vary': 'Accept-Encoding, X-OpenStack-Manila-API-Version', 'X-OpenStack-Manila-API-Version': '2.49', 'Server': 'nginx/1.15.3', 'Connection': 'keep-alive', 'Date': 'Mon, 10 Feb 2020 06:07:35 GMT', 'Content-Type': 'application/json'}
RESP BODY: {"volume_types": [{"required_extra_specs": {"driver_handles_share_servers": "1"}, "description": null, "share_type_access:is_public": true, "is_default": true, "extra_specs": {"create_share_from_snapshot_support": "True", "mount_snapshot_support": false, "revert_to_snapshot_support": "True", "snapshot_support": "1", "driver_handles_share_servers": "1"}, "id": "ad9b3eb1-b7b6-4fcf-b193-cd92e5519b86", "name": "default"}], "share_types": [{"required_extra_specs": {"driver_handles_share_servers": "1"}, "description": null, "share_type_access:is_public": true, "is_default": true, "extra_specs": {"create_share_from_snapshot_support": "True", "mount_snapshot_support": false, "revert_to_snapshot_support": "True", "snapshot_support": "1", "driver_handles_share_servers": "1"}, "id": "ad9b3eb1-b7b6-4fcf-b193-cd92e5519b86", "name": "default"}]}

+--------------------------------------+---------+-------------------------------------------+
| ID                                   | Name    | all_extra_specs                           |
+--------------------------------------+---------+-------------------------------------------+
| ad9b3eb1-b7b6-4fcf-b193-cd92e5519b86 | default | create_share_from_snapshot_support : True |
|                                      |         | mount_snapshot_support : False            |
|                                      |         | revert_to_snapshot_support : True         |
|                                      |         | snapshot_support : 1                      |
|                                      |         | driver_handles_share_servers : 1          |
+--------------------------------------+---------+-------------------------------------------+

What you expected to happen:
pvc based on mainla nfs share provisioned
How to reproduce it:
I deployed manila cis plugin via helm, with values.yaml
# Base name of the CSI Manila driver
driverName: manila.csi.openstack.org

# Enabled Manila share protocols
shareProtocols:
  - protocolSelector: NFS
    fwdNodePluginEndpoint:
      dir: /data/kubelet/plugins/cinder.csi.openstack.org
      sockFile: csi.sock
#  - protocolSelector: CEPHFS
#    fwdNodePluginEndpoint:
#      dir: /data/kubelet/plugins/csi-cephfsplugin
#      sockFile: csi.sock

# CSI Manila image spec
csimanila:
  image:
    repository: k8scloudprovider/manila-csi-plugin
    tag: v1.17.0
    pullPolicy: IfNotPresent

# DeamonSet deployment
nodeplugin:
  # Component name
  name: nodeplugin
  # CSI Manila container compute resources constraints
  nodeplugin:
    resources: {}
  # csi-node-driver-registrar
  registrar:
    image:
      repository: quay.io/k8scsi/csi-node-driver-registrar
      tag: v1.2.0
      pullPolicy: IfNotPresent
    resources: {}
  nodeSelector: {}
  tolerations: []
  affinity: {}
  # Use fullnameOverride to fully override the name of this component
  # fullnameOverride: some-other-name

# StatefulSet deployment
controllerplugin:
  name: controllerplugin
  # CSI Manila container compute resources constraints
  nodeplugin:
    resources: {}
  # CSI external-provisioner container spec
  provisioner:
    image:
      repository: quay.io/k8scsi/csi-provisioner
      tag: v1.5.0
      pullPolicy: IfNotPresent
    resources: {}
  # CSI external-snapshotter container spec
  snapshotter:
    image:
      repository: quay.io/k8scsi/csi-snapshotter
      tag: v2.0.1
      pullPolicy: IfNotPresent
    resources: {}
  nodeSelector: {}
  tolerations: []
  affinity: {}
  # Use fullnameOverride to fully override the name of this component
  # fullnameOverride: some-other-name

# Override the default app name using nameOverride
# nameOverride: some-other-name

Anything else we need to know?:
Environment:

openstack-cloud-controller-manager version:  v1.16.0
OpenStack version:   queens
Others: kubernetes 1.16

@jenningsloy318 seems like you might be hitting https://github.com/kubernetes/cloud-provider-openstack/issues/904 . It's fixed in the most recent revisions, but we seem to be lacking latest tag for manila-csi-plugin image, and v1.17.0 you're using is not recent enough so it doesn't contain this fix... While this is sorted out, could you please try building manila-csi-plugin from source and see if that works?

Another thing: you're using NFS shares, fwdNodePluginEndpoint must point to CSI NFS Node Plugin endpoint
shareProtocols:
  - protocolSelector: NFS
    fwdNodePluginEndpoint:
      dir: /data/kubelet/plugins/cinder.csi.openstack.org <-- this should be .../csi-nfsplugin

@gman0 thanks for your info, I will try to build it tomorrow.
For the fwdNodePluginEndpoint I already install cinder plugin in cluster, so this may point to that one as only one csi.sock  exists .

@jenningsloy318

For the fwdNodePluginEndpoint I already install cinder plugin in cluster, so this may point to that one as only one csi.sock exists .

Sorry but that's not going to work. When dealing with NFS shares, CSI Manila depends on CSI NFS node plugin to do the mount. You need to deploy CSI NFS and point fwdNodePluginEndpoint to CSI NFS's .sock file.

but this what document says we should do, and this config works.
https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-manila-csi-plugin.md#deployment
Before continuing, csi-controllerplugin.yaml and csi-nodeplugin.yaml manifests need to be modified: fwd-plugin-dir needs to point to the correct path containing the .sock file of the other CSI driver. Next, MANILA_SHARE_PROTO and FWD_CSI_ENDPOINT environment variables must be set. Consult --share-protocol-selector and --fwdendpoint command line flags for valid values.

@jenningsloy318 it may seemingly deploy without errors, but it will fail if you'd try to mount a PV.
From the docs:

The CSI Manila driver deals with the Manila service only. All node-related operations (attachments, mounts) are performed by a dedicated CSI Node Plugin, to which all Node Service RPCs are forwarded. This means that the operator is expected to already have a working deployment of that dedicated CSI Node Plugin.

CSI Manila itself doesn't contain the tools needed to mount a file-system. It relies on a partner Node Plugin (which is able to handle that particular file-system, in this case NFS), which does the mount on behalf of CSI Manila.
CSI Cinder cannot handle NFS. CSI Manila expects CSI NFS in fwdNodePluginEndpoint, because that's where it will forward the mount request along with NFS-specific parameters.
Similarly for other for other file-systems. E.g. if you were using CephFS Manila shares, you'd have to point fwdNodePluginEndpoint to csi-cephfs

As you suggested, I changed the fwdNodePluginEndpoint in values.yaml
shareProtocols:
  - protocolSelector: NFS
    fwdNodePluginEndpoint:
      dir: /data/kubelet/plugins/nfs.manila.csi.openstack.org
      sockFile: csi.sock

now I got following error
$ kubectl logs csi-manila-openstack-manila-csi-controllerplugin-0 -c nfs-provisioner
I0214 11:03:43.460593       1 feature_gate.go:243] feature gates: &{map[]}
I0214 11:03:43.460683       1 csi-provisioner.go:105] Version: v1.5.0-0-g36b99945f
I0214 11:03:43.460699       1 csi-provisioner.go:119] Building kube configs for running in cluster...
I0214 11:03:43.468917       1 connection.go:153] Connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi-controllerplugin.sock
W0214 11:03:53.469122       1 connection.go:172] Still connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi-controllerplugin.sock
W0214 11:04:03.469090       1 connection.go:172] Still connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi-controllerplugin.sock
W0214 11:04:13.469109       1 connection.go:172] Still connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi-controllerplugin.sock
W0214 11:04:23.469110       1 connection.go:172] Still connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi-controllerplugin.sock

$ kubectl logs csi-manila-openstack-manila-csi-nodeplugin-96ggn -c nfs-nodeplugin

I0214 11:03:30.494835       1 driver.go:90] Driver: nfs.manila.csi.openstack.org version: 0.9.0 CSI spec version: 1.2.0
I0214 11:03:30.494882       1 driver.go:104] Operating on NFS shares
I0214 11:03:30.494890       1 driver.go:166] Enabling controller service capability: CREATE_DELETE_VOLUME
I0214 11:03:30.494898       1 driver.go:166] Enabling controller service capability: CREATE_DELETE_SNAPSHOT
I0214 11:03:30.494902       1 driver.go:185] Enabling volume access mode: MULTI_NODE_MULTI_WRITER
I0214 11:03:30.494906       1 driver.go:185] Enabling volume access mode: MULTI_NODE_SINGLE_WRITER
I0214 11:03:30.494908       1 driver.go:185] Enabling volume access mode: MULTI_NODE_READER_ONLY
I0214 11:03:30.494911       1 driver.go:185] Enabling volume access mode: SINGLE_NODE_WRITER
I0214 11:03:30.494914       1 driver.go:185] Enabling volume access mode: SINGLE_NODE_READER_ONLY
W0214 11:03:31.495176       1 builder.go:80] still connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi.sock
W0214 11:03:32.495086       1 builder.go:80] still connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi.sock
W0214 11:03:33.495070       1 builder.go:80] still connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi.sock
W0214 11:03:34.495112       1 builder.go:80] still connecting to unix:///data/kubelet/plugins/nfs.manila.csi.openstack.org/csi.sock

]$ kubectl logs csi-manila-openstack-manila-csi-nodeplugin-96ggn -c nfs-registrar
I0214 11:03:30.334133       1 main.go:110] Version: v1.2.0-0-g6ef000ae
I0214 11:03:30.334180       1 main.go:120] Attempting to open a gRPC connection with: "/csi/csi.sock"
I0214 11:03:30.334192       1 connection.go:151] Connecting to unix:///csi/csi.sock
W0214 11:03:40.334399       1 connection.go:170] Still connecting to unix:///csi/csi.sock
W0214 11:03:50.334377       1 connection.go:170] Still connecting to unix:///csi/csi.sock
W0214 11:04:00.334340       1 connection.go:170] Still connecting to unix:///csi/csi.sock
W0214 11:04:10.334374       1 connection.go:170] Still connecting to unix:///csi/csi.sock

@jenningsloy318 that's not correct.
1. Deploy CSI NFS
Try this minimal testing deployment of csi-nfs node plugin daemon set:
apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-nodeplugin
---
# This YAML file contains driver-registrar & csi driver nodeplugin API objects
# that are necessary to run CSI nodeplugin for nfs
kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: csi-nodeplugin-nfsplugin
spec:
  selector:
    matchLabels:
      app: csi-nodeplugin-nfsplugin
  template:
    metadata:
      labels:
        app: csi-nodeplugin-nfsplugin
    spec:
      serviceAccount: csi-nodeplugin
      hostNetwork: true
      containers:
        - name: node-driver-registrar
          image: quay.io/k8scsi/csi-node-driver-registrar:v1.0.2
          lifecycle:
            preStop:
              exec:
                command: ["/bin/sh", "-c", "rm -rf /registration/csi-nfsplugin /registration/csi-nfsplugin-reg.sock"]
          args:
            - --v=5
            - --csi-address=/plugin/csi.sock
            - --kubelet-registration-path=/var/lib/kubelet/plugins/csi-nfsplugin/csi.sock
          env:
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          volumeMounts:
            - name: plugin-dir
              mountPath: /plugin
            - name: registration-dir
              mountPath: /registration
        - name: nfs
          securityContext:
            privileged: true
            capabilities:
              add: ["SYS_ADMIN"]
            allowPrivilegeEscalation: true
          image: quay.io/k8scsi/nfsplugin:canary
          args:
            - "--nodeid=$(NODE_ID)"
            - "--endpoint=$(CSI_ENDPOINT)"
          env:
            - name: NODE_ID
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: CSI_ENDPOINT
              value: unix://plugin/csi.sock
          imagePullPolicy: "IfNotPresent"
          volumeMounts:
            - name: plugin-dir
              mountPath: /plugin
            - name: pods-mount-dir
              mountPath: /var/lib/kubelet/pods
              mountPropagation: "Bidirectional"
      volumes:
        - name: plugin-dir
          hostPath:
            path: /data/kubelet/plugins/csi-nfsplugin
            type: DirectoryOrCreate
        - name: pods-mount-dir
          hostPath:
            path: /data/kubelet/plugins/pods
            type: Directory
        - hostPath:
            path: /data/kubelet/plugins/plugins_registry
            type: Directory
          name: registration-dir

This is taken from https://github.com/kubernetes-csi/csi-driver-nfs/tree/master/deploy/kubernetes with very minor modifications.
It should give you a running DaemonSet of csi-nfs, with the .sock file in /data/kubelet/plugins/csi-nfsplugin/csi.sock.
2. Deploy CSI Manila
Now the shareProtocols section for csi-manila Helm deployment should look something like this:
shareProtocols:
  - protocolSelector: NFS
    fwdNodePluginEndpoint:
      dir: /data/kubelet/plugins/csi-nfsplugin/csi.sock
      sockFile: csi.sock

csi-manila should now be able to connect to csi-nfs. Sorry I don't have Manila NFS backend at hand so I can't test provisioning at the moment, but hopefully this should work for you.
You should output similar to this in the logs:
I0214 12:39:10.416661       6 connection.go:261] Probing CSI driver for readiness
I0214 12:39:10.416778       6 builder.go:39] [ID:1] FWD GRPC call: /csi.v1.Identity/Probe
I0214 12:39:10.416862       6 builder.go:40] [ID:1] FWD GRPC request: {}
I0214 12:39:10.425951       6 builder.go:46] [ID:1] FWD GRPC response: {}
I0214 12:39:10.426679       6 builder.go:39] [ID:2] FWD GRPC call: /csi.v1.Identity/GetPluginInfo
I0214 12:39:10.426753       6 builder.go:40] [ID:2] FWD GRPC request: {}
I0214 12:39:10.439129       6 builder.go:46] [ID:2] FWD GRPC response: {"name":"csi-nfsplugin","vendor_version":"1.0.0-rc2"}
I0214 12:39:10.439868       6 driver.go:254] proxying CSI driver csi-nfsplugin version 1.0.0-rc2

and I checked on the nodes, there is no csi.socket files under nfs.manila.csi.openstack.org or csi-nfsplugin

There is no socket file under nfs.manila.csi.openstack.org because csi-manila hasn't finished initialization. And no socket file under csi-nfsplugin because maybe you haven't deployed csi-nfs?

I deployed manila with the helm chart manila-csi-plugin in this repo and my values.yaml file . and is there any other requirements? as when I deployed cinder csi plugin, no need to configure fwdNodePluginEndpoint

Yes, you need to deploy csi-nfs plugin and then deploy csi-manila.
CSI Cinder is designed a bit differently, it doesn't rely on 3rd party CSI plugins for its node operations.

but if I fwdNodePluginEndpoint to /data/kubelet/plugins/cinder.csi.openstack.org , when I create a pvc then manila share will be created and bound. it is strange.

Yes, that's because the partner plugin (specified in fwdNodePluginEndpoint) is not involved during provisioning. You'd see errors once you'd try to mount the PVC.

@gman0  that's for you explanation, I just missed the most important port that deploy csi-driver-nfs daemonset as you piointed out early.
I think our document should be improved, set that csi-driver-nfs as requirement before deploying manila; As I thought that we don't need this step or using any other csi socket .
thanks for you kindly help.

@jenningsloy318

I think our document should be improved

Definitely!
I'm glad it's working for you. If you don't have any further questions you may close this issue.

@jenningsloy318 Just FYI, k8scloudprovider/manila-csi-plugin:latest is now available from Docker hub: https://hub.docker.com/r/k8scloudprovider/manila-csi-plugin/tags


Fix invoke script using dev DB in non-dev

TODO: Figure out how to resolve the circular import error

Issue: Login manager requires app as an argument.

nearly there -- currently facing google access error.

>>> app.url_map
Map([<Rule '/static/<filename>' (GET, HEAD, OPTIONS) -> static>,
 <Rule '/bootstrap/static/<filename>' (GET, HEAD, OPTIONS) -> bootstrap.static>,
 <Rule '/' (GET, HEAD, OPTIONS) -> root.root>,
 <Rule '/surveys' (GET, HEAD, OPTIONS) -> surveys.view_surveys>,
 <Rule '/new' (GET, POST, HEAD, OPTIONS) -> survey.new_survey>,
 <Rule '/<survey_id>' (GET, HEAD, OPTIONS) -> survey.view_survey>,
 <Rule '/users' (GET, HEAD, OPTIONS) -> users.view_users>,
 <Rule '/login' (GET, HEAD, OPTIONS) -> login.login>,
 <Rule '/google_oauth/google' (GET, HEAD, OPTIONS) -> google.login>,
 <Rule '/google_oauth/google/authorized' (GET, HEAD, OPTIONS) -> google.authorized>,
 <Rule '/response/<survey_id>' (GET, HEAD, OPTIONS) -> response.view_response>])


Take into account filesystem overhead when defining required disk space

The current estimation of required disk space is not fail safe, it only takes into account the exact sizes of the downloaded zip and the extracted binary image. But it doesn't consider filesystem overhead or double caching issues on the wide range of different Linux systems using InputStreamHelper.
Adding 30 MiB overhead margin to the required disk space won't hurt and possibly fixes https://github.com/emilsvennesson/script.module.inputstreamhelper/issues/281

10% of 2GB is quite a lot. I would add 10MB or 20MB for additional writes/logging.

I changed it to 5%.
We also didn't take into account the EULA extraction from the x86 zip -> + 5 MB
And there is definitely a significant filesystem overhead on some systems:
https://rwmj.wordpress.com/2009/11/08/filesystem-metadata-overhead/
https://unix.stackexchange.com/questions/278647/overhead-of-using-loop-mounted-images-under-linux
So, at least 5%

We also didn't take into account the EULA extraction from the x86 zip -> + 5 MB

Good catch!

And there is definitely a significant filesystem overhead on some systems:

Sure there is overhead, in caching and CPU, but not additional file system use. Sure there may be additional paging going to disk, but that is already allocated space on disk (swap), not file system space.
Even 5% is close to 100MB additional disk space, which on smaller devices may be a dealbreaker. (If 100MB is the difference between a go and a no go, I agree that's not a good situation to be in)
I don't think it should be relative to compressed image + image size.

I can't test on RPi, but it would help if someone could measure the overhead.

Take 20MB, or if you want to be safe maybe 30MB. It's a timespan of about 3 to 4 minutes of logging, plus the EULA.

Okay, 30 MiB it is.

Just FYI, the link you shared (https://rwmj.wordpress.com/2009/11/08/filesystem-metadata-overhead/) about metadata overhead is an interesting one, but does not apply to a read-only image. If you mount an empty image and would write to a filesystem there is indeed file system metadata and a journal to take into account (depending on the filesystem used).
But in our case the image is fixed and the size is already allocated. What's more the uncompressed image may use more disk space than is actually used because even unused file system space will be allocated even when unused. (An empty 1TB image compresses well, but still is 1TB in size on disk)

I will perform some tests later tonight so we have real metrics to go by. While not 100% accurate it may give some insights on the grand order.

Here it goes, first in kiB, then in MiB.
Original filesystem
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  14479195  15181796  49% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     14139     14825  49% /storage

After removing Widevine CDM library (no change)
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  14479196  15181795  49% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     14139     14825  49% /storage

After downloading EULA (-11 MiB ?!?)
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  14467541  15193450  49% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     14128     14837  49% /storage

It reports that I need 3.1GiB of free disk space to continue.
After downloading the ChromeOS recovery image (+972MiB)
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  15462571  14198420  52% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     15100     13865  52% /storage

After extraction of the image (+2220MiB)
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  17735860  11925131  60% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     17320     11646  60% /storage

When finished (-3188MiB)
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  14471159  15189832  49% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     14132     14834  49% /storage

Notable things:

I dumped sizes every 1 second, which is not very good (1 sec can make a difference of about 20MiB)
After downloading the EULA, I saved 11MiB of disk space (probably some clean up happened?)
I ended up with 7MiB less disk space use after the complete process
In total it needed 2220MiB + 972MiB = 3192MiB or 2542410kiB + 995030kiB = 3537440kiB
According to the image used (BOB) we need<PHONE_NUMBER>b +<PHONE_NUMBER>b =<PHONE_NUMBER>b = 3294122kiB = 3216MiB = 3.141GiB

So the granularity is a problem. I would need to add OK-dialogs to get proper numbers, so I can sync to disk after download and after compress before removal.

Let's do this one more time, now wait ok-dialogs during each step.
Original filesystem
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     14132     14834  49% /storage
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  14471163  15189828  49% /storage

After downloading EULA (+4MiB)
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  14475334  15185657  49% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     14136     14830  49% /storage

It reports that I need 3.1GiB of free disk space to continue.
After downloading the ChromeOS recovery image (+970MiB)
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  15468700  14192291  52% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     15106     13860  52% /storage

After uncompressing the image (+2214MiB)
kodi01:~ # df /dev/mmcblk0p2; df -m /dev/mmcblk0p2
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  17735861  11925130  60% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     17320     11646  60% /storage

On disk this is stored in temp/
kodi01:~ # ls -l .kodi/userdata/addon_data/script.module.inputstreamhelper/temp/; \
> ls -lh .kodi/userdata/addon_data/script.module.inputstreamhelper/temp/
total 3256323
-rw-r--r--    1 root     root       4270381 Apr 15 00:41 4.10.1582.2-mac-x64.zip
-rw-r--r--    1 root     root<PHONE_NUMBER> Apr 15 00:48 chromeos_12739.111.1_bob_recovery_stable-channel_mp.bin
-rw-r--r--    1 root     root<PHONE_NUMBER> Apr 15 00:43 chromeos_12739.111.1_bob_recovery_stable-channel_mp.bin.zip
total 3G
-rw-r--r--    1 root     root        4.1M Apr 15 00:41 4.10.1582.2-mac-x64.zip
-rw-r--r--    1 root     root        2.2G Apr 15 00:48 chromeos_12739.111.1_bob_recovery_stable-channel_mp.bin
-rw-r--r--    1 root     root      970.1M Apr 15 00:43 chromeos_12739.111.1_bob_recovery_stable-channel_mp.bin.zip

When finished (-3188MiB)
kodi01:~ # df /dev/mmcblk0p2; df -m /dev/mmcblk0p2
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2        29665087  14471163  15189828  49% /storage
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/mmcblk0p2           28970     14132     14834  49% /storage

Notable things:

The EULA is downloaded before the calculation so it does not affect the comparison !
In total it needed 2214MiB + 970MiB = 3184MiB = 3.109GiB
According to the image (BOB) used we need<PHONE_NUMBER>b +<PHONE_NUMBER>b =<PHONE_NUMBER>b = 3294122kiB = 3216MiB = 3.141GiB
Looking at the files on disk compared to the reported sizes from recovery.conf, there is a 40MB discrepancy already. We request to have 40MiB more than we actually need.

The ZIP file is reported to be<PHONE_NUMBER> in size, but is actually only<PHONE_NUMBER> on disk (a difference of 80.8MiB)
The image is reported to be<PHONE_NUMBER> in size, but is actually<PHONE_NUMBER> on disk (a difference of 39.820MiB)
The difference in results with the numbers above are due to rounding errors (as I calculated from the MiB numbers)

So if we add another 30MiB we have in effect a 70MiB buffer...

Problems

The progress dialog shows 2x the running time, that's a bug
After doing this manual update, the info dialog reports I last updated on 2020-04-04, while it should state 2020-04-15 (just now)!

And I don't see this date reflected on disk anywhere:

kodi01:~ # ls -lh .kodi/userdata/addon_data/script.module.inputstreamhelper/backup/* .kodi/cdm/
.kodi/cdm/:
-rwxr--r--    2 root     root        7.4M Apr 14 23:47 libwidevinecdm.so
-rw-r--r--    2 root     root      163.5K Apr 14 23:47 recovery.conf.json
.kodi/userdata/addon_data/script.module.inputstreamhelper/backup/12607.58.0:
-rwxr--r--    1 root     root        7.4M Jan 26 00:05 libwidevinecdm.so
-rw-r--r--    1 root     root      157.5K Jan 26 00:05 recovery.conf.json
.kodi/userdata/addon_data/script.module.inputstreamhelper/backup/12607.82.0:
-rwxr--r--    1 root     root        7.4M Jan 27 21:33 libwidevinecdm.so
-rw-r--r--    1 root     root      157.6K Jan 27 21:33 recovery.conf.json
.kodi/userdata/addon_data/script.module.inputstreamhelper/backup/12739.111.1:
-rwxr--r--    2 root     root        7.4M Apr 14 23:47 libwidevinecdm.so
-rw-r--r--    2 root     root      163.5K Apr 14 23:47 recovery.conf.json

Even when running it a second time I don't see the last_updated date change.
The Cancel button during extraction does not work. Maybe that is a good thing ;-)
The Restore dialog should give an indication that the version listed is the ChromeOS version

The trend is clear, newer images now need 3.1GiB, whereas before it was 2.7GiB and before that 2.1GiB :-(

If you manually install via the settings page, you have to hit OK and reopen it before looking at the info dialog, since last_update (and any other setting change) is only saved then.

@horstle Sure but according to file timestamps there was nothing updated on 2020-04-04. There is no backup with that date either. The last_update value reflects when it was last checked, not when it was last updated, which is what we report. So if we want to keep both values, we probably should keep both values.

The progress dialog shows 2x the running time, that's a bug

I fixed this in https://github.com/emilsvennesson/script.module.inputstreamhelper/pull/316
@dagwieers You can close this, you have proven it's very unlikely this PR fixes something.

Ok. Although I don't object to adding a small buffer. As I don't know if the discrepancy is always to our advantage as it is here for BOB. The next time another image is selected we may be in a disadvantage and this may have been the cause of past issue??
I think it is strange that the reported sizes in recovery.conf do not correspond to the actual files that are downloaded.

What's more the settings on disk has not been updated either:
...
It still reflects 2020-04-04

You probably hit Cancel instead of OK, when closing the settings page, in which case it won't save the setting. For me it works if I press OK and reopen.
Anyways adding a last_check and differentiating between the two would make sense.

You probably hit Cancel instead of OK, when closing the settings page, in which case it won't save the setting. For me it works if I press OK and reopen.

No I did not, and can confirm this on two different systems. Also hitting OK shouldn't be required because it is not being updated through the settings anyway, it is a script that is being called.
I don't think last_checked is updated when manually updating Widevine. And may be updated, even when no Widevine was installed. So a mismatch between what it is used for, and how it is being used for reporting.

No I did not, and can confirm this on two different systems.

That's really strange. I cannot reproduce this.

Also hitting OK shouldn't be required because it is not being updated through the settings anyway, it is a script that is being called.

Well, the script is called through the settings, so in a way... but this isn't something you should discuss with me anyways, I can't change it.

I don't think last_checked is updated when manually updating Widevine.

It's called last_update and it wasn't updated until #290 was merged.

And may be updated, even when no Widevine was installed. So a mismatch between what it is used for, and how it is being used for reporting.

The inconsistent usage is why I proposed:

Anyways adding a last_check and differentiating between the two would make sense.


fix python3 compatibility

closes #66, #118, #137, #140, #192, #206
^ This seems to be the most pressing issue to resolve. I'm hoping to fix python3 compatibility once and for all in this branch. Looking for feedback from anyone who knows more about this stuff.
Notes on the changes in here:

os.getenv and related functions return strings, not unicode (in python 2). This will cause an issue in the draw function if you do not decode the string into utf-8 and there are non-ascii characters in the string, because concatenating a unicode object and a string causes Python to try to implicitly convert the ASCII to unicode, but can't convert the non-ascii character. Here is some that demos this:

>>> x = "powerline-shell-\xe2\x9c\x8e" # bytes to make the pencil utf-8 character
>>> x + u""
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 16: ordinal not in range(128)

In Python 3, everything is unicode. There is also no str.decode function.
However, in Python 3, the subprocess functions return bytes instead of unicode. Thus we must always decode the bytes into unicode. This works just fine though since in Python 2, subprocess returns strings which must also be decoded into unicode.

Tests:

[x] cd into directory with non-ascii character in its name (py2)
[x] cd into directory with non-ascii character in its name (py3)
[x] git branch has unicode char (py2)
[x] git branch has unicode char (py3)
[x] hg basic test (py2)
[x] hg basic test (py3)
[x] basic jobs test
[x] basic ruby/node version tests

Did some pretty extensive testing. Only thing I left out was testing the SVN changes. That I'm just hoping I'm not missing something.

Just gave SVN a quick run and found no issues. I think this change is ready to go.

Hello, this seems not to work for me with pyenv.
pyenv global 3.4.3 shows no prompt at all, all the segments disappear but no error is showed.
If I pyenv global 2.7.6 then things go back to normal


Upon saving a new indicator: Update success message for easy return to program page

Story
After I add a new indicator to a program, I want an easy way to return to the program page to check out my new indicator and continue on to any other program-related work.
Affects
Add indicator page, upon clicking the "Save changes" button.
Acceptance criteria

[ ] 1. I no longer see the temporary success banner between the "Indicator setup" title and the name of the indicator.
[ ] 2. Instead, I see a PNotify success message in the lower left corner (where my attention is, having just clicked "Save changes" that includes a link to the program page.
[ ] 3. The notification is NOT temporary; it stays put. It can be manually dismissed in any of the following ways:

[ ] 3.1. Click the X, which closes the notification and keeps you on the page if you want to keep editing the indicator there.
[ ] 3.2. Click anywhere outside the notification area -- same deal, closes the notification, keeps you on the page.
[ ] 3.3. Click "View your new indicator on the program page." This closes the notification and takes you to the program page.

Housekeeping (where applicable)

[ ] Unit test/s written
[ ] i18n strings included
[ ] Is accessible to people using screen readers

@jennyhavah I think this should be relatively easy to implement.
For the modal version, we can try it both ways and see which way looks/feels better.

Addressed in #884 as part of Mangosteen.


Change mac script to run from proj directory

Current dependencies on/for this PR:

main

PR #197   👈

This stack of pull requests is managed by Graphite.


electron sandbox in Linux, causes issues

Describe the bug
When you run the app it immediately closes. When you run it in a terminal software it shows this.
 ./Downloads/Outline-Client.AppImage
loading web app from file:///tmp/.mount_OutlinDBjzCM/resources/app.asar/www/index_electron.html?appName=Outline
(node:2664) electron: The default of contextIsolation is deprecated and will be changing from false to true in a future release of Electron.  See https://github.com/electron/electron/issues/23506 for more information
Could not load active tunnel:  [Error: ENOENT: no such file or directory, open '/home/username/.config/Outline/connection_store'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/home/username/.config/Outline/connection_store'
}
Checking for update
[2664:1022/080820.389859:FATAL:gpu_data_manager_impl_private.cc(445)] GPU process isn't usable. Goodbye.

Workaround
run the app with a terminal with --no-sandbox
./Outline-Client.AppImage --no-sandbox
To Reproduce
just open the appimage file it closes.
Expected behavior
App should launch with a double click.
Client System (please complete the following information):

Outline Client Version: 1.7.0 (in the about menu.)
Your Operating System: Arch Linux
I also have a NVIDIA gpu

@Master811129
Please check https://github.com/Jigsaw-Code/outline-client/issues/1577#issuecomment-1567407582


Can no longer place an infinite quantity of items inside a microwave

About the PR
Fixes issue #23462
Microwaves now have an item capacity of 10 items. No more items can be inserted once this item limit has been reached.
10 items is more than enough for any recipe. The largest recipe is RecipeStewSoup, which needs 7 solids + 1 container for reagents (water).
Why / Balance
Crates and lockers have finite inventory capacity. Microwaves, which shouldn't be used for item storage, and infact should be quite bad at it, can store an unlimited quantity of items. This does not make sense.
Technical details
MicrowaveComponent has new integer field called Capacity.
MicrowaveSystem will compare the number of items already inside to the capacity, and will not insert any more unless the number of items inside is less than the capacity.
KitchenMicrowave prototype has this field set to 10.
Media
https://github.com/space-wizards/space-station-14/assets/66873282/2c9ca5de-82c7-4972-a686-d6ec3c44ac26

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes
MicrowaveComponent has new integer field Capacity.

Changelog
:cl:

fix: Due to budget constraints, microwaves can no longer contain an infinite number of items. Now the limit is 10.

And what's the cook time on that stew?

And what's the cook time on that stew?

10

10 capacity is perfectly reasonable

TARDIS microwave as an upgrade option?

bluespace microwave, syndicate potted plant but drag yourself in to teleport

You've essentially removed the ability to cook three of those stews in one batch by adding three copies of the ingredients and setting the cook time to 30 seconds.

Although an interesting mechanic, I do see it as an exploit ngl.

You've essentially removed the ability to cook three of those stews in one batch by adding three copies of the ingredients and setting the cook time to 30 seconds.

Although an interesting mechanic, I do see it as an exploit ngl.

It's literally programmed to do exactly this on purpose? How does that make it an exploit.

You've essentially removed the ability to cook three of those stews in one batch by adding three copies of the ingredients and setting the cook time to 30 seconds.

Although an interesting mechanic, I do see it as an exploit ngl.

how is it an exploit to cook more of a food that can be scaled up to cook more in one batch irl

The exploit is sticking a million things on a microwave. Its reasonable to stick a few items but being unable to stick enough ingredients to make 3 instances of a recipe is a completely reasonable limitation


UI changes

add classes for add item block elements to simplify styling
make add and cancel buttons smaller and same height with inputs

Great, thanks!


Sync SLO alerts with upstream kubernetes-mixin

Sync our downstream SLO alert with the ones from upstream https://github.com/kubernetes-monitoring/kubernetes-mixin to make sure we benefit from their improvements. For instance, this could have prevented a few bugs that were found in 4.10.

/cc @tkashem

The runbook  for the KubeAPIErrorBudgetBurn  might need to be updated if we are now in sync with upstream.

/remove-lifecycle stale

/remove-lifecycle stale

Closing, I will open another PR to introduce that change again.


FR: string representation

[REQUIRED] Step 2: Describe your environment

Operating System version: Linux
Firebase SDK version: 3.0.0
Library version: 3.0.0
Firebase Product: cloud messages (auth, database, storage, etc)

[REQUIRED] Step 3: Describe the problem
Steps to reproduce:
It would be great to have a string representation of a object to have a quick preview of his contents because this is not really helpful.
> str(message)
'<firebase_admin._messaging_utils.Message object at 0x11bdcda10>'

I think it would be great if at least the str function could present the dict variant of e.g the message Object

class Message:
    def __str__(self):
        return self.__dict__

> str(Message)
{'data': None, 'notification': <firebase_admin._messaging_utils.Notification object at 0x11bdad050>, 'android': None, 'webpush': None, 'apns': None, 'fcm_options': None, 'token': None, 'topic': None, 'condition': None}

Sounds like a good idea. There's a custom MessageEncoder class that knows how to serialize all these objects. Perhaps we can use that to implement this.

Would be quite nice if there some at least some human readable for the str version. :)
Thanks for the hint with MessageEncoder

MessageEncoder does the job for the moment.
Is there also a way that is not a function in a protected module?

@Azd325 I wasn't suggesting you directly use MessageEncoder. Rather implement the functionality as a proper public API, based on MessageEncoder.

__str__() function is now implemented in Message class with the changes included in #350


feat: support node v20

@lbenie done!

:tada: This PR is included in version 1.3.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


⚠ DesiTorrents has degraded performance

In e102e74, DesiTorrents ($DST) experienced degraded performance:

HTTP code: 200
Response time: 1609 ms

Resolved: DesiTorrents performance has improved in b895e7c.


Create Bubble-cpp

What type of PR is this? (check all applicable)

[ ] 🚀 Added Name
[ ] ✨ Feature
[ ] 🌟 stared the repo
[ ] 🐛 Grammatical Error
[ ] 📝 Documentation Update
[ ] 🚩 Other

Description
Add Link of GitHub Profile

check this


add tensorflow-gpu to rosdep

I want to add tensorflow-gpu package to rosdep.
package url:tensorflow-gpu
I want to use tensorflow with GPU in rospy.

Thanks @hakuturu583 for providing the link and description.
Travis is failing because the entry is not in alphabetical order in the list


Some issue about the TOC

The shortcut Ctrl + N is already in use. Can I change it?
Can the navigation stay open or remember my last selection?

There isn't a direct method of changing bindings within the application.
Could you clarify the question?

More details for the 2nd, I would like the TOC to display automatically, while I enter the markdown view, the TOC auto displays. There are two possible solutions:

By an option value.

Remember the TOC status. If I open the TOC in markdown view and then exit, it should reappear when I return to markdown view. Vise versa.

I see: in that case, no, at the moment, there isn't a facility to make the open/closed state of the navigation pane "sticky".


Update README.md

The correct command for activating the environment in windows is 'myview\scripts\activate'

Fixes issue #168

@MihirHundiwala
Thank you for the pull request! Merged


Full control over awaitable mock

When writing asyncio tests it is to control when and what awaitable mock should return.
Here is an example to give an idea:
class MagicCoro(unittest.mock.Mock):
    """
    >>> m = unittest.mock.MagicMock()
    >>> m.coro = MagicCoro(return_value=42)
    >>> assert await m.coro() == 42
    >>> assert m.exit.is_set()
    """
    def __init__(self, run_event=None, return_value=None):
        async def coro(*args, **kwargs):
            self.enter.set()

            if self.run:
                await self.run.wait()

            self.exit.set()

            return self.value

        super().__init__(wraps=coro)

        self.enter = asyncio.Event()
        self.run = run_event
        self.exit = asyncio.Event()
        self.value = return_value

    def _get_child_mock(self, **kw):
        return unittest.mock.MagicMock(**kw)

This allows to:

provide a return value
provide an event to wait for allowing user to control completion order
assert whether coroutine was entered or exited

This is something that can be added to the helpers module:
http://asynctest.readthedocs.io/en/latest/asynctest.helpers.html

@Martiusweb I was thinking about adding an extra attribute to CoroutineMock to "await" its call.

https://docs.python.org/3/reference/datamodel.html?#awaitable-objects
i think you forgot await function.


feat: add import order rules

In light of this RFC two rules were added to our config:

import/order: displays an error when imports are not in the following order: builtin, external, parent, sibling, index and groups are not separated with a new line.

import/first: displays an error when imports are made in the module's body.

As for import/first, since we already happen to have import/order, which already orders the module's import, this could be turned off in certain cases but maybe it's best to ensure all imports are made at the top of the file.

@ritazoliveira I would also consider enabling https://github.com/benmosher/eslint-plugin-import/blob/master/docs/rules/newline-after-import.md.

Removed the mandatory new line after each import group and also added the newline-after-import rule.

This is a breaking change. @ritazoliveira next time don’t forget to sinal it in the commit message or body.

Fixed the commit to include BREAKING CHANGE in the body.


Chromecast support?

Might be an idea for a feature.
E.g. the Fennec (Firefox for Android) implements this.

I thought about that. Is there a fDroid compatible, up to date open source solution for chromecast support?

Maybe have a look at https://bugzilla.mozilla.org/show_bug.cgi?id=901803. It seems Google provides an SDK.

example code is here https://github.com/googlecast/CastVideos-android

Thank you!
I've checked with fDroid developers: The official SDK is not allowed inside fDroid and there are no Open Source alternatives at the moment. So this feature has to wait until Zapp is released to the Play Store. Then we can use Gradle to build different flavors for fDroid (without chromecast) and Play Store (with chromecast).

At least I see that NewPipe (a YouTube app, also on FDroid) has Chromecast support (at least the icon is there sometimes) - you may look at how they do it.

They support streaming to Kodi but not Chromecast, because they have the same problem. See TeamNewPipe/NewPipe#351

Google rejected the app (as will - most likely - amazon), so I'll have to use another store for this feature.

What are the reasons provided by google to reject the app from the playstore ?

See #39 - I pasted the rejection mail over there. The essence is "it violates the unauthorized use of copyrighted content policy".

Under this circumstances chromecast support is not going to happen in the near future.


Attempting a new connection right after the 'OnDisconnected' event fired leads to an error

Describe the bug
When attempting to connect to a new conference from inside a method bound to the OnDisconnected event, the SDK returns an error: LogDolbyIO: Error: Caught dolbyio::comms::exception: Invalid session state: 2, required: 0 (conference status: destroyed, line: 383).
Adding a 0.5 second delay after the OnDisconnected event right before calling the Connect method again stops the error from being thrown and things work as expected.
This happens regardless of the implementation of the client logic (C++ or Blueprint).
Tested and reproduced on my end in Unreal Engine 5.1.1 using both the 1.1.0-beta.9 and 1.0.2 releases.
Expected Behavior
When OnDisconnected fires, the underlying SDK is fully ready for a new connection. Therefore, calling Connect from inside a function bound to OnDisconnected should work without throwing the error.
Minimalistic code (recommended)
A sample UE 5.1.1 project repository demonstrating the issue can be found here.
To reproduce the issue without cloning the project:

connect to a "main-conference" Dolby.io Conference using the Dolby.io Connect node;
to the previous node, bind a Delay + a Dolby.io Disconnect node to OnConnected, so that after 10 seconds you disconnect from the "main-conference";
to the previous Dolby.io Disconnect node, bind a new Dolby.io Connect node (that connects to a "sub-conference") directly to the OnDisconnected pin;

This will lead to the aforementioned error being thrown. Adding a delay between these two nodes (it can be a small 0.5s delay) fixes the issue.
Specifications

OS: Windows 11 Pro 22H2
Versions:

Unreal Engine 5.1.1;
Dolby.io Virtual Worlds 1.1.0-beta.9 or  1.0.2 (reproduced this behavior with both);

Engine log
Here is a log that includes the error/conference IDs for back-end checks.
PIE: Play in editor total start time 0.086 seconds.
LogDolbyIO: Initializing with token: <token here>
LogDolbyIO: Initialized
LogDolbyIO: Audio device changed for direction: output to device - Speakers (Intel® Smart Sound Technology for MIPI SoundWire® Audio), direction: output, native_id: {<IP_ADDRESS>00000}.{02dfa4f7-dedb-4a4e-9546-689132abf63f}
LogDolbyIO: Connecting to conference main-conference with user name "dev-debug" (EDolbyIOConnectionMode::Active, EDolbyIOSpatialAudioStyle::Shared)
LogDolbyIO: Conference status: creating
LogDolbyIO: Conference status: created
LogDolbyIO: Conference status: joining
LogDolbyIO: Conference status: joined
LogDolbyIO: Connected to conference ID bf7ce84f-bc97-4414-9fc0-d9cd82f1b7f4 with user ID 5e8027f1-991e-3d93-887b-cb9931b89c8c
LogBlueprintUserMessages: [BP_Dolby_Debug_Actor_C_1] Connected to the main conference!
LogDolbyIO: Disconnecting
LogDolbyIO: Conference status: leaving
LogDolbyIO: Conference status: left
LogDolbyIO: Conference status: destroyed
LogDolbyIO: Connecting to conference sub-conference with user name "dev-debug" (EDolbyIOConnectionMode::Active, EDolbyIOSpatialAudioStyle::Shared)
LogDolbyIO: Error: Caught dolbyio::comms::exception: Invalid session state: 2, required: 0 (conference status: destroyed, line: 402)
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogWorld: BeginTearingDown for /Game/UEDPIE_0_L_Dolby_Debug
LogWorld: UWorld::CleanupWorld for L_Dolby_Debug, bSessionEnded=true, bCleanupResources=true
LogSlate: InvalidateAllWidgets triggered.  All widgets were invalidated
LogPlayLevel: Display: Shutting down PIE online subsystems

I can confirm this issue is easy to reproduce. This is because the Connect function first tries to open a user session (using the user name, external ID and avatar URL) and only then actually connects to a conference. When you disconnect and connect immediately it is possible (almost certain) that you will disconnect from the conference, but the session will still be open for a short while. This requires fixing some logic in when On Disconnected is triggered, so that is only fired when the session is closed or a fatal conference error occurs.

@Adam1-3 FYI

@Adam1-3 We may also want to consider moving session.open out of Connect and session.close out of Disconnect somehow. For example if a user wants to quickly switch conferences by chaining Disconnect(old)->Connect(new), it is pointless to redo the session. Without breaking compatibility, we could for example save the user data and only open a new session if it changes.

@andreibazi Fixed in 1.1.0 release.

@kubaau Thank you very much for the fast fix!


fix cellery status for composites

Purpose

Fix cellery status command to work for composites.

Potential fix for https://github.com/wso2-cellery/sdk/issues/807.

Codecov Report

Merging #811 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #811   +/-   ##
=======================================
  Coverage   53.45%   53.45%
=======================================
  Files          56       56
  Lines        2215     2215
  Branches      218      218
=======================================
  Hits         1184     1184
  Misses        980      980
  Partials       51       51

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e4434f0...aead4ce. Read the comment docs.


Client extensions

Relocated client extensions from a10-neutron-lbaas/a10-openstack

   File "/usr/local/src/a10-neutron-lbaas/a10_neutron_lbaas/neutron_ext/common/constants.py", line 17, in <module>
     from a10_neutronclient.resources import a10_device_instance
 ImportError: cannot import name a10_device_instance

  File "/usr/local/src/a10-neutronclient/a10_neutronclient/a10_device_instance.py", line 18, in <module>
    from a10_neutronclient.resources import a10_appliance
ImportError: cannot import name a10_appliance

Closing because github has fallen on it's head?


Make IGListAdapter.updatingDelegate Public, Read-Only

Hey IGListKit folks! Would you be willing to expose this, so that other objects that are given a list adapter can inspect its updater configuration?
Pull request checklist

[x] All tests pass. Demo project builds and runs.
[x] I added tests, an experiment, or detailed why my change isn't tested.
[x] I added an entry to the CHANGELOG.md for any breaking changes, enhancements, or bug fixes.
[x] I have reviewed the contributing guide

Coverage remained the same at 97.978% when pulling 09f36c0687135de9750bd268789b2478608123cb on Adlai-Holler:expose-list-adapter-updater into 0de9632c0de9e6aa5ba08ddd3f6e96369147195a on Instagram:master.

I'm fine with this change. What's your use case?

/cc: @rnystrom ?

I'm fine with this change. What's your use case?

/cc: @rnystrom ?

I'd like to be able to assert that a list adapter passed into an API does not use IGListReloadDataUpdater and that the updater is configured with allowsBackgroundReloading=NO. I could modify the API to require the user to keep track of the updater and pass it in alongside the list adapter, but it seems like unnecessary hassle.

I'd like to be able to assert that a list adapter passed into an API does not use IGListReloadDataUpdater and that the updater is configured with allowsBackgroundReloading=NO. I could modify the API to require the user to keep track of the updater and pass it in alongside the list adapter, but it seems like unnecessary hassle.

Hmm... but you have to initialize the adapter with an id<IGListUpdatingDelegate> couldn't you enforce this then?

Since the component that receives the adapter is not the one who initializes it, the adapter's owner would have to keep track of the updater separately and provide them both into this API. So @property ASCollectionNode.listAdapter would have to become -[ASCollectionNode setListAdapter:updater:].

@rnystrom - want to just import this and forward-fix internally?

@rnystrom The internal (readwrite) version of the property unfortunately needs to stay, because the tests use it:

Cool. I've renamed updatingDelegate -> updater and pushed ✨

Coverage remained the same at 97.978% when pulling 4755f7178aa98f8fe10964a07c1d8816de8ede48 on Adlai-Holler:expose-list-adapter-updater into 0de9632c0de9e6aa5ba08ddd3f6e96369147195a on Instagram:master.

Coverage remained the same at 97.978% when pulling 4755f7178aa98f8fe10964a07c1d8816de8ede48 on Adlai-Holler:expose-list-adapter-updater into 0de9632c0de9e6aa5ba08ddd3f6e96369147195a on Instagram:master.

Coverage remained the same at 97.979% when pulling bf24f583978fbc5e660b4039a054ec277bb721a9 on Adlai-Holler:expose-list-adapter-updater into 71ce9902ca7241bdfef7505429bbe114e5e4934f on Instagram:master.


Inference for low gpu and less number of points

Hello
You have done a great work I really appriciate it !
I have been trying to run the model to track some specific points on videos but I could not figure out how to do that exactly. I tried the format
model({"video": video[None], "query_points": torch.Tensor([[[1, 15, 51]]]).cuda()},
but GPU ran out of memory. Am I doing it right or is there any other method to do this ?

Hi @zeynytu, our method is meant to track all pixels in a frame together. If you want to track only a few points, you have two options. Either (1) using point tracking directly --model pt, or (2) using our method to track densely and then deduce the tracks for the points you are interested in --model dot. The inference mode in both cases is "tracks_for_queries" as is done here:
https://github.com/16lemoing/dot/blob/cdee971fb0615fe3bf7b6fd19d856ea572327ec1/test_tap.py#L35
Please provide more information on your GPU setup, video length and spatial resolution if you need further assistance on  the OOM errors.

Actually, I have a long video, and I want to track around 50 points in specific coordinates. The length of the video is not a big deal; I can trim the video into separate parts. The GPU is nvidia 3060 ti with 8 GB vRAM.

Hi @zeynytu, our method is meant to track all pixels in a frame together. If you want to track only a few points, you have two options. Either (1) using point tracking directly --model pt, or (2) using our method to track densely and then deduce the tracks for the points you are interested in --model dot. The inference mode in both cases is "tracks_for_queries" as is done here:
https://github.com/16lemoing/dot/blob/cdee971fb0615fe3bf7b6fd19d856ea572327ec1/test_tap.py#L35
Please provide more information on your GPU setup, video length and spatial resolution if you need further assistance on the OOM errors.

So, in the function 'interpolate', S=H*W?

Hi @Billy-ZTB, S is the number of initial tracks (which are then densified) so in general  S<<H*W.

Hi @Billy-ZTB, S is the number of initial tracks (which are then densified) so in general S<<H*W.

Thanks!


Merge satellite scheduler and fix QUBOSolver convergence on CPU

Issue Number: #213
Objective of pull request: Merge satellite scheduler and fix QUBOSolver convergence on CPU
Pull request checklist
Your PR fulfills the following requirements:

[X] Issue created that explains the change and why it's needed
[X] Tests are part of the PR (for bug fixes / features)
[X] Docs reviewed and added / updated if needed (for bug fixes / features)
[X] PR conforms to Coding Conventions
[X] PR applys BSD 3-clause or LGPL2.1+ Licenses to all code files
[X] Lint (pyb) passes locally
[X] Build tests (pyb -E unit) or (python -m unittest) passes locally

Pull request type
Please check your PR type:

[X] Bugfix
[X] Feature

What is the current behavior?
What is the new behavior?
Does this introduce a breaking change?

[X] No

Thanks @tim-shea for the contribution, looks good overall!
I left a couple of minor comments.
What about moving satellite_scheduler.py in the utils/generators folder?

I'm open to doing that too. Not sure if it's very useful as a general piece of code, but perhaps with some refactoring.


[MLIR][LINALG] Legalizing tensor-type linalg operations to memref-type ones.

In this PR, we used Buffer Assignment to convert the type of Linalg-GenericOp operands and results from Tensor-type to Memref-type.

This looks great, could you please send it as a patch to MLIR core, this is where it should live so we can all use it.
Thanks!

@dfki-ehna Can you please check @silvasean's comments and keep us posted. Thanks!

@dfki-ehna Any update on this PR? and please resolve conflicts.  Thanks!

We ported this PR to MLIR and we are closing this.


Added a decorator

I've added a decorator for conveniently defining jobs from functions.

Can you write pls how to use this decorator?
Thanks

Thanks RHagenaars for creating this PR. To tell you the truth I'm on the fence whether this should be a part of schedule itself or an FAQ item.
A couple of thoughts here:

Please use functools.wraps to ensure the metadata of the wrapped function is copied over.
The code example doesn't import repeat()
I don't think the code example will render correctly in the HTML docs, this would also need fixing.

@Naramsim the usage would be as follows:
@repeat(every(10).minutes)
def f():
  print('This function is repeated.')

@dbader thanks for taking time to consider it. About your thoughts:

Please use functools.wraps to ensure the metadata of the wrapped function is copied over.

That's not necessary in this case. If you look closely, you'll see that the actual decorated function is returned and not some wrapper.

The code example doesn't import repeat()

Good catch! I've added that import.

I don't think the code example will render correctly in the HTML docs, this would also need fixing.

I'm not sure how to fix that. Could you elaborate some more on what exactly needs to be fixed? I took the docstring on top of the file for an example.

I'm on the fence whether this should be a part of schedule itself or an FAQ item

That's up to you of course. I personally like decorators for things like this, because it allows you to put coupled statements - the function and the schedule - close to each other. I think that in most cases, the code is more clear when these are not separated.

Coverage remained the same at 100.0% when pulling f7b05672cc3f1b3d91edcc04b1058c1f8faa16b2 on RHagenaars:master into 90fb86fa2474a2788236b20c9939c4842eaf3e8c on dbader:master.

Coverage remained the same at 100.0% when pulling f7b05672cc3f1b3d91edcc04b1058c1f8faa16b2 on RHagenaars:master into 90fb86fa2474a2788236b20c9939c4842eaf3e8c on dbader:master.

Hi @RHagenaars, Thanks a lot for submitting this pr! I'm thrilled to add this to the library :100:
Just now I've added support for passing arguments to the function via the decorator. And I've added some examples to the docs. Would you like to let me know what you think of this?

Hi @RHagenaars, Thanks a lot for submitting this pr! I'm thrilled to add this to the library :100:
Just now I've added support for passing arguments to the function via the decorator. And I've added some examples to the docs. Would you like to let me know what you think of this?

Thanks for the review @RHagenaars! Let's merge :rocket:


no not use version in config file

Pull Request Test Coverage Report for Build 1451

2 of 2 (100.0%)  changed or added relevant lines in 2 files are covered.
27 unchanged lines in 9 files lost coverage.
Overall coverage decreased (-0.1%) to 63.601%

Files with Coverage Reduction
New Missed Lines
%

core/loadbalancer/randam.go
2
64.29%

core/invoker.go
2
50.0%

session/session_manager.go
2
64.93%

core/config/sr_config.go
2
0.0%

server/restful/context.go
3
83.02%

core/handler/tracing_handler.go
3
30.59%

core/registry/servicecenter/util.go
3
48.21%

client/rest/rest_client.go
5
74.47%

core/router/cse/translate.go
5
33.8%

Totals

Change from base Build 1449:
-0.1%

Covered Lines:
4931

Relevant Lines:
7753

💛  - Coveralls


HARP-11222: Add depthTest property for SolidLineTechnique.

Signed-off-by: Andres Mandado<EMAIL_ADDRESS>Thank you for contributing to harp.gl!
Before requesting a pull request, please remember to check the following documents:

contribution guidelines
coding style

If you are adding new functionality we would highly appreciate if you can describe what is the capability you are adding and even better if you can add some examples. Please also remember to add tests for it.
CI Check
Our bots will check whether your PR can be directly integrated into the mainline. We have some internal integration tests running on the background, our bots will inform you of the next steps and someone from our team will take a look and help if needed!
And please do not forget to sign-off your commit! You can read more about DCO here. But, in short, you just need to use git commit -s or append --signoff when you are committing to the repo.
Happy contributing!

I'll add some testing in a follow-up PR.


[spike day] DCOS-34504: Align LDAP Authentication radio buttons

Change left margin to align authentication radio buttons
Closes https://jira.mesosphere.com/browse/DCOS-34504
Testing

Go to Settings > LDAP Direcotory
Click the "Add Directory" button
Open the Authentication tab
Verify that the radio buttons are aligned.
Verify that this change doesn't break any other positions

Trade-offs
I think it will be good to add a top margin as well (something like 6px). @lsamimi , what do you think?
Dependencies
https://github.com/mesosphere/dcos-ui-plugins-private/pull/902
Screenshots
Before

After

Edit:

@lsamimi, I will add top and bottom margin and change the text, thank you.

:tada: This PR is included in version 2.42.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[DX] Remove the Symfony 3.0 Directory Structure Question

Hi guys!
This is part of the DX initiative.
I've been seeing a lot of questions/problems with the new question during installation about installing the Symfony 3.0 directory structure. Here are some examples:

symfony/symfony#11208
http://stackoverflow.com/questions/23993295/what-is-the-new-symfony-3-directory-structure
http://knpuniversity.com/screencast/symfony2-ep1#comment-1445114153

I think this option should be removed completely from installation and we should re-think how and when to introduce the directory structure. We can't burden new users with this question - they don't know what it means and if they choose 3.0, they're in big trouble (their project will look different than the docs and everything else on the web).
Thoughts?

@weaverryan Apologize for this necro, but can we please confirm the final directory structure for Symfony 3 as we approach November (circa 60-90 days from now). Are there any plans to use the structure previously outlined? Is there a different structure planned from the one used by Symfony 2 now?
If a final decision was made and presented already, I'm sorry for asking again. Everything I search for here eventually leads back to this discussion about the installer.

@ErichHartmann : if you get no response, then open up a new issue in case people aren't checking notifications from old bug reports

or rather.. CLOSED bug reports

Fair question. Currently we are planning to use the 3.0 directory structure described above - we just disabled asking users about this structure in 2.x because it's confusing to allow new users 2 structures.
The current todo - anyone can do it - is to create a pull request on the SE that moves around the dir structure to match the 3.0 structure (currently something in the DistributionBundle mutates the files to get the structure - but for 3.0, the SE just needs to have the new structure).
Hope tha clarifies :)


Cleanup 3Bot connect app [v1.3.2]

Make sure the app shows '3Bot connect' on the front page.
Rename Pages => Threefold News Circles
Rename Pay => Wallet
Remove circles

This will make it easier to keep track

[ ] Make sure the app shows '3Bot connect' on the front page.
[ ] Rename Pages => Threefold News Circles
[ ] Rename Pay => Wallet
[ ] Remove circles


Added SMT conversion for int and bool existential quantifiers

I was a little reluctant to make the translation work for any sort, but I'm willing to implement it for all sorts if you think that it should work.
Fixes Issue #1639

Reviewer checklist

[ ] Test coverage: stack test --coverage
[ ] Public API documentation: stack haddock

@ttuegel @virgil-serbanuta I think this is much better now; thanks for your feedback! could you please take another look?


[CI] Centos GPU Seg Fault

Seg Fault
Unrelated PR - #16497
Pipeline - http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fcentos-gpu/detail/PR-16497/1/pipeline
src/operator/numpy/../tensor/broadcast_reduce-inl.cuh: In function 'void mxnet::op::broadcast::ReduceImpl(cudaStream_t, const mxnet::TBlob&, const mxnet::TBlob&, const mxnet::TBlob&, mxnet::OpReqType, const mxnet::TBlob&, const mshadow::Tensor<mshadow::gpu, 1, char>&, const mxnet::op::broadcast::ReduceImplConfig<ndim>&) [with Reducer = mshadow::red::sum; int ndim = 4; DType = long int; OP1 = mxnet::op::mshadow_op::mul; OP2 = mxnet::op::mshadow_op::hypot_grad_left; cudaStream_t = CUstream_st*]':
src/operator/numpy/../tensor/broadcast_reduce-inl.cuh:564:1: internal compiler error: Segmentation fault
 void ReduceImpl(cudaStream_t stream, const TBlob& small, const TBlob& lhs, const TBlob& rhs,
 ^

Not sure if it is a one-off.. hence closing. will reopen if the issue resurfaces.


onOrderModelPostPersist event registration behaviour

Hallo,
when working on an issue for one of our customers, i found out that the following event is not registered persistently on all our installations. I would suspect that some had older Versions installed as the Event is registered inside the update function but commented out inside the install function.
If the subscribeEvent was commented out by mistake, i can provide a PR.
Best regards,
Jochen

Hi,
the onOrderModelPostPersist event is no longer used for Shopware Connector versions higher than 1.7. This ensures compatibility with Shopware version 5. Therefore, this event is only created for Shopware Connector versions 1.7 and lower, which are still compatible with Shopware version 4.
The subscribeEvent was not inadvertently commented out.
We hope we could help you with this issue. Thank you.
Best regards,
Ioana Moldovan

Hallo,
the Problem persists that you cannot create orders from aborted carts. The plenty_orders entry is then missing.
Best regards,
Jochen

Hi,
can you tell me please the shopware and shopware connector versions, where this problem occurs?
Best regards,
Ioana

Hi,
Shopware: 5.0.2
Connector: 1.7.4
Best regards,
Jochen

Hi,
ich habe Rücksprache gehalten und wir werden den Erreignis onOrderModelPostPersist auch beim Installieren des Connectors wieder reinnehmen. Ich danke dir für den Hinweis.
Gruß
Ioana

Hallo Ioana,
please also add the event to the next version's update method in case there are installations without the event at all. Our Customer for example was missing that event completly. Events can savely be registered more then once.
Best regards,
Jochen


[question] Additional options for page number positioning

Hi David,
thanks for this nice script, I was wondering if you could also add the following two options for page numbering positioning?

TM = Top Middle
BM = Bottom Middle

Thanks!

Done as requested - except that I have used TC and BC (top/bottom centre) rather than TM and BM.

Thanks David, you're a lifesaver!


How to modify confirmBtnText color for Android？

thanks

within react-native you could't you have to modify on native code within your project root open
android/app/src/main/res/values/styles.xml and add

clean and re-build again, this should do the trick

@HasnAr Thank you very much

@Adar-Li  dose it work, cause in mine project it doesn't


Move jvm pause metrics names

What changes are proposed in this pull request?
Move jvm pause metrics names so that it can be used.
Why are the changes needed?
This is for the jvm pause metrics name string reuse
Does this PR introduce any user facing changes?
NA

BTW, It would be better if we make all the metric name as constant.

@yyongycy But Feel free to do it by another ticket, for this PR, I think it is enough.

alluxio-bot, merge this please


minor improvements to do

utility.ts

[x] to_hex_string should also accept typed arrays.

export const to_hex_string = (arr: number[] | TypedArray, sep: string = ", ", prefix: string = "0x", trailing_sep: boolean = false) => {
	const num_arr: number[] = (arr as TypedArray).buffer ? Array.from(arr as TypedArray) : arr as number[]
	const str = num_arr.map(v => v.toString(16).toUpperCase()).reduce((s, v) => {
		v = v.length === 2 ? v : "0" + v
		return s + prefix + v + sep
	}, "")
	return "[" + str.substring(0, str.length - (trailing_sep ? 0 : sep.length)) + "]"
}

[x] introduce a type alias for the commonly used 2-tuple [value: any, bytesize: number] return type of all decode functions.

export type Decoded<V extends any, ByteSize: number> = [value: V, bytesize: ByteSize]
// sample usage:
const decode_u4l = (buf: Uint8Array, offset: number): Decoded<number> => [new Uint32Array(buf.slice(offset, offset+4).buffer)[0], 4]

[x] add a schema based FileWriter class or add a writeFile schema encoding function to FileParser

schema_codec.ts

[x]  introduce the convenience schema class SHeadLengthArray<S, HeadType extends NumericType> extends SArray<S>, which behaves just like SArray, but has its array length defined in the head (beginning) bytes in the provided byte format NumericType

[x] also introduce a similar SHeadBytes<HeadType extends NumericType> schema class for convenient decoding of Uint8Arrays

tasks above were completed at commit d7f7bb641d1831ffe9fb962ea53c67c2f03b2df4

primitive_codec.ts

[ ] add javascript primitive types with binary_format branding

example:
export type U4L = number | number & { format: "unsigned", bytesize: 4, endian: "little" }
export type I4L = number | number & { format: "signed", bytesize: 4, endian: "little" }
export type U1 = number | number & { format: "unsigned", bytesize: 1, endian: undefined }
/** a `cstring` is a string terminated by the `"\u0000"` (or `0x00`) byte character */
export type CSTR = string | `${string}\u0000` | `${string}\\u0000`
/** a string of length `N` */
export type STR<N extends number = 0> = string | string & { length: N }

doing so will allow you to specify the byte_format directly in the type declarations, and it will also get picked up by document generators, such as TypeDoc. moreover, it will be fully compatible with regular javascript types
sample usage:
type CanvasRectHeader_type = {
	/** x-coord pixel position with respect to the top-left corner */
	x: U4L
	/** y-coord pixel position with respect to the top-left corner */
	y: U4L
	/** pixel width (rightward) of this canvas */
	width: U4L
	/** pixel height (downward) of this canvas */
	height: U4L
	/** specify the length of `palette`. it must be at least `2` because the first entry is reserved for erasing/background. `min(palette_len) = 2` and `max(palette_len) = 255` */
	palette_len?: never & U1 // this is an opaque type, which is present in the binary format, but not in the decoded object
	/** ordered array of palette ids. `palette.length == palette_len` or `bytesizeOf(palette) / 4 == palette_len`. <br>
	 * the first entry (`palette[0]`) is always set to `0x00` because it is reserved for eraser mode
	*/
	palette: [eraser: 0, ...indexes: I4L[]]
}

class CanvasRectHeader extends SRecord<CanvasRectHeader_type> {
	constructor() {
		super(
			new SPrimitive("u4l").setName("x"),
			new SPrimitive("u4l").setName("y"),
			new SPrimitive("u4l").setName("width"),
			new SPrimitive("u4l").setName("height"),
			new SHeadPrimitive("u1", "i4l[]", [0,] as [0, ...number[]]).setName("palette"),
		)
	}
}


"Folder donejs-chat already exsits."

There's a typo if the folder already exists:
"Folder donejs-chat already exsits."

It's Canadian English for exists :maple_leaf:


The trails gets truncated

SEVERITY:  Major
MODULE: Bubble Chart
PLATFORM:

Windows 7 / Chrome 49.0
Windows 7 / Firefox 44.0
Windows 7 / Internet Explorer 11.0

STEPS TO REPRODUCE:

Navigate to the URL: http://tools-stage.gapminderdev.org/tools/bubbles
Click on India bubble
Click play and generate the trails

ACTUAL RESULTS:
The trails gets truncated
EXPECTED RESULTS:
The trails should not be truncated
NOTE: The trails are truncated now because now the minimum y –axis value is started from 30 but earlier it was started from 10
SCREENSHOT:

i adjusted the Y axis for the better general picture. yes, some of the trails will get truncated, but it's not a severe problem. if users will complain, i will adjust it back


docs: add documentation to IFileSystemUtils and all implementations

Codecov Report

Merging #389 (cdbad18) into main (3f95dea) will increase coverage by 68.94%.
The diff coverage is 0.00%.

@@            Coverage Diff            @@
##           main     #389       +/-   ##
=========================================
+ Coverage      0   68.94%   +68.94%
=========================================
  Files         0      241      +241
  Lines         0     7735     +7735
=========================================
+ Hits          0     5333     +5333
- Misses        0     2402     +2402

Files Changed
Coverage Δ

src/Microsoft.Sbom.Common/FileSystemUtils.cs
0.00% <0.00%> (ø)

src/Microsoft.Sbom.Common/UnixFileSystemUtils.cs
0.00% <ø> (ø)

...rc/Microsoft.Sbom.Common/WindowsFileSystemUtils.cs
0.00% <0.00%> (ø)

... and 238 files with indirect coverage changes


Draft: Make any places in Tractus-X EDC which hardcode specific CX Verifiable Credentials configurable

Overview
Currently, the process of data sovereignty is bound to a predefined set of ODRL policies and Verifiable Credentials - the Catena-X Policy Profile. This concept and the reference implementation should be enabled to be reused by other data spaces such as Factory-X or other Manufacturing-X dataspaces. Furthermore, it should be easy to include additional VCs, e.g. for Company Certificates. This requires a flexible and configurable reference implementation, which enables any Verifiable Credentials or any ODRL policies to be maintained, negotiated and verified.
Explain the topic in 2 sentences
Transform the hardcoded references to CX Credentials from the edc-extension to configurable one, which can be configured for a deployed EDC initially and updated at a later point in time after a restart without any data loss.
What's the benefit?
ODRL policy framework based on Verifiable Credentials can be easily extended and applied to other data spaces.
What are the Risks/Dependencies ?
It must not influence existing deployments to ensure that running Catena-X data exchange still works as before.
Detailed explanation
Current implementation
Hardcoded reference to CX Verifiable Credentials
Proposed improvements
At the following steps in E2E negotiation and data transfer processes, CX VCs are hardcoded in Tractus-X EDC implementation:

on Data Consumer Side: Initial call from EDC to Wallet: get STS Token uses deployment variable default-sts-scope... = e.g. https//w3id.org/catenax/DataExchangeGovernanceCredential , https//w3id.org/catenax/MembershipCredential , https//w3id.org/catenax/DismantlerCredential , https//w3id.org/catenax/BPNCredential ,

on Data Provider Side: after catalog request by Data Consumer and get Verifiable Presentation by Data Provider: check of Verifiable Presentation content against ODRL constraints (credentialResult) like in examples [1]

and 4. Analog calls like 1 and 2 if negotiation is triggered

and 6. Analog calls like 1 and 2 if data access token is refreshed is triggered

and 8. Analog calls like 1 and 2 if data are accessed and transferred

Links:

MembershipCredentialConstraintFunction,
FrameworkAgreementCredentialConstraintFunction
DismantlerCredentialConstraintFunction

Configuration should be done

at deployment
at a later point in time without any data loss

Most likely this means it will happen in Helm Chart but further proposals are welcome.
Most likely only provider side need to be adapted. Consumer side uses in scope already the VCs based on configuration before.
Out of scope
...
Feature Team
Contributor

@hemantxpatel
Axel Albrecht as representative

Committer

@arnoweiss
Committer 2

User Stories

Issue 1, linked to specific repository
Issue 2, linked to another specific repository

Acceptance Criteria

[ ] Concept how to do the flexible configuration in a discussion in Tractus-X EDC
[ ] Tractus-X EDC: https://github.com/eclipse-tractusx/tractusx-edc/blob/main/edc-extensions/cx-policy/src/main/java/org/eclipse/tractusx/edc/policy/cx/ adapted to a configurable framework

Test Cases

Test Case 1

Steps

Do something
Click something
Add something

Expected Result

Expectation
Expectation
Expectation

Architectural Relevance

The following items are ensured (answer: yes) after this issue is implemented:

[x] This feature aligns with our current architectural guidelines

Data Sovereignty: All data sharing activities across company boundaries follow the Catena-X Regulatory Framework, in particular the Data Exchange Governance, and the Dataspace Protocol via a compliant Connector (like the tractusx-edc or similar, see Connector KIT)

[x] CX-0010 Business Partner Number
[x] CX-0013 Identity of Member Companies
[x] CX-0018 Data Space Connectivity (EDC)
[x] CX-0049 DID Document Schema
[x] CX-0050 Framework Agreement Credential
[x] CX-0149 Verified Company Identity

Interoperability: Digital Twins are used (compliant to the Digital Twin KIT and the Industry Core KIT)

[ ] CX-0001 EDC Discovery API
[ ] CX-0002 Digital Twins in Catena-X
[ ] CX-0018 Data Space Connectivity (EDC)

Data Format:

[ ] The data model is based on a published Semantic Model

[ ] The impact on the overall system architecture has been assessed. The Feature does not require changes to the architecture or any existing standard? Please have a look here on the overarching architecture
[ ] Potential risks or conflicts with existing architecture has been assessed

Justification: (Fill this out, if at least one of the checkboxes above cannot be ticked. Contact the Architecture Management Committee to get an approval for the justification)
Additional information

[x] I am aware that my request may not be developed if no developer can be found for it. I'll try to contribute a developer (bring your own developer)

[1]

Constraint (in Policy)
VC Type
claim
value

{
    "leftOperand": "cx-policy:FrameworkAgreement",
    "operator": "eq",
    "rightOperand": "DataExchangeGovernance:1.0"
}

 DataExchangeGovernanceCredential
 type & credentialSubject.contractVersion
 DataExchangeGovernanceCredential & 1.0

{
    "leftOperand": "Membership",
    "operator": "eq",
    "rightOperand": "active"
}

 MembershipCredential
 type
 MembershipCredential

{
    "leftOperand": "BusinessPartner",
    "operator": "eq",
    "rightOperand": "BPNLPLACEHOLDER"
}

 MembershipCredential
 credentialSubject.holderIdentifier
 BPNLPLACEHOLDER

Refinement in progress. Refinement will be ready for review only by Nov 6, 2024. Please do not review this issue now.

I like to contribute

These aren't hardcoded, except for potentially the Membership Credential mapping, but there are definitely a number of things that can be improved. The "EDC way" of doing things is to create an extension per dataspace for handling policies and credentials as opposed to runtime configuration. With this in mind:

Check to ensure the Membership Credential is not automatically used in a "core" TX module
Ensure all C-X policy and credential support are in identifiable extensions that can be excluded from another dataspace deployment

Note that a lot of additional items are required for the TX distribution to be usable outside of C-X:

No use of BPNs
No use of the EDC Discovery Service.

If the above are followed, another dataspace can use the core TX distribution without the C-X specific extensions.
It may be worth refining this issue to be "Separate C-X specific functionality and extensions in TX EDC"

Based on proposal to create dedicated EDC extensions per Verifiable Credential, new sig-release issue will be planned based on this decision if first additional VC are known. This issue will not be planned for 2503


Why is a PCF file being provided for the xc7/linux_litex_demo?

The XDC file should be enough? Why is a PCF file being provided as well?

@acomodi - Do you know why?

@acomodi - Do you know why?


Any plans for an @dispatch decorator?

I saw in this blog post by @e-schultz that the @select and @dispatch features were on the roadmap (authored a year ago). I see that select is implemented but was curious if there are plans to implement the dispatch decorator that the blog post mentions?

Dispatch was on the roadmap, but fell off of my radar for awhile.  I do think it's worth revisiting though.
Initially, I was moving towards favouring ActionServices that did the dispatch for you instead of just returning the POJO, but starting to rethink that.
Given:
export class SomeComponent {
  constructor(private actions:Actions) { }
   @dispatch()
   something() {
       return actions.myActionCreator('a','b','c');
   }
}

I would see dispatch being called with the result - but the result still getting returned. This would fit the use-case of your action-creator being a service for whatever reason.
Could also possibly do something like:
import { myActionCreator } from './somePlace';
@Component({
template: `<button (click)="myAction('hello!')">Click</button>`
});
export class MyComponent {

   @dispatch()
   myAction = myActionCreator;
}

This could work in the case here your action creators are just pure/exported functions that don't need access to Angular DI.
If you had a case where you didn't need to do any logic within the component, and the template called myAction(x,y,z) directly.
Not opposed to adding this, any thoughts / use cases I might be missing? @SethDavenport

@jspengeman have an initial pass of this as a pr - going to do some more testing, but shouldn't be too long until it's ready.

@e-schultz Thanks for the detailed reply. Additionally, thanks getting something going so quickly. I would have contributed but I was curious what the maintainers thoughts were first. Appreciate the initiative.
Ill take a look at the PR.

I like this - injecting NgRedux and dispatching manually in ActionServices is convenient but feels a bit weird to write.
Doing an @dispatch on a property like this is a lot nicer:
@Injectable()
export class ActionService {
  @dispatch myAction = (payload) => ({ type: 'MINE', payload })
  @dispatch myActionToo = (payload) => ({ type: 'MINE2', payload })
  @dispatch myThirdAction = (payload) => ({ type: 'MINE3', payload })
}
``

(as long as AoT can handle those arrow functions, I haven't tried)

Whatever we do, it should allow people to easily unit test their action creators though.

@dispatch has been released as part or 6.3.0.


Map Recursion issue

I've added a test that exposes an issue I'm running into with recursive map merges.  I was expecting the merge to copy values from the src to the dst recursively, overwriting existing values.  That doesn't appear to be the case.  Top level values are being overwritten but "deeper" values are not.
In the test in this example you can see that the KeyValue map from a is merged into b but values are not overwritten if b is already set e.g. b.KeyValue["Key1"] should be getting set to "not value 1"
Is this a bug or is it functioning as designed?

PR 8 (with MergeWithOverwrite) solves this issue.

I read your message. Thanks for contacting. I'm under a heavy load of work and I'm not able to pull all the requests but I will try to check them this week.

Any news ?:D

Nope, sorry. I'm under a lot of pressure and I have little time for anything else.


But report: the related question does no match to the querry language.

Describe the bug
The answer corresponsed to the querry language, like Chinese and Japanese, but the related question is still in English
To Reproduce
Ask a question in Chinese, like "亚马逊河流在亚洲吗", which means Is Amazon river in Asia?.
Then the related questions would be in En
Expected behavior
Would you like to modify the prompt?
Screenshots

What model are you using?

What model are you using?

GROQ,
I would try openai later.

What model are you using?

GPT3.5turbo  and GROQ are both like this

What model are you using?

UPDATE: OPENAI(3.5) work well in, maybe 80%, in most cases.  GROQ is not ok in most cases.

GROQ

OpenAI's models (GPT4 and GPT4o) would give responses most of the time in the language the question is asked. Open source models are still reaching the level of closed sourced models but they're still a little dumb so this will be fixed in the future as we get to see more models.