fix: Testing instructions for AWS/IBM Cloud

Update docs regarding how to run E2E (end to end) tests locally.
How to test
Prerequisites

AWS or IBM Cloud with OCP cluster

Run
Checkout PR branch locally
git clone -b fix/testing-instructions --single-branch --depth 1 https://github.com/mateusoliveira43/oadp-operator

Read instructions in https://github.com/mateusoliveira43/oadp-operator/blob/fix/testing-instructions/docs/developer/testing/TESTING.md on how to set up environment variables to run tests in AWS or IBM Cloud.
Install OADP operator
make deploy-olm

Run E2E tests setup (creates DPA spec that will be used for tests run) (NORMALLY THIS STEP IS NOT NECESSARY, if running make test-e2e)
make test-e2e-setup

Install Ginkgo CLI (NORMALLY THIS STEP IS NOT NECESSARY, if running make test-e2e)
make install-ginkgo

Run MySQL application RESTIC test (this is basically make test-e2e with --ginkgo.focus)
export CLUSTER_TYPE=$(oc get infrastructures cluster -o jsonpath='{.status.platform}' | tr A-Z a-z)
ginkgo run -mod=mod tests/e2e/ -- \
-settings="/tmp/test-settings/oadpcreds" \
-oc_cli="$(which oc)" \
-provider="$CLUSTER_TYPE" \
-creds_secret_ref="cloud-credentials" \
-credentials="$OADP_CRED_FILE" \
-ci_cred_file="$CI_CRED_FILE" \
-velero_namespace="$OADP_TEST_NAMESPACE" \
-velero_instance_name="velero-test" \
-timeout_multiplier=1 \
-artifact_dir="/tmp" \
--ginkgo.label-filter="$(echo "($(echo '! aws && ! gcp && ! azure && ! ibmcloud' | sed -r "s/[&]* [\!] $CLUSTER_TYPE|[\!] $CLUSTER_TYPE [&]*//")) || $CLUSTER_TYPE")" \
--ginkgo.timeout=2h \
--ginkgo.focus="MySQL application RESTIC"

You should get a result similar to this:
2023/09/18 10:51:35 Using velero prefix: velero-e2e-79d163fe-8452-456f-89e9-84336054c473
Running Suite: OADP E2E Suite - MATEUS_PATH/tests/e2e
=====================================================================================
Random Seed:<PHONE_NUMBER>

Will run 1 of 25 specs
I0918 10:51:36.805585   22971 request.go:690] Waited for 1.047442962s due to client-side throttling, not priority and fairness, request: GET:MATEUS_ENDPOINT/apis/batch/v1?timeout=32s
SSSSSSSSSSSSSSS
------------------------------
P [PENDING]
Must-gather backup restore tests backup, restore applications, and must gather Mongo application DATAMOVER
MATEUS_PATH/tests/e2e/must-gather_suite_test.go:357
------------------------------
SSSSCluster type: ibmcloud
2023/09/18 10:51:45 Waiting for velero pod to be running
2023/09/18 10:51:50 pod: velero-5d8c698675-l46k2 is not yet running with status: {Pending [{Initialized False 0001-01-01 00:00:00 +0000 UTC 2023-09-18 10:51:45 -0300 -03 ContainersNotInitialized containers with incomplete status: [velero-plugin-for-aws]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-09-18 10:51:45 -0300 -03 ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-09-18 10:51:45 -0300 -03 ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-09-18 10:51:45 -0300 -03  }]    <IP_ADDRESS> <IP_ADDRESS> [{<IP_ADDRESS>}] 2023-09-18 10:51:45 -0300 -03 [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2023-09-18 10:51:48 -0300 -03,FinishedAt:2023-09-18 10:51:48 -0300 -03,ContainerID:cri-o://5e0a0db1c4cd80fc762e73d58cfe2d9ece4c0078e0a4bf054ce3bdd42320ea0c,}} {nil nil nil} true 0 quay.io/konveyor/openshift-velero-plugin:latest quay.io/konveyor/openshift-velero-plugin@sha256:ed2469f89a933a044a0ff7aac5b7b1c40a9c5c28f5b64f5cc659274b257db2c1 cri-o://5e0a0db1c4cd80fc762e73d58cfe2d9ece4c0078e0a4bf054ce3bdd42320ea0c <nil>} {velero-plugin-for-aws {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 quay.io/konveyor/velero-plugin-for-aws:latest   <nil>}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 quay.io/konveyor/velero:latest   0xc000d54bff}] Burstable []}
2023/09/18 10:51:55 Waiting for restic pods to be running
2023/09/18 10:51:55 Checking for correct number of running Restic pods...
2023/09/18 10:51:55 Installing application for case mysql-restic-e2e
diff found for key: volumes
diff found for key: metadata
diff found for key: readOnlyRootFilesystem
diff found for key: allowHostDirVolumePlugin
diff found for key: allowHostIPC
diff found for key: allowHostPorts
diff found for key: allowHostPID
diff found for key: groups
diff found for key: allowHostNetwork
updating resource: security.openshift.io/v1, Kind=SecurityContextConstraints; name: mysql-persistent-scc
checking for the NAMESPACE: mysql-persistent
 2023/09/18 10:52:31 Running pre-backup function for case mysql-restic-e2e
2023/09/18 10:52:31 Verifying backup/restore data of todolist
I0918 10:52:32.826295   22971 request.go:690] Waited for 1.010845971s due to client-side throttling, not priority and fairness, request: GET:MATEUS_ENDPOINT/apis/apiextensions.k8s.io/v1?timeout=32s
2023/09/18 10:52:35 Request error: <nil>
2023/09/18 10:52:35 Retrying in 15s
2023/09/18 10:52:50 Request error: <nil>
2023/09/18 10:52:50 Retrying in 1m0s
Data before the curl request:
 [{"Id":1,"Description":"time to make the donuts","Completed":false}]

PrebackState: true, so make a curl request
Writing data to backupFile (backup-data.txt):
 [{"Id":1,"Description":"time to make the donuts","Completed":false},{"Id":3,"Description":"2023-09-18 10:53:50.477012445 -0300 -03 m=+135.144109741","Completed":false},{"Id":4,"Description":"Monday","Completed":false}]

2023/09/18 10:53:50 Sleeping for 0s to allow application to be ready for case mysql-restic-e2e
2023/09/18 10:53:50 Creating backup mysql-restic-e2e-874a8e31-562a-11ee-8a6a-047bcb64f61d for case mysql-restic-e2e
2023/09/18 10:54:22 Backup for case mysql-restic-e2e succeeded
2023/09/18 10:54:22 Uninstalling app for case mysql-restic-e2e
2023/09/18 10:54:38 Creating restore mysql-restic-e2e-874a9290-562a-11ee-8a6a-047bcb64f61d for case mysql-restic-e2e
2023/09/18 10:55:31 Running restic post restore script for case mysql-restic-e2e
2023/09/18 10:55:31 Running post restore script for mysql-restic-e2e-874a9290-562a-11ee-8a6a-047bcb64f61d
2023/09/18 10:55:32 command: /usr/bin/bash MATEUS_PATH/docs/scripts/dc-restic-post-restore.sh mysql-restic-e2e-874a9290-562a-11ee-8a6a-047bcb64f61d
2023/09/18 10:55:32 stdout: using OADP Namespace openshift-adp
restore: mysql-restic-e2e-874a9290-562a-11ee-8a6a-047bcb64f61d
label: mysql-restic-e2e-874a9290-562a-11ee-8a6a-047bcb64f61d
Deleting disconnected restore pods
No resources found
Found deployment mysql-persistent/todolist, setting replicas: 1, paused: false
deploymentconfig.apps.openshift.io/todolist patched
2023/09/18 10:55:32 stderr: &{%!s(int=32768)   %!s(int=0) %!s(int64=0)}
2023/09/18 10:55:32 err: %!s(<nil>)
checking for the NAMESPACE: mysql-persistent
 2023/09/18 10:55:32 Running post-restore function for case mysql-restic-e2e
2023/09/18 10:55:33 Verifying backup/restore data of todolist
I0918 10:55:34.081966   22971 request.go:690] Waited for 1.011579663s due to client-side throttling, not priority and fairness, request: GET:MATEUS_ENDPOINT/apis/velero.io/v2alpha1?timeout=32s
Data came from backup-file
 [{"Id":1,"Description":"time to make the donuts","Completed":false},{"Id":3,"Description":"2023-09-18 10:53:50.477012445 -0300 -03 m=+135.144109741","Completed":false},{"Id":4,"Description":"Monday","Completed":false}]

Data from the response after restore
 [{"Id":1,"Description":"time to make the donuts","Completed":false},{"Id":3,"Description":"2023-09-18 10:53:50.477012445 -0300 -03 m=+135.144109741","Completed":false},{"Id":4,"Description":"Monday","Completed":false}]

•SSSS2023/09/18 10:56:35 Deleting Velero CR
I0918 10:56:36.597290   22971 request.go:690] Waited for 1.043888909s due to client-side throttling, not priority and fairness, request: GET:MATEUS_ENDPOINT/apis/build.openshift.io/v1?timeout=32s

Ran 1 of 25 Specs in 306.372 seconds
SUCCESS! -- 1 Passed | 0 Failed | 1 Pending | 23 Skipped
PASS

Ginkgo ran 1 suite in 5m8.423868638s
Test Suite Passed

/hold
Want everyone with AWS/IBM Cloud to run steps from PR description before merging, and checking if it works :smile:

nice!  Thank you @mateusoliveira43 this got me past my initial error.  FYI.. I had to export bucket_file vs.. bucket
e.g.
#export OADP_BUCKET=/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/KEYS/bucket
export OADP_BUCKET_FILE=/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/KEYS/bucket

Thank you!  Still hitting something but I need to dig into it.  The test is running through :)

ah nice.. cleaned up MI BUCKET laddi and I'm good to go!  Thank you!!!
------------------------------
2023/09/18 19:33:14 Deleting Velero CR
I0918 19:33:15.577040  154123 request.go:690] Waited for 1.000056678s due to client-side throttling, not priority and fairness, request: GET:https://api.cluster-wdhfips0906.wdhfips0906.mg.dog8code.com:6443/apis/operator.openshift.io/v1?timeout=32s

Ran 1 of 25 Specs in 165.641 seconds
SUCCESS! -- 1 Passed | 0 Failed | 1 Pending | 23 Skipped
PASS

Ginkgo ran 1 suite in 2m48.83036395s
Test Suite Passed
[whayutin@thinkdoe oadp-operator]$

@weshayutin we can debug together, but from your snippet, I suspect OADP_BUCKET was in the wrong format. It should be the bucket name, not a file path (but yes, you can use OADP_BUCKET_FILE, but I feel the other variable is easier to set up)

We're not quite there yet :)
[whayutin@thinkdoe oadp-operator]$ export CLUSTER_TYPE=$(oc get infrastructures cluster -o jsonpath='{.status.platform}' | tr A-Z a-z)
ginkgo run -mod=mod tests/e2e/ -- \
-settings="/tmp/test-settings/oadpcreds" \
-oc_cli="$(which oc)" \
-provider="$CLUSTER_TYPE" \
-creds_secret_ref="cloud-credentials" \
-credentials="$OADP_CRED_FILE" \
-ci_cred_file="$CI_CRED_FILE" \
-velero_namespace="$OADP_TEST_NAMESPACE" \
-velero_instance_name="velero-test" \
-timeout_multiplier=1 \
-artifact_dir="/tmp" \
--ginkgo.label-filter="$(echo "($(echo '! aws && ! gcp && ! azure && ! ibmcloud' | sed -r "s/[&]* [\!] $CLUSTER_TYPE|[\!] $CLUSTER_TYPE [&]*//")) || $CLUSTER_TYPE")" \
--ginkgo.timeout=2h \
--ginkgo.focus="MySQL application RESTIC"
Debug: Read json file: <nil>Debug:Unmarshaled dpa Read json file: <nil>2023/09/22 10:12:55 Using velero prefix: velero-e2e-51d592fb-74de-4ba1-9c53-9e1161cd1b82
Running Suite: OADP E2E Suite - /home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e
=========================================================================================
Random Seed:<PHONE_NUMBER>

Will run 1 of 25 specs
SETTINGS FILE AAAAAAAAAAAAAAAAAAAA: /tmp/test-settings/oadpcreds
ERROR: <nil>
Debug: Read json file: <nil>Debug:Unmarshaled dpa Read json file: <nil>I0922 10:12:57.601016   58669 request.go:690] Waited for 1.049025542s due to client-side throttling, not priority and fairness, request: GET:https://api.cluster-wdhfips0906.wdhfips0906.mg.dog8code.com:6443/apis/velero.io/v2alpha1?timeout=32s
SSSSCluster type: aws
2023/09/22 10:13:04 Waiting for velero pod to be running
2023/09/22 10:13:10 pod: velero-796f6b4884-p2wpl is not yet running with status: {Pending [{Initialized False 0001-01-01 00:00:00 +0000 UTC 2023-09-22 10:13:05 -0600 MDT ContainersNotInitialized containers with incomplete status: [velero-plugin-for-aws]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2023-09-22 10:13:05 -0600 MDT ContainersNotReady containers with unready status: [velero]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2023-09-22 10:13:05 -0600 MDT ContainersNotReady containers with unready status: [velero]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2023-09-22 10:13:05 -0600 MDT  }]    <IP_ADDRESS> <IP_ADDRESS> [{<IP_ADDRESS>}] 2023-09-22 10:13:05 -0600 MDT [{openshift-velero-plugin {nil nil &ContainerStateTerminated{ExitCode:0,Signal:0,Reason:Completed,Message:,StartedAt:2023-09-22 10:13:09 -0600 MDT,FinishedAt:2023-09-22 10:13:09 -0600 MDT,ContainerID:cri-o://148faf964280a554abdedad583f982947ac982fe0b64c76ac9873c66a2bc5c90,}} {nil nil nil} true 0 quay.io/konveyor/openshift-velero-plugin:latest quay.io/konveyor/openshift-velero-plugin@sha256:ed2469f89a933a044a0ff7aac5b7b1c40a9c5c28f5b64f5cc659274b257db2c1 cri-o://148faf964280a554abdedad583f982947ac982fe0b64c76ac9873c66a2bc5c90 <nil>} {velero-plugin-for-aws {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 quay.io/konveyor/velero-plugin-for-aws:latest   <nil>}] [{velero {&ContainerStateWaiting{Reason:PodInitializing,Message:,} nil nil} {nil nil nil} false 0 quay.io/konveyor/velero:latest   0xc000a5b28f}] Burstable []}
2023/09/22 10:13:15 Waiting for restic pods to be running
2023/09/22 10:13:15 Checking for correct number of running Restic pods...
2023/09/22 10:13:15 Installing application for case mysql-restic-e2e
diff found for key: volumes
diff found for key: allowHostPorts
diff found for key: allowHostPID
diff found for key: allowHostNetwork
diff found for key: readOnlyRootFilesystem
diff found for key: allowHostDirVolumePlugin
diff found for key: groups
diff found for key: metadata
diff found for key: allowHostIPC
updating resource: security.openshift.io/v1, Kind=SecurityContextConstraints; name: mysql-persistent-scc
checking for the NAMESPACE: mysql-persistent
 2023/09/22 10:13:36 Running pre-backup function for case mysql-restic-e2e
2023/09/22 10:13:36 Verifying backup/restore data of todolist
I0922 10:13:37.727218   58669 request.go:690] Waited for 1.000656469s due to client-side throttling, not priority and fairness, request: GET:https://api.cluster-wdhfips0906.wdhfips0906.mg.dog8code.com:6443/apis/flowcontrol.apiserver.k8s.io/v1beta2?timeout=32s
Data before the curl request:
 [{"Id":1,"Description":"time to make the donuts","Completed":false}]

PrebackState: true, so make a curl request
Writing data to backupFile (backup-data.txt):
 [{"Id":1,"Description":"time to make the donuts","Completed":false},{"Id":3,"Description":"2023-09-22 10:13:39.538073759 -0600 MDT m=+43.584036276","Completed":false},{"Id":4,"Description":"Friday","Completed":false}]

2023/09/22 10:13:39 Sleeping for 0s to allow application to be ready for case mysql-restic-e2e
2023/09/22 10:13:39 Creating backup mysql-restic-e2e-ef5b37d5-5962-11ee-874b-047bcb15f3ce for case mysql-restic-e2e
2023/09/22 10:14:00 Backup for case mysql-restic-e2e succeeded
2023/09/22 10:14:00 Uninstalling app for case mysql-restic-e2e
2023/09/22 10:14:11 Creating restore mysql-restic-e2e-ef5b4501-5962-11ee-874b-047bcb15f3ce for case mysql-restic-e2e
2023/09/22 10:15:13 Running restic post restore script for case mysql-restic-e2e
2023/09/22 10:15:13 Running post restore script for mysql-restic-e2e-ef5b4501-5962-11ee-874b-047bcb15f3ce
2023/09/22 10:15:14 command: /usr/bin/bash /home/whayutin/OPENSHIFT/git/OADP/oadp-operator/docs/scripts/dc-restic-post-restore.sh mysql-restic-e2e-ef5b4501-5962-11ee-874b-047bcb15f3ce
2023/09/22 10:15:14 stdout: using OADP Namespace openshift-adp
restore: mysql-restic-e2e-ef5b4501-5962-11ee-874b-047bcb15f3ce
label: mysql-restic-e2e-ef5b4501-5962-11ee-874b-047bcb15f3ce
Deleting disconnected restore pods
No resources found
Found deployment mysql-persistent/todolist, setting replicas: 1, paused: false
deploymentconfig.apps.openshift.io/todolist patched
2023/09/22 10:15:14 stderr: &{%!s(int=32768)   %!s(int=0) %!s(int64=0)}
2023/09/22 10:15:14 err: %!s(<nil>)
checking for the NAMESPACE: mysql-persistent
 2023/09/22 10:15:29 Running post-restore function for case mysql-restic-e2e
2023/09/22 10:15:29 Verifying backup/restore data of todolist
I0922 10:15:30.683628   58669 request.go:690] Waited for 1.027106677s due to client-side throttling, not priority and fairness, request: GET:https://api.cluster-wdhfips0906.wdhfips0906.mg.dog8code.com:6443/apis/console.openshift.io/v1?timeout=32s
Data came from backup-file
 [{"Id":1,"Description":"time to make the donuts","Completed":false},{"Id":3,"Description":"2023-09-22 10:13:39.538073759 -0600 MDT m=+43.584036276","Completed":false},{"Id":4,"Description":"Friday","Completed":false}]

Data from the response after restore
 [{"Id":1,"Description":"time to make the donuts","Completed":false},{"Id":3,"Description":"2023-09-22 10:13:39.538073759 -0600 MDT m=+43.584036276","Completed":false},{"Id":4,"Description":"Friday","Completed":false}]

•SSSSSSSSSSSSSSSSSSS
------------------------------
P [PENDING]
Must-gather backup restore tests backup, restore applications, and must gather Mongo application DATAMOVER
/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e/must-gather_suite_test.go:357
------------------------------
2023/09/22 10:15:45 Deleting Velero CR
I0922 10:15:46.903208   58669 request.go:690] Waited for 1.049490332s due to client-side throttling, not priority and fairness, request: GET:https://api.cluster-wdhfips0906.wdhfips0906.mg.dog8code.com:6443/apis/velero.io/v2alpha1?timeout=32s

Ran 1 of 25 Specs in 175.331 seconds
SUCCESS! -- 1 Passed | 0 Failed | 1 Pending | 23 Skipped
PASS

Ginkgo ran 1 suite in 2m58.533506487s
Test Suite Passed
[whayutin@thinkdoe oadp-operator]$
[whayutin@thinkdoe oadp-operator]$
[whayutin@thinkdoe oadp-operator]$ ## NOW RUN MAKE TEST-E2E ##############
[whayutin@thinkdoe oadp-operator]$
[whayutin@thinkdoe oadp-operator]$
[whayutin@thinkdoe oadp-operator]$
[whayutin@thinkdoe oadp-operator]$ make test-e2e
$CLUSTER_TYPE is [aws]
mkdir -p /tmp/test-settings
TARGET_CI_CRED_FILE="/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/KEYS/ci-credentials" AZURE_RESOURCE_FILE="/var/run/secrets/ci.openshift.io/multi-stage/metadata.json" CI_JSON_CRED_FILE="/Users/drajds/.aws/osServicePrincipal.json" \
OADP_JSON_CRED_FILE="/var/run/oadp-credentials/azure-credentials" OADP_CRED_FILE="/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/KEYS/credentials" OPENSHIFT_CI="true" \
PROVIDER="aws" BUCKET="cvpbucket" BSL_REGION="us-east-1" SECRET="cloud-credentials" TMP_DIR=/tmp/test-settings \
VSL_REGION="us-east-1" BSL_AWS_PROFILE="default" /bin/bash "tests/e2e/scripts/aws_settings.sh"
ginkgo run -mod=mod tests/e2e/ -- -credentials=/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/KEYS/credentials \
-velero_namespace=openshift-adp \
-settings=/tmp/test-settings/oadpcreds \
-velero_instance_name=velero-test \
-timeout_multiplier=1 \
--ginkgo.label-filter="( ! gcp && ! azure && ! ibmcloud) || aws" \
#--focus="MySQL application CSI" \
-ci_cred_file=/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/KEYS/ci-credentials \
-provider=aws \
-creds_secret_ref=cloud-credentials \
-artifact_dir=/tmp \
-oc_cli=/home/whayutin/.local/bin/oc \
--ginkgo.timeout=2h
Debug: Read json file: <nil>Debug:Unmarshaled dpa Read json file: <nil>2023/09/22 10:16:29 Using velero prefix: velero-e2e-fe393430-803c-4fbf-9f63-590b3d2eea48
Running Suite: OADP E2E Suite - /home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e
=========================================================================================
Random Seed:<PHONE_NUMBER>

Will run 24 of 25 specs
SETTINGS FILE AAAAAAAAAAAAAAAAAAAA: /tmp/test-settings/oadpcreds
ERROR: <nil>
Debug: Read json file: <nil>Debug:Unmarshaled dpa Read json file: <nil>------------------------------
[BeforeSuite] [FAILED] [0.204 seconds]
[BeforeSuite]
/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e/e2e_suite_test.go:92

  [FAILED] Unexpected error:
      <*fs.PathError | 0xc000b55f50>:
      open : no such file or directory
      {
          Op: "open",
          Path: "",
          Err: <syscall.Errno>0x2,
      }
  occurred
  In [BeforeSuite] at: /home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e/e2e_suite_test.go:118 @ 09/22/23 10:16:29.606
------------------------------
2023/09/22 10:16:29 Deleting Velero CR
I0922 10:16:30.954952   59839 request.go:690] Waited for 1.050076699s due to client-side throttling, not priority and fairness, request: GET:https://api.cluster-wdhfips0906.wdhfips0906.mg.dog8code.com:6443/apis/helm.openshift.io/v1beta1?timeout=32s
[AfterSuite] [FAILED] [2.934 seconds]
[AfterSuite]
/home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e/e2e_suite_test.go:125

  [FAILED] Unexpected error:
      <*errors.errorString | 0xc0008fd0a0>:
      resource name may not be empty
      {
          s: "resource name may not be empty",
      }
  occurred
  In [AfterSuite] at: /home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e/e2e_suite_test.go:134 @ 09/22/23 10:16:32.54
------------------------------

Summarizing 2 Failures:
  [FAIL] [BeforeSuite]
  /home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e/e2e_suite_test.go:118
  [FAIL] [AfterSuite]
  /home/whayutin/OPENSHIFT/git/OADP/oadp-operator/tests/e2e/e2e_suite_test.go:134

Ran 0 of 25 Specs in 3.139 seconds
FAIL! -- A BeforeSuite node failed so all tests were skipped.
--- FAIL: TestOADPE2E (3.14s)
FAIL

Ginkgo ran 1 suite in 6.350804755s

Test Suite Failed
make: *** [Makefile:454: test-e2e-ginkgo] Error 1

/unhold

/retest

I think this can be improved a bit, question if within this PR or separate ones?

There is no OC CLI in the PATH

bin/setup-envtest missing
oc missing.

$ make install-ginkgo
which: no oc in (/home/migi/.local/bin:/home/migi/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin)
/bin/sh: line 1: get: command not found
$CLUSTER_TYPE is []
bash: line 1: /home/migi/Development/Upstream/OADP/mpryc-oadp-operator/bin/setup-envtest: No such file or directory
go install -v -mod=mod github.com/onsi/ginkgo/v2/ginkgo
go: downloading golang.org/x/tools v0.11.0

There is no OC CLI in the PATH

tests/e2e/scripts/_settings.sh missing
oc missing

$ make test-e2e-setup
which: no oc in (/home/migi/.local/bin:/home/migi/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin)
/bin/sh: line 1: get: command not found
$CLUSTER_TYPE is []
bash: line 1: /home/migi/Development/Upstream/OADP/mpryc-oadp-operator/bin/setup-envtest: No such file or directory
cat: /var/run/oadp-credentials/new-velero-bucket-name: No such file or directory
mkdir -p /tmp/test-settings
TMP_DIR=/tmp/test-settings \
OPENSHIFT_CI="true" \
PROVIDER="" \
SECRET="cloud-credentials" \
AZURE_RESOURCE_FILE="/var/run/secrets/ci.openshift.io/multi-stage/metadata.json" \
CI_JSON_CRED_FILE="/Users/drajds/.aws/osServicePrincipal.json" \
OADP_JSON_CRED_FILE="/var/run/oadp-credentials/azure-credentials" \
OADP_CRED_FILE="/var/run/oadp-credentials/new-aws-credentials" \
BUCKET="" \
TARGET_CI_CRED_FILE="/Users/drajds/.aws/.awscred" \
VSL_REGION="" \
BSL_REGION="us-east-1" \
BSL_AWS_PROFILE="default" \
/bin/bash "tests/e2e/scripts/_settings.sh"
/bin/bash: tests/e2e/scripts/_settings.sh: No such file or directory

Not logged to the custer:

$ make install-ginkgo
E0927 11:09:56.226559 1999614 memcache.go:238] couldn't get current server API group list: the server has asked for the client to provide credentials
E0927 11:09:56.280106 1999614 memcache.go:238] couldn't get current server API group list: the server has asked for the client to provide credentials
E0927 11:09:56.334531 1999614 memcache.go:238] couldn't get current server API group list: the server has asked for the client to provide credentials
E0927 11:09:56.391443 1999614 memcache.go:238] couldn't get current server API group list: the server has asked for the client to provide credentials
E0927 11:09:56.453311 1999614 memcache.go:238] couldn't get current server API group list: the server has asked for the client to provide credentials
error: You must be logged in to the server (the server has asked for the client to provide credentials)
$CLUSTER_TYPE is []
bash: line 1: /home/migi/Development/Upstream/OADP/mpryc-oadp-operator/bin/setup-envtest: No such file or directory
go install -v -mod=mod github.com/onsi/ginkgo/v2/ginkgo

After being logged to the cluster:

#### After oc login to the cluseter:
$ make install-ginkgo
$CLUSTER_TYPE is [aws]
bash: line 1: /home/migi/Development/Upstream/OADP/mpryc-oadp-operator/bin/setup-envtest: No such file or directory
go install -v -mod=mod github.com/onsi/ginkgo/v2/ginkgo

@mpryc we currently do not install oc, like in Pelorus. Would let this improvement to another PR
bin/setup-envtest missing comes from here https://github.com/openshift/oadp-operator/blob/master/Makefile#L172-L178 It is not necessary for E2E, but it runs every time. Would let this improvement to another PR
tests/e2e/scripts/_settings.sh missing was because you were not logged in, but I consider that a logic mistake, will try to fix now

/override ci/prow/4.12-operator-e2e-gcp

/retest

/override ci/prow/4.12-operator-e2e-gcp

/override ci/prow/4.12-operator-e2e-gcp

/override ci/prow/4.12-operator-e2e-gcp


Keybind list

Is there currently a way to put the controls at the bottom of the screen like this?

Instructional Buttons


carthage support?

Can you please add Carthage support?

Hi nitrag,
Thanks for the suggestion. I will add Carthage support in it soon. Let me know in case you have any further suggestions.
Thanks & Regards

+1 👍

Hi zdenektopic,
Carthage support is added for FAPanels. Please have a look into it and enjoy. I appreciate you suggest me about Carthage. Please feel free to ask me more to add in FAPanels.
Thanks & Regards


In ChannelHandlerContext.java, the description about AttributeKey may not accurate

In ChannelHandlerContext.java, line 81 , the description about AttributeKey following:

For example, the following handler will have as many independent {@link AttributeKey}s
as how many times it is added to pipelines, regardless if it is added to the
same pipeline multiple times or added to different pipelines multiple times

 public class FactorialHandler extends ChannelInboundHandlerAdapter {

   private final AttributeKey<Integer> counter = AttributeKey.valueOf("counter");

   // This handler will receive a sequence of increasing integers starting
   // from 1.
    @Override
   public void channelRead(ChannelHandlerContext ctx, Object msg) {
     Integer a = ctx.attr(counter).get();

     if (a == null) {
       a = 1;
     }

     attr.set(a * (Integer) msg);
   }
 }

above description is contradict with the the document  https://netty.io/wiki/new-and-noteworthy-in-4.1.html

ChannelHandlerContext.attr(..) == Channel.attr(..)

also, the description is contradict with the source code in AbstractChannelHandlerContext.
    @Override
    public <T> Attribute<T> attr(AttributeKey<T> key) {
        return channel().attr(key);
    }

In my opinion, only keep one map per Channel internally, and the ChannelHandlerContext  no longer keep a map.
In the same channel, all ChannelHandlerContext shares the map;
Not in the same channel, the map object is independent;
To verify my idea, i use the demo and test;
public class MyServer {
    public static void main(String[] args) {
        EventLoopGroup bossGroup = new NioEventLoopGroup(1);
        EventLoopGroup workerGroup = new NioEventLoopGroup();

        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(SocketChannel ch) throws Exception {
                            ChannelPipeline pipeline = ch.pipeline();
                            pipeline.addLast(new MyReplayingDecoder());
                            pipeline.addLast(new FactorialHandler());
                            pipeline.addLast(new FactorialHandler());
                        }
                    });

            ChannelFuture channelFuture = serverBootstrap.bind(11211).sync();
            channelFuture.channel().closeFuture().sync();
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }
}

public class FactorialHandler extends ChannelInboundHandlerAdapter {
    private final AttributeKey<Integer> counter = AttributeKey.valueOf("counter");

    // This handler will receive a sequence of increasing integers starting
    // from 1.
    @Override
    public void channelRead(ChannelHandlerContext
                                    ctx, Object msg) {

        // when a client connect to server, and send a message to here, the ctx.attr(counter) info is DefaultAttributeMap$DefaultAttribute@1530
        // in the same channel, the ctx.attr(counter) info is immutable;
        // then, a another client connect to server , and send a message to here, the ctx.attr(counter) info is DefaultAttributeMap$DefaultAttribute@1567
        Integer a = ctx.attr(counter).get();

        System.out.println(ctx.channel() + " " + System.identityHashCode(ctx.channel()) + "  " + ctx.attr(counter).toString() + " " + System.identityHashCode(ctx.attr(counter)));

        if (a == null) {
            a = 1;
        }

        ctx.channel().attr(counter).set(a * (Integer) msg);
        System.out.println(ctx.channel().attr(counter).get());

        ctx.fireChannelRead(msg);
    }
}

Netty version 4.1.45.Final
I'm not sure I'm right
Looking forward to your answer, thank you very much!

@fanshanhong nice catch... seems like we missed updating the docs: https://github.com/netty/netty/pull/10487

wow, thanks for your answer, i can understand the AttributeKey better.


Progress bar can significantly impact cmdlet performance

Repro Steps :
Compare perf of WebClient, BITS and Invoke-WebRequest:
Measure-Command { $wc = New-Object System.Net.WebClient; $wc.DownloadFile("https://github.com/dotnet/corefx/archive/master.zip", "C:\Users\crispop\Desktop\master.zip") }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 3
Milliseconds      : 992
Ticks             : 39928426
TotalDays         : 4.62134560185185E-05
TotalHours        : 0.00110912294444444
TotalMinutes      : 0.0665473766666667
TotalSeconds      : 3.9928426
TotalMilliseconds : 3992.8426
Vs

Measure-Command { Invoke-WebRequest "https://github.com/dotnet/corefx/archive/master.zip" -OutFile "C:\Users\crispop\Desktop\master.zip" }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 50
Milliseconds      : 821
Ticks             : 508212825
TotalDays         : 0.000588209288194444
TotalHours        : 0.0141170229166667
TotalMinutes      : 0.847021375
TotalSeconds      : 50.8212825
TotalMilliseconds : 50821.2825

Vs

Measure-Command { Start-BitsTransfer -Source "https://github.com/dotnet/corefx/archive/master.zip" -Destination "C:\Users\crispop\Desktop\masterBits2.zip" }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 4
Milliseconds      : 506
Ticks             : 45062594
TotalDays         : 5.21557800925926E-05
TotalHours        : 0.00125173872222222
TotalMinutes      : 0.0751043233333333
TotalSeconds      : 4.5062594
TotalMilliseconds : 4506.2594

Expected: Transfer speeds don't differ too much (maybe Invoke-WebRequest should be a bit faster than BITS which performs the transfer in background.)
Actual: 10x slowdown for Invoke-WebRequest

How can you be sure that it's the progress bar? With $ProgressPreference = 'SilentlyContinue', you can suppress the progress bar UI, which will surely boost Invoke-WebRequest. Also, use -UseBasicParsing when downloading files with iwr (which you shouldn't do since that's not what iwr is for...). You cannot suppress parsing, but you can suppress DOM parsing.
Moreover, I did your tests and found that the standard deviation is rather great (poor network here in China).
Please do observe Task Manager for the network speed and also do more statistics. If the network is perfect (I hosted a file locally and tried to download via localhost, all three were superfast even the file is 80MB+, with progress bar suppressed.)
And, could you include the $PSVersionTable?

PSEdition                      Core
CLRVersion
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   <IP_ADDRESS>
GitCommitId                    v6.0.0-alpha.9
PSRemotingProtocolVersion      2.3
PSVersion                      6.0.0-alpha
WSManStackVersion              3.0
SerializationVersion           <IP_ADDRESS>

I also see the progress bar slowing down Invoke-WebRequest by >50X on a fast internet connection. Every time I write a script that uses Invoke-WebRequest I need to include code to temporarily change $ProgressPreference, then restore it later (because I do want progress for other things).
Here's my output from using Invoke-WebRequest to download the same file as @SteveL-MSFT, without and then with progress:
PS D:\> $ProgressPreference = 'SilentlyContinue'
PS D:\> Measure-Command { Invoke-WebRequest "https://github.com/dotnet/corefx/archive/master.zip" -OutFile "D:\temp\master.zip" }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 6
Milliseconds      : 72
Ticks             : 60729893
TotalDays         : 7.02892280092593E-05
TotalHours        : 0.00168694147222222
TotalMinutes      : 0.101216488333333
TotalSeconds      : 6.0729893
TotalMilliseconds : 6072.9893

PS D:\> $ProgressPreference = 'Continue'
PS D:\> Measure-Command { Invoke-WebRequest "https://github.com/dotnet/corefx/archive/master.zip" -OutFile "D:\temp\master2.zip" }

Days              : 0
Hours             : 0
Minutes           : 6
Seconds           : 11
Milliseconds      : 664
Ticks             :<PHONE_NUMBER>
TotalDays         : 0.00430166988078704
TotalHours        : 0.103240077138889
TotalMinutes      : 6.19440462833333
TotalSeconds      :<PHONE_NUMBER>
TotalMilliseconds :<PHONE_NUMBER>

PS D:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14931.1000
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14931.1000
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

It would be nice to be able to have reasonable progress when downloading a file. Perhaps if there were a way to adjust the progress reporting interval to several KB instead of every byte (!!) then the overhead wouldn't be so bad.

This issue also affects Expand-Archive. For a large archive (10 MB with 3495 files), expanding it takes over 4X long with progress compared to without.

Changed title to be a generic issue with the progress bar.

Yeah, this is pretty bad ...
   Duration CommandLine
   -------- -----------
 0.0356085s 1..1e3 | % { $null }
 0.7734768s 1..1e3 | % { Write-Host "..." }
 4.8700151s 1..1e3 | % { Write-Progress "..." }

I remember noticing how much of a blocker progress writing was when I wrote the PSCX GAC provider back in 2007:
            foreach (AssemblyName name in Fusion.GetAssemblies(_type))
            {
                ++i;

                if (i % 5 == 0)
                {
                    WriteProgress(name.Name);
                }

https://github.com/Pscx/Pscx/blob/master/Src/Pscx/Providers/AssemblyCache/AssemblyNameCache.cs#L79
We decided to only write a record on every fifth iteration.

It's very simple (write a record on every fifth iteration). 👍  Just seems the compromise is not always the optimal.
May be possible to write asynchronously every 3-5 seconds?

@iSazonov Yeah. I'm just saying that it's a problem that's been there since the beginning. Skipping records doesn't solve all issues.

@oising This is not a criticism from me. I just asked your opinion on "async" since you have experience with the problem.

I did some research in codes and found that a partial solution already exists in Checkpoint-Computer
            if (recordType == ProgressRecordType.Processing)
            {
                TimeSpan timeSinceProgressWasWrittenLast = DateTime.UtcNow - lastTimeProgressWasWritten;
                if (timeSinceProgressWasWrittenLast < TimeSpan.FromMilliseconds(200))
                {
                    return;
                }
            }
            lastTimeProgressWasWritten = DateTime.UtcNow;

I used this for the test. Working call chain for WriteProgress is cmdlet.cs -> MshCommandRuntime.cs -> InternalHostUserInterface.cs -> ConsoleHostUserInterface.cs -> ConsoleHostUserInterfaceProgress.cs
In HandleIncomingProgressRecord (ConsoleHostUserInterfaceProgress.cs) I "mask" _progPane.Show(_pendingProgress);
            TimeSpan timeSinceProgressWasWrittenLast = DateTime.UtcNow - _lastTimeProgressWasWritten;
            if (timeSinceProgressWasWrittenLast > TimeSpan.FromMilliseconds(200))
            {
                _progPane.Show(_pendingProgress);
                _lastTimeProgressWasWritten = DateTime.UtcNow;
            }

Performance tests have shown that after this change, the difference between SilentlyContinue and Continue is less 3%.
I played with TimeSpan.FromMilliseconds(200)) and found that this value is optimal.
TimeSpan.FromMilliseconds(500)) is not giving a improvement, but the display becomes noticeably discrete.
If this is the correct place for the patch, I can do PR.
@daxian-dbw @lzybkr Could you comment please?

That's an interesting idea, I use a similar hack in PSReadline to minimize rendering calls.
There is one big problem with this approach - if we skip an update and don't see any progress updates for a long time, the current "hung" state would be misleading because the last update wasn't rendered.
If we could render updates on a timer, that might be ideal.

@lzybkr Thanks for comment!
I agree completely, and I will do the patch to call _progPane.Show(_pendingProgress); on a timer.

It turns out the timer fix causes some potential race conditions (see the issue #2800 and discussions in PR #2771). So the timer change was reverted with PR #2806 for now, and we need to different approach to address this problem.
It would be very helpful if someone can run a profiler on Invoke-WebRequest to get more accurate data for analysis.

@daxian-dbw
Maybe:
1.The timer set up a new  _progPaneUpdateFlag
2. HandleIncomingProgressRecord  check ``_progPaneUpdateFlag then call _progPane.Show(_pendingProgress) and reset the _progPaneUpdateFlag`
It seems that we can simply control the time delta in  HandleIncomingProgressRecord  , but my earlier tests showed that is significantly slower.

From @daxian-dbw

@iSazonov That is an interesting proposal -- by doing all the work in the same thread we certainly can avoid all race conditions. However, if  Write-Progress  is not called very frequently, then it's possible that the progress bar will not be updated ( Show  will not be called) for a long time.
Let's move this discussion to the issue, so more people can jump in 😄

@daxian-dbw

However, if Write-Progress is not called very frequently, then it's possible that the progress bar will not be updated ( Show will not be called) for a long time.

The hung problem was not full resolved by my PR too because Write-Progress is not called very frequently . The PR solved only a performance problem. I planned to solve this problem in a subsequent PR.
Actually there are three problems in one place, which might require different solutions:

Call HandleIncomingProgressRecord too often. There is no need to update the pane structures so often. My above proposal I believe is good enough to protect from this.
Hung of Freeze of pane. Write-Progress is not called very frequently user sees that the pane freeze. It may be a good solution to display a indicator updated by the same or other timer.
If Write-Progress show a progress pane and the script make output to console the user sees that the display shatters. These two processes should be merged  to form a single buffer for display.

@SteveL-MSFT Can you confirm that now web cmdlets performance is better with progress bar?

@iSazonov using same repro as above
With alpha.17:
TotalSeconds      : 65.3537138

with your fix:
TotalSeconds      : 10.9945305

Looks good!

Great! Many thanks to @daxian-dbw!

@iSazonov thank you for pushing the fix through!
There are 3 minor remaining issues and they are tracked by #3366

Not sure if it is of interest still, but the speed when run under ISE instead of console is about the same as running command without progress. So the ISE console does not exhibit the same issue, or different buffering is used

@NiclasLindgren  Thanks for your comment!
ISE is not in the PowerShell Core repo, it is not ported and no plans is.

Yes, my point was, that "powershell for windows" (v4.0) has the same problem, When run under ISE the "console" does not slow down the performance, I thought it might be interesting for those with access to ISE to check what they do. Either way, the solution discussed here helped and I am happy =)

@NiclasLindgren Welcome to test PowerShell Core Beta.4! We have some console fixes in last time from @lzybkr so you may be surprised how much the console is faster. Also we wait a feedback for loading and using Windows PowerShell modules in PowerShell Core Beta.4.

@niclaslindgren the long and short of it is that rendering progress bars in WPF is orders of magnitude fast than drawing progress bars in the windows console. It's not about the code path per-se.

@NiclasLindgren haven't peeked at the ISE code, but I suspect the progress bar is being rendered on a separate thread while the console uses the same thread

Hello folks!
I'm very sorry for digging up this old issue again, but I had to notice it still seem to be unresolved 6 years later.
I ran my test using Invoke-WebRequest to download a bigger file. With no additional flags or settings, the download speed gets limited to 2-3 MBit/s on my machine - compared to 100 MBit/s+ without the progress bar. This was tested in the stock powershell console delivered with the latest Win11 22H2 update (PSVersion: 5.1.22621.169).
As I read above, there is, or used to be a fix. Is it still there?

@max06 this has already been fixed in PowerShell. Windows PowerShell most likely will never get that fix.

@ThomasNieto Thank you for your quick response!
So, Windows Powershell is different from this one? I never knew that 😲
I'm also not using PS very often, a colleague reported this issue while setting up a new server - I'll forward your hint to him.


Package build failing due to sample code

Please describe the bug
We maintain a few nested projects in our swift-composable-architecture package, and they tend to always target the latest/greatest to demo new features. Recently, we added a macro dependency to one of these sample apps, but now the main build is failing, even though these targets are outside the main package:
https://swiftpackageindex.com/builds/136FD6AC-6A5C-486C-A3BB-5BA52D99533B
We do have an spi.yml specifying which schemes should be built, but I assume that even the presence of the other targets in the workspace are getting in the way.
Is there a way to specify just the project file and not the workspace in the spi.yml? Or is there another configuration option I'm not thinking of?

I just locked all the Swift versions to 5.9 to see if it helps. Will close if the build succeeds and if you think this is the right fix!

@finestructure Seems that bumping to 5.9 fixed a bunch! I'm a lil surprised by the visionOS failure, but think our Package.resolved file regressed. Hoping that'll get fixed soon!

@finestructure Actually, looks like locking to 5.9 is now locking compatibility as well:

(visionOS is failing incidentally, but I think I figured out the fix)
The package definitely builds and works in 5.7+... though the badge still says 5.7-5.9, but I think it's probably cached?

Oh, I misunderstood what you meant by "locking to 5.9". I thought you wanted it to be 5.9+ only (which should have puzzled me more than it did 😅).
I do believe the failures are actual failures. When building with Swift 5.8 I can see errors like
/Users/builder/builds/o86TiJKT/3/finestructure/swiftpackageindex-builder/spi-builder-workspace/Sources/swift-composable-architecture-benchmark/StoreSuite.swift:7:14: error: type 'Feature' does not conform to protocol 'Reducer'
  var store: StoreOf<Feature>!

in the logs which I can reproduce running locally both via
env DEVELOPER_DIR=/Applications/Xcode-14.3.1.app swift build

and in Xcode 14.3.1:

This is in Sources/swift-composable-architecture-benchmark - so not in the Examples.
I'll try older tags of TCA to see when this started failing.

One thing I notice is that on each build, Package.resolved changes and is pulling in newer dependencies that are locked. Running with
env DEVELOPER_DIR=/Applications/Xcode-14.3.1.app swift build --disable-automatic-resolution

to prevent this I find that 1.3.0 is the last tag that builds successfully with Xcode 14.3.1/Swift 5.8.

Ah, those are internal benchmarks that I didn't realize would build for the given target. I'll look into scoping that now.

to prevent this I find that 1.3.0 is the last tag that builds successfully with Xcode 14.3.1/Swift 5.8.

I guess I'm a little confused then how SPI was reporting success for 5.7 as recently as TCA 1.5.5?

That might be because I only tested with a similar command on my machine, not with the actual command that our builder uses. I'll try this again with 1.5.5 and the actual build command tomorrow just to confirm that this used to work!

Just to confirm: the iOS/5.8 build passes for 1.5.5 with the actual build command:

For reference, the full build command is:
env DEVELOPER_DIR=/Applications/Xcode-14.3.1.app xcrun xcodebuild -IDEClonedSourcePackagesDirPathOverride=$PWD/checkout/.dependencies -skipPackagePluginValidation -derivedDataPath $PWD/checkout/.derivedData build -scheme ComposableArchitecture -destination generic/platform=ios

I'm sure it was the missing -scheme parameter in my quick test above that made the difference.

Ok, looking at the diff 1.5.5...1.5.6 I think I see what happened. This change:

made the scheme config only apply for 5.9. Was there a reason you added these? I don't think you want any swift_version constraints in that file anymore (not even for the doc targets, 5.9 is now the default).
I'll propose a PR.

Ok, this isn't the whole story.  xcrun xcodebuild -resolvePackageDependencies also fails when running it with 5.8 against 1.5.6 and I need to investigate why.

The error is
xcodebuild: error: Could not resolve package dependencies:
  product 'DependenciesMacros' required by package 'tic-tac-toe' target 'AuthenticationClient' not found in package 'swift-dependencies'.

I'm seeing that same error when opening the project (on my patch-1 branch, which is just the .spi.yml change ahead of main at revision 6ed03a729ba5b17528ec6cea45b53ea472f281fb) in Xcode-14.3.1 (Swift 5.8).

The toc-tac-toe target looks a bit different in setup than the other examples in that it doesn't have an embedded swift-composable-architecture package with a Package.swift and a<EMAIL_ADDRESS>manifest. There's just a tic-tac-toe package in it which looks structurally different.
The problem seems to be the .product(name: "DependenciesMacros", package: "swift-dependencies") dependency but I tried removing it (and the associated import DependenciesMacros) but it then fails where @DependencyClient is being used.
I'm not sure how to make this work. I get the sense that perhaps Tic-tac-toe should be excluded from building the same way the other targets seem to be.
FWIW, if I just remove the target from the xcworkspace the xcodebuild -resolvePackageDependencies command works. So my guess is applying whatever it is that makes the other target "disappear" should fix this.
Does that help?

@finestructure The tic-tac-toe package is in the Examples/ directory, and is only there as a code sample. Our code samples all target 5.9 for simplicity and so that we can show off the latest/greatest features. We don't care to factor the Examples into SPI at all if possible. Is there a way to tell SPI to ignore the root xcworkspace (which is what brings the package and example projects together) and just build/run the package?

@stephencelis So normally you'd do that via the scheme: parameter in the .spi.yml file, which you have, and this would fix the build.
However, what we also do, as part of the processing, is run xcodebuild -resolvePackageDependencies and that doesn't take this parameter. I'll need to investigate if we can skip running package resolution if a scheme has been provided, which might help us avoid this problem.
What's interesting though is that Tic-tac-toe seems to be the only example tripping us up here. Perhaps there's some config that could be applied?
Another idea would be to set up a new ignore: directive or something that removes Examples from the checkout before running any of these commands.
Maybe we wouldn't even need a directive for this - there's probably no harm simply running rm -rf Examples in all package checkouts... 🤔 I mean what could possibly go wrong, right? 😆

Maybe we wouldn't even need a directive for this - there's probably no harm simply running rm -rf Examples in all package checkouts... 🤔 I mean what could possibly go wrong, right? 😆

In fact, let me try this. It's not a big change and in my local testing just now this would resolve the issue without further interventions (besides https://github.com/pointfreeco/swift-composable-architecture/pull/2679, that'll definitely need merging).
We can make it configurable later if we feel we need more control over this.

That's deployed and re-run:

Thanks for the background and for looking into this!


Why is jQuery being loaded?

I can't find the dependency. Is it Bootstrap?

Ah, that wasn't Peruse, it was a book's index.html.


Add arm64 support to install script

Although arm64 images were made available as a release, the install script would not pull them in automatically.

@davideshay would you be able to provide steps to manually to test this without having an ARM linux machine handy?

@davideshay we require that all commits on a PR are signed off as well as signed --can you update your commit? Shout out if you have any questions!

Commit was signed and signed off. Thanks.


added the solar time

I have added the solar time and tested it in my local.  I have changed the expected values to remove the errors. Could you confirm ? @MadhaviJain90

I don't see any differences from the Solar Time PR I had coded -> https://github.com/UrbanSystemsLab/climateiq-cnn/pull/170, except a minor alteration in the expected value of sin and cos of solar time. LGTM. Can be merged!

@MadhaviJain90
The difference is that time was handled incorrectly in your previous PR, (error when testing). This is the mainly edited part:
# Convert time variable to datetime objects
times = xarray.DataArray(
[
# Check if the value is a Unix timestamp (numeric type)
(
numpy.datetime64(
datetime.datetime.utcfromtimestamp(t).strftime(
"%Y-%m-%dT%H:%M:%S"
)
)
if isinstance(t, (int, float))
else numpy.datetime64("".join(t.astype(str)).replace("_", "T"))
)
for t in times.values
]
)
    print("Debug - Converted Times:", times)

    # Extract hours and minutes from the datetime objects separately
    utc_hours = times.dt.hour  # Extract the hour component
    utc_minutes = times.dt.minute  # Extract the minute component

    # Convert to fractional hours (hours + minutes/60)
    utc_hours_minutes = utc_hours + utc_minutes / 60.0

    print("Debug - UTC Hours and Minutes (in fraction):", utc_hours_minutes)


Autopeer

Codecov Report

Merging #70 (1dc4443) into prep-release (2670a0b) will decrease coverage by 3.59%.
The diff coverage is 12.36%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@               Coverage Diff                @@
##           prep-release      #70      +/-   ##
================================================
- Coverage         27.87%   24.29%   -3.59%
================================================
  Files                68       80      +12
  Lines              3408     4553    +1145
================================================
+ Hits                950     1106     +156
- Misses             2458     3447     +989

Impacted Files
Coverage Δ

src/django_peerctl/autopeer/__init__.py
0.00% <0.00%> (ø)

src/django_peerctl/autopeer/schema.py
0.00% <0.00%> (ø)

src/django_peerctl/autopeer/tasks.py
0.00% <0.00%> (ø)

src/django_peerctl/autopeer/workflow.py
0.00% <0.00%> (ø)

.../management/commands/peerctl_set_session_device.py
0.00% <0.00%> (ø)

src/django_peerctl/peer_session_workflow.py
0.00% <0.00%> (ø)

src/django_peerctl/rest/serializers/peerctl.py
0.00% <0.00%> (ø)

.../django_peerctl/rest/serializers/service_bridge.py
0.00% <0.00%> (ø)

src/django_peerctl/rest/views/peerctl.py
0.00% <0.00%> (ø)

src/django_peerctl/rest/views/service_bridge.py
0.00% <0.00%> (ø)

... and 16 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


[SPARK-25088][CORE][MESOS][DOCS] Update Rest Server docs & defaults.

What changes were proposed in this pull request?
(a) disabled rest submission server by default in standalone mode
(b) fails the standalone master if rest server enabled & authentication secret set
(c) fails the mesos cluster dispatcher if authentication secret set
(d) doc updates
(e) when submitting a standalone app, only try the rest submission first if spark.master.rest.enabled=true
otherwise you'd see a 10 second pause like
18/08/09 08:13:22 INFO RestSubmissionClient: Submitting a request to launch an application in spark://...
18/08/09 08:13:33 WARN RestSubmissionClient: Unable to connect to server spark://...
I also made sure the mesos cluster dispatcher failed with the secret enabled, though I had to do that on slightly different code as I don't have mesos native libs around.
How was this patch tested?
I ran the tests in the mesos module & in core for org.apache.spark.deploy.*
I ran a test on a cluster with standalone master to make sure I could still start with the right configs, and would fail the right way too.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2049/
Test PASSed.

Test build #94572 has started for PR 22071 at commit b4ca224.

Test build #94572 has finished for PR 22071 at commit b4ca224.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94572/
Test FAILed.

Test build #4241 has started for PR 22071 at commit b4ca224.

@tnachen

Test build #4244 has started for PR 22071 at commit b4ca224.

Test build #4244 has finished for PR 22071 at commit b4ca224.

This patch fails Spark unit tests.
This patch does not merge cleanly.
This patch adds no public classes.

Test build #4245 has started for PR 22071 at commit b4ca224.

Test build #4245 has finished for PR 22071 at commit b4ca224.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #94706 has started for PR 22071 at commit 897b587.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2149/
Test PASSed.

Test build #94706 has finished for PR 22071 at commit 897b587.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94706/
Test PASSed.

LGTM as well

+1

Merged to master

any objections about putting this in prior branches as well?

in this case maybe ok. perhaps just rel note this iff there's another 2.2.x or 2.1.x releases?

Hi @squito, if I may, I have a few comments to share regarding this change and probably you can help me to understand it in better details.
From  Spark Security Doc page it looks like spark.master.rest.enabled property belongs to Spark Standalone Master only. IMO, It makes sense, because it's the server side who is responsible for enabling and disabling REST endpoint according to the source code.
Now, from the [comment in the code] (https://github.com/apache/spark/blob/v2.4.0/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L176-L180) of SparkSubmit.scala it looks like REST-based gateway introduced in Spark 1.3 is the default way of communication with Spark Standalone Master with a fallback to legacy RPC.
And here's the problem: now spark.master.rest.enabled became a client-side property, not Spark Standalone Master property. So in order to communicate with Master, a user have to provide spark.master.rest.enabled with each job she submits because without it, the submission will fail. This is a pretty breaking change.
What is the reason to fail the start of MesosClusterDispatcher if spark.authenticate.secret is set? Mesos uses different mechanisms for authentication and there's no need to fail the Dispatcher - this property can be just ignored or logged at most.
TL;DR It looks like spark.master.rest.enabled should reside only in Spark Standalone Master code and shouldn't leak into the client side of SparkSubmit. Also, it can be completely ignored with Mesos Dispatcher, because REST interface is the only endpoint it exposes.
Please let me know what you think. Thanks.


Polymorphism and its types in C++

Description 📜
OOPs C++ Polymorphism and its types with example
Fixes #2694

Type of change 📝

[x] Documentation (Content Creation in the form of codes or tutorials)

Domain of Contribution 📊

[x] C/CPP

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

can u pls assign this ?
c/c++ batch 9

@Mitali-13 you cannot claim any issue through the pr section,plus someone else is working on it

@ChetasShree I think its resolved now completely if is there any correction left in this please let me know ? and thank you for reviewing

Please mention an index line in readme along with relative link as well, and the rest is good to go

Please mention an index line in readme along with relative link as well, and the rest is good to go
sorry but what is index line I mean i don't know about this


🐛 Bug ➾NodeJS v18 not supported

🚥 Status (Internal Taqueria Team Use Only)

[ ] 🔬 Investigated and Verified
[ ] ⚗ Solution Identified and Designed
[ ] 🧫 Dev Implementation of Fix
[ ] 🧪 Fix Tested and Validated
[ ] 🏆 PR Merged

🆘 What happened?
I tried to use the Taco-shop scaffold, but when it came time to start the dapp, I got a failure. I was using  node v18.4.0 (npm v8.12.1)
The Taqueria Scaffold requirement says  Node.js v16.17.1 or later, but this is to correct as node v18.4.0 (npm v8.12.1) fails with an error.  If I switch to   node v16.17.1 (npm v8.15.0) it works fine
🆘 Steps to Reproduce?

use nvm 18.4.0
start the dapp in the Taco-Shop Scaffold
error
switch to node v16.17.1
start the dapp it works fine
note docs say " Node.js v16.17.1 or later" which is shown above is not correct

🪵 Relevant log output
Error: error:0308010C:digital envelope routines::unsupported
    at new Hash (node:internal/crypto/hash:67:19)
    at Object.createHash (node:crypto:133:10)
    at module.exports (/home/mike/taco-shop/app/node_modules/webpack/lib/util/createHash.js:135:53)
    at NormalModule._initBuildHash (/home/mike/taco-shop/app/node_modules/webpack/lib/NormalModule.js:417:16)
    at handleParseError (/home/mike/taco-shop/app/node_modules/webpack/lib/NormalModule.js:471:10)
    at /home/mike/taco-shop/app/node_modules/webpack/lib/NormalModule.js:503:5
    at /home/mike/taco-shop/app/node_modules/webpack/lib/NormalModule.js:358:12
    at /home/mike/taco-shop/app/node_modules/loader-runner/lib/LoaderRunner.js:373:3
    at iterateNormalLoaders (/home/mike/taco-shop/app/node_modules/loader-runner/lib/LoaderRunner.js:214:10)
    at iterateNormalLoaders (/home/mike/taco-shop/app/node_modules/loader-runner/lib/LoaderRunner.js:221:10)
/home/mike/taco-shop/app/node_modules/react-scripts/scripts/start.js:19
  throw err;

### 🐘 How impactful is this bug?

🤔 Notable but managable

### ⏱ Prevalance

if they use npm 18+

### 💻 Operating System

Windows -> Linux VM

### 🕸 System Architecture

x64

### 🌮 Taqueria Version

v0.24.1

### 🌿 Node.js Version

v18.4.0

### ☎ Contact Details

<EMAIL_ADDRESS>
### Code of Conduct

- [X] I agree to follow this project's Code of Conduct

Fixed. NodeJS v20 is also supported.


[patch] fix channel length

LGTM

Codecov Report

Merging #21 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master     #21   +/-   ##
======================================
  Coverage    96.3%   96.3%
======================================
  Files          10      10
  Lines         569     569
======================================
  Hits          548     548
  Misses         12      12
  Partials        9       9

Impacted Files
Coverage Δ

service/role.go
97.23% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2f2a1f...520af68. Read the comment docs.


Call to undefined function RenokiCo\PhpK8s\Traits\Cluster\yaml_parse()

I got error while using php-k8s for laravel 8 (php7.4)
i'ts fixed with some changes in LoadsFromKubeConfig.php line 92
return $cluster->loadKubeConfigFromArray(Yaml::parse($yaml), $context);
i use
composer require pragmarx/yaml

Install the yaml php extension.


Job modifiers inherit icons in game, but are marked as missing by plugin

A possible enhancement:
Some modifiers such as job_<x>_add, where the corresponding job has an icon field set, use the icon corresponding to the name set in the field instead of the job's name.
Currently, the plugin will complain that the icon matching the modifier's name is missing, but this relation could potentially be inferred.
An example:
Here, the Iodizium Mining Drone has its icon set to the Iodizium Miner job, and in game the drone modifier uses the icon for job_giga_iodizium_miner_add.
The plugin warns that the icon matching the modifier name is missing, which is true, but that's not the icon which the game ultimately uses. It could potentially infer the relation from the job and use the other icon like it does in game.

As always, great work with the plugin, it helps our development work tremendously. Thank you for your efforts.

So the logic to get the actual modifier icon path should be optimized.
And it seems like there are some other 'inherit icons' situations.

Plugin now simply get the modifier icon path by template expression "gfx/interface/icons/modifiers/$.dds"， where "$" is the modifier name.
It seems like there should be some addtional inference logic (and are there others?),
and the icon file could also be a png file.

Some others:

Economic category modifiers use icons related to their resources (there's some nuance to this one, worth careful investigation, particularly for modifiers which don't point at a specific resource)
Scripted modifiers can specify an icon in their definition.

All of these (note the warnings in the scroll bar) have icons in-game

Not sure what's the icon generation rule for template-based modifier (e.g. shipsize_<ship_size>_build_speed_mult).


Streamlined API

I believe the API could be streamlined to support the common use cause of a business that wants to start receiving payments.
ie
//Final implementation
Daraja daraja = new Daraja.Builder() //could be injected or single instanced
                 .setConsumerKey("")
                  .setConsumerSecrtet("")
                    .setCallbackUrl("")
                    .setBusinessShortCode("")
                    .setAccountType(TransactionType.CustomerBuyGoodsOnline)
                    .setParty(""); //not sure if partyA or partyB

//simpler API requests that returns the DarajaListener
daraja.pay("phone", "amount", "description"); //creates an LNMExpress class from Daraja and does api request

Currently, LNMExpress class holds to much scope, some of which should be held in the Daraja class.
I am willing to create a PR to demonstrate the same

Hi @gilokimu
The PR is much welcome 💯

I have invited you as a collaborator on the project as well 👍 🚀

Thanks,
Maybe next weekend.

Awesome 🦅 🚀

Hey, forgot to respond here. I took a look at the API and wasn't sure how the other endpoints fit in context of android app, while I can still implement it for the sake, I did want to take a dev centric approach and not replicate the API given. I am still willing to take a stub at it and so any pointers will help.
TL;DR
How do business use the API endpoints declared?

The rest of the API's would not work on the Android SDK, as the callback URLs need to be specified, which can be a bit difficult because the phone IP could keep on changing depending on the network it is connected to. Ideally, if we could manage to solve the IP issue, we could have the whole sdk setup on this SDK

We might have to update the README when we are done with this PR

Yeah, and the common use case would be to have server side handle the callback. I will restructure the current API implementation then

That would be the most appropriate use case, yes

@gilokimu
I might have an idea we can look at. Do you see how Firebase works, i.e the Cloud Messaging or Remote Config for example, Firebase is able to get your device regardless of the connected network you are connected to. The way I see it, its like they have a way to ping your device i.e in a callback way. If we are able to look at this, we could make the whole SDK work in the Android SDK. Let me get in touch with you actually, but we can move on with the discussion here

I am using cloud functions to get a baseUrl and I am able to receive the callback from Mpesa, at the moment I am getting the error : "The ReceiverParty information is invalid" but this is a few minutes into it. The idea is to create a payment request on firebase as well and have the cloud function validate it - That answers that bit.

Code restructure see this PR #25 :

Daraja.Builder(Config.CONSUMER_KEY, Config.CONSUMER_SECRET)
                .setBusinessShortCode(Config.BUSINESS_SHORTCODE)
                .setPassKey(AppUtils.getPassKey())
                .setTransactionType(Config.ACCOUNT_TYPE)
                .setCallbackUrl(Config.CALLBACK_URL)
                .setEnvironment(Env.SANDBOX)
                .build();

And to make payment Request
 public DarajaListener<LNMResult> makePaymentRequest(
            String token,
            String phoneNumber,
            String amount,
            String accountReference,
            String description,
            final DarajaListener<LNMResult> listener
)

Still WIP - I do want to check the correct implementation for the access token and listeners
This is a very good guide : https://peternjeru.co.ke/safdaraja/ui/ and made things clear

Let me go through it

I think its better now. We should decide on way forward around the lib API class v2/Daraja.java and whether we should deprecate the old approach. I have also added the items I have been attempting into the project tab.

We can deprecate the v1, and roll out v2. We can merge all unfinished work, review and roll out the v2 update

Ok, I will create stable version tomorrow morning that you can merge. I will also review current open PR, I will keep some things in that PR - not all

A few merge requests later, I am done.
This is another breaking change for anyone who is using the earlier version. Just review and let me know

On it

A few merge requests later, I am done.
This is another breaking change for anyone who is using the earlier version. Just review and let me know

On it

A few merge requests later, I am done.
This is another breaking change for anyone who is using the earlier version. Just review and let me know

Merged PR #29

After 2 years, haha
Can we work on updating the documentation on the README? @gilokimu

Closing this issue, for being stale for a while now 😶🌫


actions: Add caching for Docker layers

Though their build processes may be different, I tested with Lykos and it brought a 2m30s build down to 34s with no code changes.
The benefit may be marginal at times but it should avoid wasted time for frequent commits that change little.

Out of curiosity, I rebuilt this specific PR to check what times we might be able to expect from cache.
The initial build took approximately 6m 46s, and unfortunately the rebuild seems to be similar. It did appear to finish in 5m 6s - unclear if this is a faster runner or caching helped out with some base layers, but bundle install appeared to be largely similar.
I'd hope in the future if we ever add other components this would help out, so +1 from me - thank you for integrating!

Out of curiosity, I rebuilt this specific PR to check what times we might be able to expect from cache.
The initial build took approximately 6m 46s, and unfortunately the rebuild seems to be similar. It did appear to finish in 5m 6s - unclear if this is a faster runner or caching helped out with some base layers, but bundle install appeared to be largely similar.
I'd hope in the future if we ever add other components this would help out, so +1 from me - thank you for integrating!

Not sure why the results would show that little difference, according to logs it did receive the cache but from some scrolling (on mobile) I couldn't see anywhere it was actually used. On Lykos builds there was a bunch of "CACHED" build steps but none here. Oh well.

Not sure why the results would show that little difference, according to logs it did receive the cache but from some scrolling (on mobile) I couldn't see anywhere it was actually used. On Lykos builds there was a bunch of "CACHED" build steps but none here. Oh well.


Patch Django to catch manage.py errors

We should be able to capture errors resulting from a crashing manage.py task, see #19

Just reread the code. That actually works out of the box, Django only swallows some of its own exceptions, not all of them.


Fixed #26903 -- Added support for date and timestamp containment with in postgres range field.

I prepared new PR based on #6885 due to inactivity. I also added tests for aware datetimes.

Use self.assertCountEqual to fix the test failures since ordering is undefined.

I rebased from master.

I used subTest() to make the tests less repetitive  -- see if that looks better. Are any negative, non-matching tests needed?

Thanks, tests are more DRY now, and you're right, we missed non-matching tests.

Do you want to add those additional test cases?

I added non-matching tests.


Changes to Production DB for the next release -MZ

Change Project_List_LU to Project_List_Bridge
Change Project_List to Project_List_LU
Add view -- dbo.ProjectList_Grid_VW

Add procedure: dbo.InsertIntoList
Add procedure: dbo.DeleteFromList

Change Project_List_LU to List
Add procedures (to keep in line with naming convention/try to establish one):
[sp_InsertProjectList]
[sp_DeleteProjectList]
Please use these to update the bridge table.  I've updated the insertlist/deletelist to point to the List table.

Update sp_DeleteProject so that if a proj is deleted, it is removed from the Project_List_Bridge

Bundle_Project_Map_VW
Bundle_Project_VW

sp_Insert_NewList
sp_DeleteList

Project_Main_VW_Unchanged-removed [feature_type], [shape], [shape_nad83] fields from view in dev, replicate in production

Added where sponsor status <> 'deleted' to all project/grid views, to project list grid vw, to two bundle vws.


How to send DOS_stats if destination is a local VS?

Hi,
I have the situation that the logging cluster gets loadbalanced through the very same BIG-IP which is exporting the data.
If I use the crontab method I can use the VS IP as destination. The approach with an external monitor is great, but does not work if the destination is a local virtual server :( Any other idea?
Greetings,
Eric

@sv3n-mu3ll3r

Eric,
I will look into it.
Thanks for the info. Cheers, sVen

solved


setup code climate

#37

@mohawkjohn can you please do the needful? I don't have sufficient privileges.


Automatic PR from RestAPI-PR1639

Generated from RestAPI PR: https://github.com/Azure/azure-rest-api-specs/pull/1639

Codecov Report

Merging #1421 into master will decrease coverage by 0.01%.
The diff coverage is 4%.

@@            Coverage Diff             @@
##           master    #1421      +/-   ##
==========================================
- Coverage   52.61%   52.59%   -0.02%
==========================================
  Files        3344     3344
  Lines       85048    85073      +25
==========================================
+ Hits        44746    44747       +1
- Misses      40302    40326      +24

Impacted Files
Coverage Δ

.../operations/virtual_network_gateways_operations.py
16.83% <4%> (-1.16%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fcb6e98...5fa5fe1. Read the comment docs.

PR from 2017-09-01 not generated yet


Documentation changes for num_fc_layers

The user guide documentation has typo errors as null for num_fc_layers. It should be zero throughout.
I have attached a screenshot below:

These changes should be corrected for other definitions as well

@prakass1 the default value for num_fc_layers is null because the user doesn't need to pick a number for it. Users are expected to either use num_fc_layers or fc_layers.
num_fc_layers being None indicates that the user didn't chose a value for it. Take a look at the explanation for these two arguments:
fc_layers (default null): it is a list of dictionaries containing the parameters of all the fully connected layers. The length of the list determines the number of stacked fully connected layers and the content of each dictionary determines the parameters for a specific layer. The available parameters for each layer are: fc_size, norm, activation, initializer and regularize. If any of those values is missing from the dictionary, the default one specified as a parameter of the encoder will be used instead. If both fc_layers and num_fc_layers are null, a default list will be assigned to fc_layers with the value [{fc_size: 512}, {fc_size: 256}]. (only applies if reduce_output is not null).
num_fc_layers (default null): if fc_layers is null, this is the number of stacked fully connected layers (only applies if reduce_output is not null).
For more details: https://uber.github.io/ludwig/user_guide/#parallel-cnn-encoder

@msaisumanth explained in detail, closing this.


Propagate NodeGroup tags to ASG

Is your feature request related to a problem?
The basic idea os the feature request is to be able to propagate the tags defined in a NodeGroup resource (talking about the EKS controller) to the ASG created to manage the node group.
The motivation behind it is to be able to scale the node group from 0. According to [1] and [2], when there's taints applied to the node group, it cannot be discovered from the Launch Configuration, so it is needed to tag the ASG accordingly (check [1] for a concise example).
[1] https://aws.github.io/aws-eks-best-practices/cluster-autoscaling/#scaling-from-0
[2] https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#auto-discovery-setup
Describe the solution you'd like
I want the exact tags I set in the NodeGroup definition to be propagated to the ASG group. Example:
Let's say I set the following tag in my node group definition:
tags:
  k8s.io/cluster-autoscaler/node-template/taint/spotInstance: "true:PreferNoSchedule"

I want the exact same tag to be applied to the resulting ASG after created.
Describe alternatives you've considered
Perhaps to have an ASG resource in the EC2 controller with some flag such as import: true to tell the controller the resource already exists so I just want to patch it.

I can understand how this would be applicable, but I think unfortunately this suggestion runs counter to one of the fundamentals of ACK. We designed the controllers so that an ACK controller only uses the SDK for whichever service it manages, and no others. That is, the EKS controller will only access EKS APIs. Your suggestion here would require that the EKS controller makes calls to the ASG APIs.
The only way I can envision this working is whereby we have a workflow that adopts the ASGs that were created and then we update their tags. This would require the ASG controller be installed, and that the ASG controller supports creating these resources. Currently we don't even have an ASG controller - so that is the first step.

@RedbackThomson if I understood correctly, there's no way to get this done, right? Because the EKS controller would need to make calls to the ASG controller and vice-versa.
Despite that, I believe this feature would be key to follow the EKS best practices (scaling from 0) and to save some good money sometimes xD. Do you have any other suggestion to get it going?

Because the EKS controller would need to make calls to the ASG controller and vice-versa.
Correct. Not with the tools provided by the ACK project.

Perhaps once the ASG controller supports the node group resources, some combination of adopting those resources and modifying their tags would be sufficient? That's probably my best guess for now.


fail on bin/check_license.sh errors

With the fix of #1671, bin/check_license.sh now succeeds on master.
This PR allows bin/linters.sh to fail with check_license.sh check,
so that this healthy status will continue.
What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

@jmuk: Adding do-not-merge/release-note-label-needed because the release note process has not been followed.

One of the following labels is required "release-note", "release-note-action-required", or "release-note-none".
Please see: https://github.com/kubernetes/community/blob/master/contributors/devel/pull-requests.md#write-release-notes-if-needed.
Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/release-note-none

/assign @yutongz

@jmuk: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
9cb2af3662bf65a48968fa84cf3bae2b4b74d6e0
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-pilot-e2e


Add support for johnny-cache

ATM, we have to blacklist the page_* tables in johnny-cache, as mptt uses custom queries to move nodes circumventing the cache invalidation of johnny-cache. It would be nice to add optional support for this along the following lines:
Try to import johnny, if not present create a no-op function for invalidation.
In https://github.com/feincms/feincms/blob/master/feincms/admin/tree_editor.py#L380 _move_node invalidate the cache for the page tables.
Would that be acceptable? If so I'll fork and patch.

Hi Tino
Can't this be solved in the underlying library, in django-mptt proper? I think a fix for this really belongs there, and not in FeinCMS.
If that isn't possible for some reason, we can still try offering hooks in FeinCMS so that it plays nicer with johnny-cache. Btw, we already have a similar hook -- _refresh_changelist_caches. If we convert this to a signal instead would this work for FeinCMS/johnny-cache too?
Thanks,
Matthias

That probably makes more sense. Filed it upstream: https://github.com/django-mptt/django-mptt/issues/165

Thanks!


 Calling setState() is illegal from within componentWillUpdate. Unfortunately, there's a bug (react#5097) which, if I read that correctly, will trigger an infinite loop when setState() is called from within componentDidUpdate but there is no change in state.

Yes! I had changed the code.

You should change your state to props,or call jquery plug-in here!


Prevent non-catchable errors from occurring when opening a box

Leaving hanging Futures that can raise Exceptions like happens to be nasty, because they will not be caught by a try-catch higher up the callstack. They can only get caught by zone-level error handling, which is not great.
This is necessary for my hive_multiisolate package to function with 100% safety (hopefully).

@TheOneWithTheBraid Are you handling PRs, or is misir still doing it ?

@themisir Sorry, tagging you due to not seeing any answers.

@themisir Sorry, tagging you due to not seeing any answers.

Np, I left a review message above. Since I didn't heard back from you, I have implemented it myself - https://github.com/hivedb/hive/commit/68b6e62921f0fced82f950ed363c2a415e78e7e6#diff-5405afd0b66478f672f1f6a400d2400403aac2b2993d57fb313a64aaa902cd47R115-R119 ;

@themisir Sorry, tagging you due to not seeing any answers.

Np, I left a review message above. Since I didn't heard back from you, I have implemented it myself - 68b6e62#diff-5405afd0b66478f672f1f6a400d2400403aac2b2993d57fb313a64aaa902cd47R115-R119 ;

Well that's weird, I don't see that anywhere. Can you grab a link to it so I can figure out why I can't find/see that ?

@themisir Also, I just tested your modification and it's not equivalent to this PR and doesn't fix the problem due to completeError :/

If it's not visible for you it might be my own mistake too - i haven't finished the review, thought you will see the comment anyway. I don't know if github does show comments before finishing the review or not.
Nonetheless, what's your current behavior? Does it work in an unexpected way, or there's something else?

Yeah, not seeing it. Learn something new everyday :)
You can run this test with hive's master branch to see the problem getting raised. completeError will create an exception that cannot be caught, which is why I used complete with a boolean (since we rethrow the error, no need to transmit that info through the completer, that would just raise the same error twice. A descriptive message of what happened is better, I feel, so that's what I did in this PR).
As for your review, it's indeed probably better to catch any errors raised by the closing of the box so that it's the original error that gets rethrown. Should I amend this PR ?

@themisir Tagging you :)

@TheOneWithTheBraid Could you consider this PR, please ? Many thanks !

Hey sorry for very late reply, I wonder what's the problem with the current implementation. I believe it should throw an error when openBox fails. What's the behavior you are expecting and what's actually happening?

Hey sorry for very late reply, I wonder what's the problem with the current implementation. I believe it should throw an error when openBox fails. What's the behavior you are expecting and what's actually happening?

The PR as-is does throw an error (see rethrow, line 127). The problem is that the exception created by completeError will create another exception that cannot be caught because it will be out of this async scope. To prevent this behavior, the PR does a normal complete that returns the value false, which allows the async scope that was waiting on the box to get opened to raise its own exception, catchable this time.
To see the behavior in action, run the test I mentioned here :)


Fix Import Errors, Unused Variables, and Undefined Variable Issues in Cloud Project Script

Summary:
This pull request addresses several issues identified in the test_wordcloud_cli.py and test_wordcloud.py scripts related to import errors, unused variables, and an undefined variable. The changes ensure that the code adheres to best practices and resolves the errors reported by the linters.
Issues Resolved:

Import Errors:

Issue: The matplotlib module was not recognized in the test_wordcloud_cli.py script.
Resolution: Added the missing matplotlib module to the requirements.txt file to ensure it is installed and available in the environment.

Unused Variables:

Issue 1: The local variable posargs in the test_main_passes_arguments_through function was not used.
Resolution: Replaced the unused variable posargs with _ to indicate that the variable is intentionally unused.
Issue 2: The local variable text in multiple functions was not used.
Resolution: Replaced the unused variable text with _ in the relevant functions to avoid confusion and comply with code quality standards.
Issue 3: The local variable args in the check_argument function was not used.
Resolution: Replaced the unused variable args with _ to follow best practices for handling unused variables.
Issue 4: The local variable image_file in the test_cli_writes_to_stdout function was not used.
Resolution: Replaced the unused variable image_file with _ to adhere to clean code principles.

Undefined Variable:

Issue: The variable unicode was not defined in the test_wordcloud.py script.
Resolution: Replaced the undefined unicode with str to ensure compatibility with Python 3, where unicode is no longer a built-in type.

Benefits:

Improved Code Quality:  By addressing unused variables and import errors, the code becomes cleaner and more maintainable.
Enhanced Compatibility: Fixing the undefined variable issue ensures that the script runs correctly in Python 3 environments.
Adherence to Best Practices: The updates align with best practices for handling unused variables and import dependencies.

Great work on addressing the identified issues! 🚀
Here’s a summary of what was done:

Fixed import errors by ensuring all necessary modules are included and available.
Replaced unused variables with _ to clean up the code and improve readability.
Corrected the undefined unicode variable issue to ensure compatibility with Python 3.


feat: Add support for building documentation via facets

This adds supports for building the documentation via a Lake facet as described in the README. The meta if is in place so that mathlib can still update a nightly version and have doc-gen breaking without its own build breaking in order for doc-gen not to becoome the critical path of a nightly update. Furthermore it also helps compile times as barely any users wants to actually use it.
The facet itself also has a huge performance advantage, on my local machine the old doc-gen4 from master with the unified (non parallel) build approach took 8-9 min to run. The new one that uses the full power of being parallelized by Lake on a per module basis only takes 2 minutes.
Furthermore once this PR is merged the CI of doc-gen4 should run through and deploy the first version that has (at least partial) support for linking back from the LeanInk rendered code into doc-gen pages.

The version incompatibilities between doc-gen and mathlib4 have existed for much longer than this already due to the doc-gen CI trying to build mathlib4 docs :D. Usually mathlib4 is the reason that I wait or push for the next version update on master so there isn't really a change with this, I've more or less come to terms with my dependency on mathlib4 already.
The idea of adding the optional flag was to make it so that this dependency is not bidirectional, i.e. mathlib4 can keep updating as they like without breaking their main build and doc-gen has to move with them at some point in time (or alternatively has already moved further and mathlib4 has not).
For making the CI on doc-gen's side work I've been thinking about substituting the require line to a local file system dependency inside of mathlib4s lakefile.lean inside of my CI though I do not know how brittle this might be.
Maybe @tydeu has an idea on how one might nicely do such a thing with lake?

mathlib4 can keep updating as they like without breaking their main build

Ah, then I understood this the wrong way.  I thought the plan was to enable CI for docs in both mathlib4 and doc-gen4.

bors merge


Notebook: Format Notebook

Issue Type: Feature Request
Notebook: Format Notebook not working in "file.ipynb "
jupyter notebook cell
notebook.format
{
"key": "ctrl+shift+i",
"command": "notebook.format",
"when": "!editorTextFocus && activeEditor == 'workbench.editor.notebook'"
}
Extension version: 2020.5.78807
VS Code version: Code 1.45.1 (5763d909d5f12fe19f215cbfdd29a91c0fa9208a, 2020-05-14T08:27:22.494Z)
OS version: Linux x64 5.3.0-51-generic

@AbdelrahmanHamdy1996
You're using a feature that's currently not supported.
The command you are trying to use is something tha'ts not hooked up with thye extension, please note the notebook functionality in VSCode hasn't been finalized either.
We are actively working with VSCode on this and you can follow up on our progress here https://github.com/microsoft/vscode-python/issues/10496

Thanks for your feedback. The VS Code notebook API is still in deep development. We will consider this as it moves along. Please keep track of it's state through:  https://github.com/microsoft/vscode/issues/91987.


Everything working but TrackEvent?

I have implemented everything exactly as shown from the Install section.
I can see events when I refresh my page in the app.  I am getting page hits, dependencies and all of the built in app insights data.
I wrote a very simple method to track a custom event on the main page that is already sending good page data to insights.  It executes with no error but I don't see any custom events in the app insights portal.  I also don't see this custom event being sent to appinsights in the developer network calls.  Have I missed a configuration step?
private async Task TrackEvent()
{
await AppInsights.TrackEvent("My Event");
await AppInsights.Flush();
}
protected override async Task OnInitializedAsync()
{
await TrackEvent();
}

To make this even weirder,  I just checked application insights again this morning.  I have been testing this app all morning.  There is now 1 of my custom events in application insights.  I have probably ran this app 100 times and have tons of data in my app insights portal.  I can't get another custom event to be sent and have no idea how that one got there.  I can refresh my initial page and see the page/depency data all show up in app insights within a few seconds.   The trackevent code from above is in my main index.razor page and gets called everytime the app loads.

I found out what is going on but still don't know how to fix my scenario. The index.razor page of my app contains that code shown at the top of this issue and nothing else.  If I open the app it navigates to index.razor,  OnInitializedAsync fires and calls the TrackEvent method.  The custom event is never sent to app insights but all other telementry is sent.  If I go to another page in my app and then navigate back to index.razor the custom event is sent along with all other telemetry.  If I manually refresh the index.razor page in the browser the Trackevent code is called but no custom event is sent. If I create a button on the page that calls the trackevent method the custom event is sent.

Try moving your code to the OnAfterRenderAsync event.


ubx: print relevent UBX-MON-VER output

I was looking for an easy way to track specific firmware version of a ublox module. There are better ways to do this and capture the information properly per GPS driver instance PX4 side, but this looked like a potentially harmless start. Thoughts?

when decoding UBX-MON-VER print full module, firmware, protocol version output

INFO  [gps] u-blox firmware version: SPG 3.01
INFO  [gps] u-blox protocol version: 18.00
INFO  [gps] u-blox module: NEO-M8N-0

Good starting point, but can you print it on a single line?
We can also enable errors and warnings from the gps.

Good starting point, but can you print it on a single line? We can also enable errors and warnings from the gps.

The structure is a bit awkward with each line coming from a separate call to payloadRxAddMonVer().
We could store these small strings, print a simple ublox summary after a successful configure, and maybe also expose it to the front end gps status.

Ah right. Feel free to merge as is then.

Thanks, I'll come back to this again. I also want to grab serial numbers per unit.


fix: updated UI according to design fixes

What does this PR do?

Changed title on Home page + added subtitle (Explore the Verax Ecosystem -> Explore the ways of getting the attestation + "Find out how you can get attestations from the following providers that you can use across the ecosystem” as a subtitle)
Changed width of container on Home page
Implemented click on <tr/> in tables body
Added icon for external links
Added Arbitrum + Arbitrum Goerli networks

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (2e71910) 93.63% compared to head (cda8cfd) 93.63%.

Additional details and impacted files
@@           Coverage Diff           @@
##              dev     #492   +/-   ##
=======================================
  Coverage   93.63%   93.63%
=======================================
  Files          20       20
  Lines         330      330
  Branches       76       76
=======================================
  Hits          309      309
  Misses         10       10
  Partials       11       11

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Feature Request: Text Customization Settings

Thank you for this nice extension. My request is for settings to control the style of the text, specifically the font size and the font family. In VS Code these are set via the editor.fontSize and editor.fontFamily parameters, and allow users to choose the font that is easiest for them to read.

Thank you for your comment.
The font size change has already been implemented in 30ebe3d2a8f8879cffa11126b5932d4e5ce956d1 and will be included in the next release.
As for font families, unlike VSCode that runs locally, cannot get the list of installed fonts due to browser security reasons.
I can implement it if you want to choose a generic font family ( https://developer.mozilla.org/en-US/docs/Web/CSS/font-family#generic-name ), but it's probably not what you want, right?

I think just the font size is more than enough! Thank you!

I was mistaken,
VSCode also specified the font family directly instead of selecting from the list of installed fonts.
I will try to see if I can implement a UI for text input in Monaco.

v0.1.6 was released with additional font size and font family settings.
Font families can be changed from the editor context menu.

Great work, thank you!


[BE] show dollars instead of cents in discount value

Coverage decreased (-0.08%) to 89.954% when pulling 6d5ae6363042db9b1078ba3e6a1c2431977cf4ff on feature/fix-discount-value into 4bf1c1d34b3f58e511cecb01a440708cce0324d6 on feature/coupons.

FYI the coverage dropped because of a new line in migration


Document using base box and dev branch

Purpose / why
It's not actually documented

that the islandora playbook's dev branch now defaults to the pre-built base box.
that the islandora playbook only spins up latest code if you're on the dev branch AND you change the base box.

Note: i think it gets a little long; do you think it should be less verbose? I want it to be simple for people to test PRs, especially, and that means successfully provisioning dev.
in part, this documents #179, and extends the documentation on the dev branch added in #180 to the "official" documentation.
What changes were made?
Reorganized a little of the installation section for clarity.
Verification
Try it? Ideally... i'm not sure if the pre-built box, which doesn't use ansible, can be launched with the vagrant file if you dont have ansible installed.
There are a number of things that seem illogical when written out.

since 'main' is the branch that is pinned to the most recent release, and 'dev' is the branch focused on getting a bleeding edge environment, the fact that dev defaults to the quick and easy evaluation base box while main uses the really long ansible provisioning, it seems a little backwards to me. I am having a hard time in this documentation describing when you would want to use the 'main" branch vs the 'dev' branch.
There was a section under the CentOS instructions that included something about vb-guest additions. I'm not sure if it means "if you are on a centos host you need to do this" or "if you want to spin up a centos guest you need to do this." Or both?

Interested Parties
@daniel-dgi

Checklist

Pull-request reviewer should ensure the following

[ ] Does this PR link to related issues?
[ ] Does the proposed documentation align with the Islandora Documentation Style Guide?
[ ] Are the changes accurate, useful, free of typos, etc?

Person merging should ensure the following

[ ] Does mkdocs still build successfully? (This is indicated by TravisCI passing. To test locally, and see warnings, see How To Build Documentation.)
[ ] If pages are renamed or removed, have all internal links to those pages been fixed?
[ ] If pages are added, have they been linked to or placed in the menu?
[ ] Did the PR receive at least one approval from a committer, and all issues raised have been addressed?

Wrong Danny tagged, I think. The current is @dannylamb

Git-fu accomplished; main is up to date, and this pr is now current to main. One file changed!
@Islandora/8-x-committers - please read and review the changes!


MaxListenersExceededWarning: Possible EventEmitter memory leak detected.

whenever i run the script i see the progress bar, but it gives an error 5 seconds in (node:22296) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 end listeners added. Use emitter.setMaxListeners() to increase limit and then it starts another ffmpeg prosses and the cycle repeats. I will attach a screen recording
https://user-images.githubusercontent.com/46354545/123453295-5e1dc580-d5ad-11eb-909a-443a0fa79064.mp4

it seems my screen recorder did not record properly


V4226p1

Register D.S.N. "4.4.312" and "5.4.312" on Office 365 as "networkerror".
Fix error message pattern in Sisimai::Reason::SecurityError.

Coverage increased (+0.1%) to 95.289% when pulling f2f6599caaa5bd68c40af1b4ebe3e75bcf19e39c on v4226p1 into 6962e98dfb3f369b9feca2d4fe014a04e6d646af on master.