Semaphores not working

Summary
What happened/what you expected to happen?
Running the semaphore example in the documentation (https://argoproj.github.io/argo/synchronization/) fails when tryring to look up the synchronization concurrency value from the provided configmap. The error message is
Failed to acquire the synchronization lock. Sync configuration key 'workflow' not found in ConfigMap

It should run without failure and use the value from the configmap to limit concurrency of the workflow execution.
Diagnostics
What Kubernetes provider are you using?
Docker Desktop Mac
What version of Argo Workflows are you running?
v2.11.5
For the configmap (kubectl -n argo create configmap my-config --from-file=my-config.yaml)
apiVersion: v1
kind: ConfigMap
metadata:
 name: my-config
data:
  workflow: "1"  # Only one workflow can run at given time in particular namespace
  template: "2"  # Two instance of template can run at a given time in particular namespace

For the workflow
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: synchronization-wf-level-
spec:
  entrypoint: whalesay
  synchronization:
    semaphore:
      configMapKeyRef:
        name: my-config
        key: workflow
  templates:
  - name: whalesay
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["hello world"]

Paste the logs from the workflow controller:
kubectl logs -n argo $(kubectl get pods -l app=workflow-controller -n argo -o name) | grep ${workflow}
time="2021-01-13T18:22:38Z" level=warning msg="Failed to acquire the lock" error="Sync configuration key 'workflow' not found in ConfigMap" key=argo/synchronization-wf-level-jscns
time="2021-01-13T18:22:38Z" level=info msg="Updated phase  -> Failed" namespace=argo workflow=synchronization-wf-level-jscns
time="2021-01-13T18:22:38Z" level=info msg="Updated message  -> Failed to acquire the synchronization lock. Sync configuration key 'workflow' not found in ConfigMap" namespace=argo workflow=synchronization-wf-level-jscns
time="2021-01-13T18:22:38Z" level=info msg="Marking workflow completed" namespace=argo workflow=synchronization-wf-level-jscns
time="2021-01-13T18:22:38Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=5164561 workflow=synchronization-wf-level-jscns
time="2021-01-13T18:23:46Z" level=warning msg="Failed to acquire the lock" error="Sync configuration key 'workflow' not found in ConfigMap" key=argo/synchronization-wf-level-24fv8
time="2021-01-13T18:23:46Z" level=info msg="Updated phase  -> Failed" namespace=argo workflow=synchronization-wf-level-24fv8
time="2021-01-13T18:23:46Z" level=info msg="Updated message  -> Failed to acquire the synchronization lock. Sync configuration key 'workflow' not found in ConfigMap" namespace=argo workflow=synchronization-wf-level-24fv8
time="2021-01-13T18:23:46Z" level=info msg="Marking workflow completed" namespace=argo workflow=synchronization-wf-level-24fv8
time="2021-01-13T18:23:46Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=5164682 workflow=synchronization-wf-level-24fv8
time="2021-01-13T18:24:16Z" level=warning msg="Failed to acquire the lock" error="Sync configuration key 'workflow' not found in ConfigMap" key=argo/synchronization-wf-level-th45v
time="2021-01-13T18:24:16Z" level=info msg="Updated phase  -> Failed" namespace=argo workflow=synchronization-wf-level-th45v
time="2021-01-13T18:24:16Z" level=info msg="Updated message  -> Failed to acquire the synchronization lock. Sync configuration key 'workflow' not found in ConfigMap" namespace=argo workflow=synchronization-wf-level-th45v
time="2021-01-13T18:24:16Z" level=info msg="Marking workflow completed" namespace=argo workflow=synchronization-wf-level-th45v
time="2021-01-13T18:24:16Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=5164737 workflow=synchronization-wf-level-th45v
time="2021-01-13T18:56:49Z" level=warning msg="Failed to acquire the lock" error="Sync configuration key 'workflow' not found in ConfigMap" key=argo/synchronization-wf-level-hkh89
time="2021-01-13T18:56:49Z" level=info msg="Updated phase  -> Failed" namespace=argo workflow=synchronization-wf-level-hkh89
time="2021-01-13T18:56:49Z" level=info msg="Updated message  -> Failed to acquire the synchronization lock. Sync configuration key 'workflow' not found in ConfigMap" namespace=argo workflow=synchronization-wf-level-hkh89
time="2021-01-13T18:56:49Z" level=info msg="Marking workflow completed" namespace=argo workflow=synchronization-wf-level-hkh89
time="2021-01-13T18:56:49Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=5168627 workflow=synchronization-wf-level-hkh89

Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

@sarabala1979 should people be testing with v2.12.4?

@sarabala1979 should people be testing with v2.12.4?

@sarabala1979 should people be testing with v2.12.4?

Just tested with v2.12.4 and it fails the same way. Perhaps an issue with Docker Desktop on Mac?

@sarabala1979 should people be testing with v2.12.4?

Just tested with v2.12.4 and it fails the same way. Perhaps an issue with Docker Desktop on Mac?

FWIW I can use see the configmap with kubectl and it has the right values
kubectl -n argo describe configmap my-config
Name:         my-config
Namespace:    argo
Labels:       <none>
Annotations:  <none>

Data
====
semaphore.yaml:
----
apiVersion: v1
kind: ConfigMap
metadata:
 name: my-config
data:
  workflow: "1"  # Only one workflow can run at given time in particular namespace
  template: "2"  # Two instance of template can run at a given time in particular namespace
Events:  <none>

FWIW I can use see the configmap with kubectl and it has the right values
kubectl -n argo describe configmap my-config
Name:         my-config
Namespace:    argo
Labels:       <none>
Annotations:  <none>

Data
====
semaphore.yaml:
----
apiVersion: v1
kind: ConfigMap
metadata:
 name: my-config
data:
  workflow: "1"  # Only one workflow can run at given time in particular namespace
  template: "2"  # Two instance of template can run at a given time in particular namespace
Events:  <none>

@indiejames are you submitting the in the same namespace?  can you provide the problem workflow  yaml using kubectl get wf <name> -o yaml?

@indiejames are you submitting the in the same namespace?  can you provide the problem workflow  yaml using kubectl get wf <name> -o yaml?

Yes, it's the same namespace.  Sync configuration key 'workflow'
    not found in ConfigMap
  phase: Failed
  startedAt: "2021-01-13T21:58:42Z"

Yes, it's the same namespace.  Sync configuration key 'workflow'
    not found in ConfigMap
  phase: Failed
  startedAt: "2021-01-13T21:58:42Z"

it looks like your env problem. I am able to run the workflow with the above CM and WF.
Name:                synchronization-wf-level-zn797
Namespace:           argo
ServiceAccount:      default
Status:              Succeeded
Conditions:
 Completed           True
Created:             Wed Jan 13 17:46:12 -0800 (7 seconds ago)
Started:             Wed Jan 13 17:46:12 -0800 (7 seconds ago)
Finished:            Wed Jan 13 17:46:19 -0800 (now)
Duration:            7 seconds
ResourcesDuration:   4s*(1 cpu),2s*(100Mi memory)

STEP                               TEMPLATE  PODNAME                         DURATION  MESSAGE
 ✔ synchronization-wf-level-zn797  whalesay  synchronization-wf-level-zn797  5s

it looks like your env problem. I am able to run the workflow with the above CM and WF.
Name:                synchronization-wf-level-zn797
Namespace:           argo
ServiceAccount:      default
Status:              Succeeded
Conditions:
 Completed           True
Created:             Wed Jan 13 17:46:12 -0800 (7 seconds ago)
Started:             Wed Jan 13 17:46:12 -0800 (7 seconds ago)
Finished:            Wed Jan 13 17:46:19 -0800 (now)
Duration:            7 seconds
ResourcesDuration:   4s*(1 cpu),2s*(100Mi memory)

STEP                               TEMPLATE  PODNAME                         DURATION  MESSAGE
 ✔ synchronization-wf-level-zn797  whalesay  synchronization-wf-level-zn797  5s

I repeated the instructions from the description and submitted a workflow via the UI and saw the same error.
This is in AWS using EKS with Argo version 2.11.5.

I repeated the instructions from the description and submitted a workflow via the UI and saw the same error.
This is in AWS using EKS with Argo version 2.11.5.

Tried v2.12.5 and still seeing the same behavior.

Tried v2.12.5 and still seeing the same behavior.

I repeated the instructions from the description and submitted a workflow via the UI and saw the same error.
This is in AWS using EKS with Argo version 2.11.5.

@chris-durbin Please make sure you are submitting to the same namespace from UI.

I repeated the instructions from the description and submitted a workflow via the UI and saw the same error.
This is in AWS using EKS with Argo version 2.11.5.

@chris-durbin Please make sure you are submitting to the same namespace from UI.

time="2021-01-13T18:22:38Z" level=info msg="Updated message  -> Failed to acquire the synchronization lock. Sync configuration key 'workflow' not found in ConfigMap" namespace=argo workflow=synchronization-wf-level-jscns
time="2021-01-13T18:22:38Z" level=info msg="Marking workflow completed" namespace=argo workflow=synchronization-wf-level-jscns

Based above log, controller able to find configmap but it doesn't find the workflow.  Can you provide live configmap using kubectl get cm my-config -o yaml

time="2021-01-13T18:22:38Z" level=info msg="Updated message  -> Failed to acquire the synchronization lock. Sync configuration key 'workflow' not found in ConfigMap" namespace=argo workflow=synchronization-wf-level-jscns
time="2021-01-13T18:22:38Z" level=info msg="Marking workflow completed" namespace=argo workflow=synchronization-wf-level-jscns

Based above log, controller able to find configmap but it doesn't find the workflow.  Can you provide live configmap using kubectl get cm my-config -o yaml

Tested again with the same error message.
I'm using the 'argo' namespace when I create the ConfigMap and the same 'argo' namespace when I submit the workflow.

Tested again with the same error message.
I'm using the 'argo' namespace when I create the ConfigMap and the same 'argo' namespace when I submit the workflow.

@chris-durbin can you provide the output of kubectl get cm my-config -o yaml -n argo?  This feature is working as expected in locally and in our environment also. Can we have a quick zoom call?

@chris-durbin can you provide the output of kubectl get cm my-config -o yaml -n argo?  This feature is working as expected in locally and in our environment also. Can we have a quick zoom call?

@indiejames @chris-durbin  thanks for the support. Found the issue configmap creation in @indiejames env and fix it.
Entire configamp yaml add as the value in this configmap.
Not working  Configmap:
apiVersion: v1
data:
  my-config.yaml: |
    apiVersion: v1
    kind: ConfigMap
    metadata:
     namespace: argo
     name: my-config
    data:
      workflow: "1"  # Only one workflow can run at given time in particular namespace
      template: "2"  # Two instance of template can run at a given time in particular namespace
kind: ConfigMap

Working Configmap:
apiVersion: v1
kind: ConfigMap
metadata:
 name: my-config
 namespace: argo
data:
  workflow: "1"  # Only one workflow can run at given time in particular namespace
  template: "3"

@indiejames @chris-durbin  thanks for the support. Found the issue configmap creation in @indiejames env and fix it.
Entire configamp yaml add as the value in this configmap.
Not working  Configmap:
apiVersion: v1
data:
  my-config.yaml: |
    apiVersion: v1
    kind: ConfigMap
    metadata:
     namespace: argo
     name: my-config
    data:
      workflow: "1"  # Only one workflow can run at given time in particular namespace
      template: "2"  # Two instance of template can run at a given time in particular namespace
kind: ConfigMap

Working Configmap:
apiVersion: v1
kind: ConfigMap
metadata:
 name: my-config
 namespace: argo
data:
  workflow: "1"  # Only one workflow can run at given time in particular namespace
  template: "3"


Upgraded node version from 14.16 to 16.14.0

Description
Changes include upgrading the node version from from 14.16 to 16.14.0.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #185
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[x] Created/updated tests
[x] Extended the documentation

Merged!


HOTT-1313: Fix intermittent geographical area failures

Jira link
https://transformuk.atlassian.net/browse/HOTT-1313
What?
I have added/removed/altered:

[x] Added spec cache clear before every geographical area controller request

Why?
I am doing this because:

We seem to get the wrong response from the cache intermittently

So for some background, I was able to reproduce (though intermittently) this issue by running the specs for the geographical areas controller 50 times. When I added the cache clearing the spec failures went away.


PRIMARY KEY not supported error is required

PRIMARY KEY is not supported yet.
However GlueSQL allows CREATE TABLE SingleItem (id INTEGER PRIMARY KEY) but do nothing for PRIMARY KEY attr.
Until the feature is implemented, it should be an error.
It is expected to play with codes in https://github.com/gluesql/gluesql/tree/main/src/executor/alter

I'll try it!


Increase context timeout

Faced issue where the context timed out without the decryption process,
Reproduced that error, by adding sleep before calling https://github.com/IBM/ibm-csi-common/blob/master/pkg/utils/get_key.go#L127
Got error in the controller,
prankul@Prankuls-MacBook-Pro ibm-vpc-block-csi-driver % kubectl logs -n kube-system ibm-vpc-block-csi-controller-0 iks-vpc-block-driver
{"level":"info","timestamp":"2021-11-19T12:28:04.417Z","caller":"cmd/main.go:84","msg":"IBM CSI driver version","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","DriverVersion":"vpcBlockDriver-"}
{"level":"info","timestamp":"2021-11-19T12:28:04.417Z","caller":"cmd/main.go:85","msg":"Controller Mutex Lock enabled","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","LockEnabled":false}
{"level":"info","timestamp":"2021-11-19T12:28:04.417Z","caller":"ibmcloudprovider/volume_provider.go:48","msg":"NewIBMCloudStorageProvider-Reading provider configuration...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:04.417Z","caller":"config/config.go:63","msg":"parsing conf file","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","confpath":"/etc/storage_ibmc/slclient.toml"}
{"level":"warn","timestamp":"2021-11-19T12:28:04.417Z","caller":"ibmcloudprovider/volume_provider.go:60","msg":"Failed to parse VPC_API_VERSION, setting default value","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:04.417Z","caller":"ibmcloudprovider/volume_provider.go:65","msg":"Fetching clusterInfo","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:04.418Z","caller":"ibmcloudprovider/volume_provider.go:72","msg":"Fetched clusterInfo..","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:04.418Z","caller":"ibmcloudprovider/volume_provider.go:75","msg":"Creating NewAPIKeyImpl...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:04.418Z","caller":"ibmcloudprovider/volume_provider.go:81","msg":"Created NewAPIKeyImpl...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:04.418Z","caller":"utils/get_key.go:95","msg":"Creating GRPC client","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:04.418Z","caller":"utils/get_key.go:98","msg":"Dialing for connection..","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:04.423Z","caller":"utils/get_key.go:129","msg":"VPC G2 APIKey decrpyting...","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"info","timestamp":"2021-11-19T12:28:06.423Z","caller":"utils/get_key.go:133","msg":"VPC G2 APIKey decrpyted","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver"}
{"level":"fatal","timestamp":"2021-11-19T12:28:06.423Z","caller":"ibmcloudprovider/volume_provider.go:84","msg":"Unable to get API key","name":"ibm-vpc-block-csi-driver","CSIDriverName":"IBM VPC block driver","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}

While logs for secret-sidecar had logs,
prankul@Prankuls-MacBook-Pro ibm-vpc-block-csi-driver % kubectl logs -n kube-system ibm-vpc-block-csi-controller-0 storage-secret-sidecar
{"level":"info","timestamp":"2021-11-19T12:27:38.738Z","caller":"server/server.go:122","msg":"nonBlockingGRPCServer-serve...","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider","Endpoint":"unix:/sidecardir/provider.sock"}
{"level":"info","timestamp":"2021-11-19T12:27:38.738Z","caller":"server/server.go:76","msg":"nonBlockingGRPCServer-Setup...","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider","Endpoint":"unix:/sidecardir/provider.sock"}
{"level":"info","timestamp":"2021-11-19T12:27:38.738Z","caller":"server/server.go:105","msg":"Start listening GRPC Server","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider","Scheme":"unix","Addr":"/sidecardir/provider.sock"}
{"level":"info","timestamp":"2021-11-19T12:27:38.738Z","caller":"server/server.go:128","msg":"Listening GRPC server for connections","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider","Addr":{"Name":"/sidecardir/provider.sock","Net":"unix"}}
{"level":"info","timestamp":"2021-11-19T12:27:39.636Z","caller":"server/container_apikey.go:31","msg":"In GetContainerAPIKey()","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider"}
{"level":"info","timestamp":"2021-11-19T12:27:39.636Z","caller":"server/container_apikey.go:32","msg":"Input","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider","Cipher":"8biVd3Aj93XARetNwgPsTClFPDkAIdFdGBhmJ2cz2zitY8GFnPI0098EsO/xz/hG0U72rv+NmwYBtrCUodKAPA=="}
{"level":"info","timestamp":"2021-11-19T12:27:39.637Z","caller":"decryptor/decryptor.go:79","msg":"Successfully fetched ClusterID..","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider","ClusterID":"c674k2t20iq9c9sem4ug"}
{"level":"info","timestamp":"2021-11-19T12:27:39.637Z","caller":"decryptor/decryptor.go:104","msg":"Successfully decrypted cipher","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider"}
{"level":"info","timestamp":"2021-11-19T12:27:42.401Z","caller":"server/container_apikey.go:31","msg":"In GetContainerAPIKey()","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider"}
{"level":"info","timestamp":"2021-11-19T12:27:42.401Z","caller":"server/container_apikey.go:32","msg":"Input","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider","Cipher":"8biVd3Aj93XARetNwgPsTClFPDkAIdFdGBhmJ2cz2zitY8GFnPI0098EsO/xz/hG0U72rv+NmwYBtrCUodKAPA=="}
{"level":"info","timestamp":"2021-11-19T12:27:42.401Z","caller":"decryptor/decryptor.go:79","msg":"Successfully fetched ClusterID..","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider","ClusterID":"c674k2t20iq9c9sem4ug"}
{"level":"info","timestamp":"2021-11-19T12:27:42.401Z","caller":"decryptor/decryptor.go:104","msg":"Successfully decrypted cipher","Name":"armada-storage-secret","ProviderName":"armada-storage-secret-provider"}

Call for VPC Api key decryption didn't come as the context timed out after 1 second

Have changed the timeout to 5 min,
Testing similar changed, and added delay of 30 seconds in the VPC G2 APi call.


add target variable to templating environment

this would allow the user to do things like this in their models:
select *
from snowplow.sessions
{ if target == dev }
where session_start_timestamp between 1 and 7 days ago
{ end }

there is a ton of value to be had out of an extremely simple environment variable, this is just one example.
make sure to add this usage to the best practices guide in the help docs once this feature is added.

This is a good idea and it should be super quick to add. I think that in practice, relying on the name of the target is a bad idea. Different people within the same org can have different target names b/c it's defined in the profiles.yml file. Moreover, there's no way this would work reliably in a package management world.
If the divide is dev/prod, should we add an environment var to the target? That sounds like a more dependable way to accomplish what you're looking for

We talked about this yesterday. I'm still not sure I agree with you. Rails has database.yml and the three entries in there (development, test, production) are standard across rails deployments. What is the thing that is preventing us from doing the same? If target isn't that, then I think we should change it to that. People don't need arbitrary control over this, I think we should be opinionated about what the correct usage is here. I would be ok going so far as to validate acceptable values. You can name your profile whatever you want, but your target should be dev/prod/test. If that's not what you're thinking target is, then let's talk further.


[PE-D][Tester D] Email attribute doesn't match definition of valid emails

Referring to the wikipedia [article] (https://en.wikipedia.org/wiki/Email_address) on emails, the santisation check on email.
For instance,  the example<EMAIL_ADDRESS>is an invalid email, but it should be a valid email:

Labels: severity.Low type.FunctionalityBug
original: ZShunRen/ped#12

Good catch! But I think it is reasonable to expect that user only comes across the unquoted email format (which is also typically the only format accepted by email service providers).

Closed since not in scope.


updated how-to-use-openai-in-azure-ml.md

I have updated the "how-to-use-openai-in-azure-ml.md" with new screenshots  and also modified the text based on the newest UI.

@swatig007
Can you review the proposed changes? Note that the validation warnings must be resolved before merging. The following new file names also need a hyphen before the number "2":

articles/machine-learning/media/how-to-use-openai-models-in-azure-ml/finetune-parameters2.png
articles/machine-learning/media/how-to-use-openai-models-in-azure-ml/model-catalog2.png

When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#assign-reviewer:nagau

@nagau - can you help us advance (or close out) this aging PR that was directed to you (https://github.com/MicrosoftDocs/azure-docs/pull/115759#issuecomment-1753492561).
Thanks, PR review team

@ssalgadodev
Can you review this old PR and determine whether it needs to be closed or merged?
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge. If the changes no longer apply, add an explanation and a #please-close comment to close the PR.
@MicrosoftDocs/public-repo-pr-review-team

I sent @ssalgadodev a Teams message today.
@MicrosoftDocs/public-repo-pr-review-team

#please-close

I believe this pr is a duplicate of this one that has been merged: https://github.com/MicrosoftDocs/azure-docs-pr/pull/254387/files


Jupyter notebook test

First version of jupyter notebook test as discussed in #485

There are still the following issues:

the path (NBDIR) is hard wired and maybe wrong
bazel integration
is this the right place for this file?

I think the hardwired NBDIR should not be a problem since the folder should not move around after they included it into the docs generator. (see this comment )
I am not quite sure if we want to put this into the bazel testing pipeline since it is kind of self contained. We could add it to the travis pipeline though. @seanpmorgan Maybe we can add another test after the bazel build pipeline and before the build.
@PhilipMay do you know if this testing environment runs !pip install lalala arguments?

@PhilipMay do you know if this testing environment runs !pip install lalala arguments?

Yes it runs pip this way. I uninstalled tensorflow with pip, started make tutorial_test and then checked if tensorflow got installed. It was...

From my point of view this is done. These steps still miss:

integrate to travis - do this in other PR?
the test now shows errors so fix the notebooks - also separate PR?

Thanks @PhilipMay! I'm seeing several notebooks fail for DeadKernelError: Kernel died possibly from timeout. Could you increase the timeout for the tests and see if it fixes it.

@seanpmorgan I can see no DeadKernelError: Kernel died messages in my log (see below). The timeout is already increased from 600 (default) to 2100. Can you maybe post your messages? Where do these differences come from?
FAIL: test_optimizers_lazyadam (testipynb.testipynb.NbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/testipynb/testipynb.py", line 96, in test_func
    assert passing, msg
AssertionError:

 ... optimizers_lazyadam FAILED

SyntaxError in cell [2]
-----------
!pip install tensorflow-gpu==2.0.0rc0
!pip install tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function, unicode_literals

import tensorflow as tf
import tensorflow_addons as tfa
import tensorflow_datasets as tfds
import numpy as np
from matplotlib import pyplot as plt
-----------

======================================================================
FAIL: test_template (testipynb.testipynb.NbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/testipynb/testipynb.py", line 96, in test_func
    assert passing, msg
AssertionError:

 ... template FAILED

NameError in cell [5]
-----------
result = model(tf.constant(np.random.randn(10,5), dtype = tf.float32)).numpy()

print("min:", result.min())
print("max:", result.max())
print("mean:", result.mean())
print("shape:", result.shape)
-----------

----------------------------------------------------------------------
Ran 6 tests in 94.164s

FAILED (failures=6)
make: *** [tutorials-test] Error 1
(tf2) mikes-MacBook:addons mike$ clear

(tf2) mikes-MacBook:addons mike$ make tutorials-test
python tools/ci_testing/tutorials_test.py

--------------------- Testing image_ops.ipynb ---------------------
/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/jupyter_client/session.py:371: DeprecationWarning: Session._key_changed is deprecated in traitlets 4.1: use @observe and @unobserve instead.
  def _key_changed(self):

 ... image_ops FAILED

SyntaxError in cell [2]
-----------
!pip install -q tensorflow-gpu==2.0.0rc0
!pip install -q tensorflow-addons~=0.5

from __future__ import absolute_import, division, print_function, unicode_literals

import numpy as np
import tensorflow as tf
import tensorflow_addons as tfa

import matplotlib.pyplot as plt
-----------

----------------- >> begin Traceback << -----------------

  File "<ipython-input-2-b46ec5485964>", line 7
    import tensorflow as tf
           ^
SyntaxError: from __future__ imports must occur at the beginning of the file

----------------- >> end Traceback << -----------------

F
--------------------- Testing layers_normalizations.ipynb ---------------------

 ... layers_normalizations FAILED

SyntaxError in cell [2]
-----------
!pip install -q tensorflow==2.0.0rc0
!pip install -q tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function
import tensorflow as tf
import tensorflow_addons as tfa
-----------

----------------- >> begin Traceback << -----------------

  File "<ipython-input-2-ea1d9ed991dc>", line 6
SyntaxError: from __future__ imports must occur at the beginning of the file

----------------- >> end Traceback << -----------------

F
--------------------- Testing layers_weightnormalization.ipynb ---------------------

 ... layers_weightnormalization FAILED

SyntaxError in cell [2]
-----------
!pip install tensorflow-gpu==2.0.0rc0
!pip install tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function, unicode_literals

import tensorflow as tf
import tensorflow_addons as tfa
import numpy as np
from matplotlib import pyplot as plt
-----------

----------------- >> begin Traceback << -----------------

  File "<ipython-input-2-6458efa20296>", line 6
    import tensorflow_addons as tfa
           ^
SyntaxError: from __future__ imports must occur at the beginning of the file

----------------- >> end Traceback << -----------------

F
--------------------- Testing losses_triplet.ipynb ---------------------

 ... losses_triplet FAILED

SyntaxError in cell [2]
-----------
!pip install -q tensorflow-gpu==2.0.0rc0
!pip install -q tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function, unicode_literals

import io
import numpy as np
import tensorflow as tf
import tensorflow_addons as tfa
import tensorflow_datasets as tfds
from matplotlib import pyplot as plt
from google.colab import files
-----------

----------------- >> begin Traceback << -----------------

  File "<ipython-input-2-aeb553bdcf4f>", line 6
    import numpy as np
           ^
SyntaxError: from __future__ imports must occur at the beginning of the file

----------------- >> end Traceback << -----------------

F
--------------------- Testing optimizers_lazyadam.ipynb ---------------------

 ... optimizers_lazyadam FAILED

SyntaxError in cell [2]
-----------
!pip install tensorflow-gpu==2.0.0rc0
!pip install tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function, unicode_literals

import tensorflow as tf
import tensorflow_addons as tfa
import tensorflow_datasets as tfds
import numpy as np
from matplotlib import pyplot as plt
-----------

----------------- >> begin Traceback << -----------------

  File "<ipython-input-2-444c83055f39>", line 6
    import tensorflow_addons as tfa
           ^
SyntaxError: from __future__ imports must occur at the beginning of the file

----------------- >> end Traceback << -----------------

F
--------------------- Testing template.ipynb ---------------------
2019-09-25 16:20:27.875261: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-09-25 16:20:27.899003: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fd57d5aa130 executing computations on platform Host. Devices:
2019-09-25 16:20:27.899078: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/logging/__init__.py", line 1945, in shutdown
    h.flush()
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/absl/logging/__init__.py", line 891, in flush
    self._current_handler.flush()
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/absl/logging/__init__.py", line 785, in flush
    self.stream.flush()
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/ipykernel/iostream.py", line 341, in flush
    if self.pub_thread.thread.is_alive():
AttributeError: 'NoneType' object has no attribute 'thread'

 ... template FAILED

NameError in cell [5]
-----------
result = model(tf.constant(np.random.randn(10,5), dtype = tf.float32)).numpy()

print("min:", result.min())
print("max:", result.max())
print("mean:", result.mean())
print("shape:", result.shape)
-----------

----------------- >> begin Traceback << -----------------

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-5-3ed927615e9f> in <module>
----> 1 result = model(tf.constant(np.random.randn(10,5), dtype = tf.float32)).numpy()
      2
      3 print("min:", result.min())
      4 print("max:", result.max())
      5 print("mean:", result.mean())

NameError: name 'np' is not defined

----------------- >> end Traceback << -----------------

F
======================================================================
FAIL: test_image_ops (testipynb.testipynb.NbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/testipynb/testipynb.py", line 96, in test_func
    assert passing, msg
AssertionError:

 ... image_ops FAILED

SyntaxError in cell [2]
-----------
!pip install -q tensorflow-gpu==2.0.0rc0
!pip install -q tensorflow-addons~=0.5

from __future__ import absolute_import, division, print_function, unicode_literals

import numpy as np
import tensorflow as tf
import tensorflow_addons as tfa

import matplotlib.pyplot as plt
-----------

======================================================================
FAIL: test_layers_normalizations (testipynb.testipynb.NbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/testipynb/testipynb.py", line 96, in test_func
    assert passing, msg
AssertionError:

 ... layers_normalizations FAILED

SyntaxError in cell [2]
-----------
!pip install -q tensorflow==2.0.0rc0
!pip install -q tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function
import tensorflow as tf
import tensorflow_addons as tfa
-----------

======================================================================
FAIL: test_layers_weightnormalization (testipynb.testipynb.NbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/testipynb/testipynb.py", line 96, in test_func
    assert passing, msg
AssertionError:

 ... layers_weightnormalization FAILED

SyntaxError in cell [2]
-----------
!pip install tensorflow-gpu==2.0.0rc0
!pip install tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function, unicode_literals

import tensorflow as tf
import tensorflow_addons as tfa
import numpy as np
from matplotlib import pyplot as plt
-----------

======================================================================
FAIL: test_losses_triplet (testipynb.testipynb.NbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/testipynb/testipynb.py", line 96, in test_func
    assert passing, msg
AssertionError:

 ... losses_triplet FAILED

SyntaxError in cell [2]
-----------
!pip install -q tensorflow-gpu==2.0.0rc0
!pip install -q tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function, unicode_literals

import io
import numpy as np
import tensorflow as tf
import tensorflow_addons as tfa
import tensorflow_datasets as tfds
from matplotlib import pyplot as plt
from google.colab import files
-----------

======================================================================
FAIL: test_optimizers_lazyadam (testipynb.testipynb.NbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/testipynb/testipynb.py", line 96, in test_func
    assert passing, msg
AssertionError:

 ... optimizers_lazyadam FAILED

SyntaxError in cell [2]
-----------
!pip install tensorflow-gpu==2.0.0rc0
!pip install tensorflow-addons~=0.5
from __future__ import absolute_import, division, print_function, unicode_literals

import tensorflow as tf
import tensorflow_addons as tfa
import tensorflow_datasets as tfds
import numpy as np
from matplotlib import pyplot as plt
-----------

======================================================================
FAIL: test_template (testipynb.testipynb.NbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/miniconda3/envs/tf2/lib/python3.6/site-packages/testipynb/testipynb.py", line 96, in test_func
    assert passing, msg
AssertionError:

 ... template FAILED

NameError in cell [5]
-----------
result = model(tf.constant(np.random.randn(10,5), dtype = tf.float32)).numpy()

print("min:", result.min())
print("max:", result.max())
print("mean:", result.mean())
print("shape:", result.shape)
-----------

----------------------------------------------------------------------
Ran 6 tests in 96.071s

FAILED (failures=6)
make: *** [tutorials-test] Error 1

@seanpmorgan what do you think are the next steps? Fix the DeadKernelError: Kernel died  issue you have and I can't reproduce - right?

Hi Philip could you please verify how you're testing the code? With the current commit I'm getting:
root@a4cbd4d4bf35:/addons# make tutorials-test
makefile:35: *** missing separator.  Stop

The indentation must be consistent with the other commands.
Also, could you please run in the docker container so we have consistent environments and that is how this will be ran during automated testing.
For a non-gpu environment I'm getting a mix of issues related to #532 (which is unrelated to this) and DeadKernel errors:
tutorialtest_output.txt
When running in gpu environment just to mitigate the #532 issues I get
tutorialtest_gpu_output.txt
As you can see a number of imports fail (matplotlib, tensorflow_datasets, numpy). So what we'll need to do is create a separate install_nbtest_dependecy.sh script which runs for this make command.
We can't merge this until we know that the setup is capable of running these tests

Hi Philip could you please verify how you're testing the code? With the current commit I'm getting:
root@a4cbd4d4bf35:/addons# make tutorials-test
makefile:35: *** missing separator.  Stop

This is fixed now.

As you can see a number of imports fail (matplotlib, tensorflow_datasets, numpy). So what we'll need to do is create a separate install_nbtest_dependecy.sh script which runs for this make command.

@seanpmorgan Do you think we could use the Colab Notebook container as an environment since this is the environment people will actually see this in? I am not quite sure if they are public though. Maybe a googler can help out? @karmel

I don't quite follow-- what colab nb container are you referring to, exactly?

I always thought that the "standard" Google Colab Notebook is actually a container running on Googles Servers. In case it is, do you know if the container is public?

@MarkDaoust @yashk2810 @tomerk , who know more about how the nb tests are run and what the options here might be.

One option is, since we are creating a subsite for addons, their notebooks will automatically get tested by the infrastructure we have in place for all the other subsites and TensorFlow. If the notebooks fail, an email will be sent to the addons team to look into what went wrong.
For CI testing, I can look into opensourcing the stuff that we use to test the notebooks.
@seanpmorgan @PhilipMay One thing to note for enabling the tests with our infra is to not save the outputs in the notebook and install whichever version of addons you want to test your notebook with. You can follow the template notebook (https://github.com/tensorflow/docs/blob/master/tools/templates/notebook.ipynb).

For CI testing, I can look into opensourcing the stuff that we use to test the notebooks.

Good news! Thank Yash. It sound like we can share the same notebook testing library (and even workflow) if opensourcing. Would you mind telling us the opensouring schedule if possible?

Would you mind telling us the opensouring schedule if possible?

Probably within a few weeks :)

Hi @yashk2810 - what is the status here?
Thanks
Philip

Bumping it again... @yashk2810 @karmel

Hi @PhilipMay,
Opensourcing what we use will take some time since it needs to be decoupled from a lot of things that we use and will require some effort. I will triage and set a priority on it. But you are free to explore any other alternative that you have in mind :)
On a positive note, since addons is now a subsite tensorflow.org/addons, every notebook is already getting tested. If something fails, we will send an email to the appropriate handle.

@seanpmorgan does it mean, that the issue is redundant and could be closed?

IMO yes, provided that the above system works. @yashk2810 What email are failures sent to?
@PhilipMay would this be an acceptable solution?

@PhilipMay would this be an acceptable solution?

For me this is 100% ok. Why not. No need to implement something that is already implemented.

On a positive note, since addons is now a subsite tensorflow.org/addons, every notebook is already getting tested. If something fails, we will send an email to the appropriate handle.

Are we sure that "send an email to the appropriate handle" does work and that this will always be converted to a ticket here in GitHub?

always be converted to a ticket here in GitHub?
I don;t think it will be converted to a ticket on Github.

But, is there an email with all the maintainers of addons that we can forward it to? Then you can monitor that email for failure emails from our side.

But, is there an email with all the maintainers of addons that we can forward it to? Then you can monitor that email for failure emails from our side.

I dont know... @seanpmorgan do you?
Before we close this PR we should test the mail functionality.

Also, can it be on CI level? So we get the alert before merging to master
and running nightly builds.
On Tue, Dec 10, 2019, 21:23 Philip May<EMAIL_ADDRESS>wrote:

But, is there an email with all the maintainers of addons that we can
forward it to? Then you can monitor that email for failure emails from our
side.
I dont know... @seanpmorgan https://github.com/seanpmorgan do you?
Before we close this PR we should test the mail functionality.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/addons/pull/540?email_source=notifications&email_token=AAF5KKQGKFTI3AMKFWLVTP3QX73EBA5CNFSM4IZRBPZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGQZG5Q#issuecomment-564237174,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAF5KKVKQTHUK7O473ILJILQX73EBANCNFSM4IZRBPZQ
.

(@yashk2810 -- not sure how these particular tests are set up, but when I previously tried in several ways to get internal testing emails forwarded to external aliases, I failed. We could possibly set up an internal alias with external members or something like that, but note that there may be some hoops to jump through.)

We could possibly set up an internal alias

Yup, that's how it works. The email is sent to our internal alias and then we forward it to the appropriate external handle after scrubbing all the internal email id's.
We have been doing this for translations already.

@yashk2810 The email for failing tests is<EMAIL_ADDRESS>It's open to all as a google group.
Do you have any more information as to when the notebooks are tested? Is this only done once the API docs are re-generated after every release? We'll be releasing 0.7 in the next few days. I can put a broken tutorial as a test on the branch.

The email for failing tests is<EMAIL_ADDRESS>
Thanks

done once the API docs are re-generated after every release

Yes, currently. If the changes made to the notebooks are very often, then I can move it to weekly and eventually nightly.

I can put a broken tutorial as a test on the branch

If you want to test it out, sure. But we pull docs from the master branch. So you won;t have to put it on the 0.7 branch :)

The email for failing tests is<EMAIL_ADDRESS>
Thanks

done once the API docs are re-generated after every release

Yes, currently. If the changes made to the notebooks are very often, then I can move it to weekly and eventually nightly.

I can put a broken tutorial as a test on the branch

If you want to test it out, sure. But we pull docs from the master branch. So you won;t have to put it on the 0.7 branch :)

Great thanks for your help @yashk2810! We'll put up a test prior to requesting the next update of API docs.
I did want to confirm that addons docs are pulled from master though. We only generate API docs on our release branches so can you confirm that the addons site generates them from master branch separately from what we do?

addons site generates them from master branch

So, for the narrative docs (guides, tutorials, etc) we pull from master.
For API docs we use your pip package to generate the api docs.
Does that make sense?

Thanks all for the support on this! Closing this PR and created a subsequent issue to track this testing:
https://github.com/tensorflow/addons/issues/768


Database migration process

Description
Currently, we have no defined database migration process. Any schema changes should ideally be handled through an automatic migration process implemented within the library.
Motivation
For schema changes, the database migration should be a seamless process for consumers.
Requirements

Research for ways on how to best approach this;
Have a look at Bee on how they tackle this problem;
We would need to introduce versioning within the database

Some earlier thoughts on this topic: https://github.com/Thoralf-M/wallet-core/blob/dev/documentation/database_migration.md
Are you planning to do it yourself in a pull request?
No.

Done in https://github.com/iotaledger/iota-sdk/pull/420


StellarPGx v1.2.5 incorrectly calls GeT-RM samples NA10847, NA19920 and NA19908 that were correctly called by the original version

Hello there,
while testing StellarPGx v1.2.5 on CYP2D6-calling on samples from GeT-RM I encountered something unexpected. On your publication on StellarPGx, samples NA10847, NA19920 and NA19908 are correctly called as *1/*41, *1/*4x2 and *1/*46 respectively. However v1.2.5 incorrectly calls these samples as *39/*41, *1/*4 and *43/*45 respectively.
The BAM files for these samples were downloaded from the ENA website and used as is.
Stellar was called as:
nextflow run main.nf -dsl1 -profile standard --build b37 --gene cyp2d6

For NA10847 the output was:
 --------------------------------------------

CYP2D6 Star Allele Calling with StellarPGx

--------------------------------------------

Initially computed CN = 2

Sample core variants:
42523805~C>T~0/1;42523943~A>G~0/1

Candidate alleles:
['39.v1_41.v1']

Result:
*39/*41

Activity score:
1.5

Metaboliser status:
Normal metaboliser (NM)

For NA19920:
--------------------------------------------

CYP2D6 Star Allele Calling with StellarPGx

--------------------------------------------

Initially computed CN = 3

Sample core variants:
42522613~G>C~0/1;42523507~A>C~0/1;42523943~A>G~1/1;42524947~C>T~0/1;42526694~G>A~0/1

Candidate alleles:
[1.v1_4.v2]

Result:
Possible novel allele or suballele present: interpret with caution; experimental validation and expert review through PharmVar is recommended

Likely background alleles:
[*1/*4]

Activity score:
Indeterminate

Metaboliser status:
Indeterminate

For NA19908:
--------------------------------------------

CYP2D6 Star Allele Calling with StellarPGx

--------------------------------------------

Initially computed CN = 2

Sample core variants:
42522613~G>C~0/1;42523943~A>G~0/1;42525077~C>T~0/1;42526717~C>T~0/1

Candidate alleles:
['1.v1_46.v1', '43.v1_45.v1']

Result:
*43/*45

Activity score:
1.5

Metaboliser status:
Normal metaboliser (NM)

Is this an error on my end?
Thank you.

@VasilisFlouris,
Thanks for raising this. It's not an error on your side.
There have been some updates to StellarPGx since the version we published in CPT which (inevitably) affect a few calls.
For NA10847, the 42522613G>C variant is called in the initial steps. However, it is subsequently filtered out because it has a Quality by Depth (QD) = 8.7. The current StellarPGx version considers variants with QD ≥ 10. If the QD filter in the main.nf file is relaxed to say 5, then the allele call will be *1/*41 as per the original call. However, relaxing thresholds could also affect calls in other samples (could be non-GETRM) as this region (and the CYP2D locus in general) is prone to artefactual variants resulting from read misalignments across CYP2D6 and CYP2D7.
For NA19920, the copy number is still correctly identified. However, the current StellarPGx version flags 42523507~A>C as a potential novel core variant.  This variant is supported by a QD = 23.5 and AD=21/85 (not so uncommon for samples with a duplication). Therefore, that's why StellarPGx flagged that there is a potential novel star allele. Of course, this would need experimental validation and again highlights challenges relating to variant calling in this region.
For the NA19908, the quick tests I've run point to some subvariants being filtered out according to the more stringent cutoffs in the current StellarPGx version. These were used to discriminate *1/*46 from *43/*45 variant combinations in the first version.


Support for Pre/Post Deployment Scripts

I don't see any way to enable execution of pre/post deployment scripts.  These scripts would (could) be used to run changes that are not supported by Skeema or, in my case, DML that wouldn't belong in the main Skeema project.
An example is if you look at how SQL Server Data Tools works.  It's a declarative state based database project but you can define 1 script file as a "predeployment" script and another script file as a "postdeployment" script.  This script can use include syntax to call other scripts.  I have primarily used this in the past to support DML changes before/after the main DDL process executes.
Liquibase is a migration based change tool but works similarly to how the SSDT pre/post deployment script task works.  it also logs all changes to a table using an id, author, and filename so no single file executes twice.
Would it be possible to get a feature like this added?  Simply defining an optional pre-deployment script file that gets executed before the skeema push and an optional post-deployment script file that gets executed after the skeema push would be helpful even if there were no other additional logic added.  Being able to source files like the mysql command line tool would be enough to allow any configuration needed. (see this SO link for reference on the file sourcing https://stackoverflow.com/questions/1976396/mysql-include-a-script-within-script)
Execution would look like this

execute pre-deployment script (if exists)
execute skeema push
execute post-deployment script (if exists)

This would be very useful for any data manipulation that has to occur in say a dev pipeline or also for initialization data to be added to the db.

Thank you for the feature request. There's some previous discussion along these lines in https://github.com/skeema/skeema/issues/22#issuecomment-459907251 a while back. There are still some open questions from that thread, and I'd welcome feedback on what would work best for different users' requirements.
In the meantime, there are a few work-arounds to achieve similar behavior, but they're a bit awkward:

If you only want a single pre-deploy hook and/or post-deploy hook for the entire push operation (rather than per dir/db/table), simply write a custom script that wraps skeema push, rather than using skeema push directly. The downside is you may need to duplicate some configuration logic between your wrapper and Skeema.

For pre/post push hooks per DDL statement, the ddl-wrapper option can help with this. Your custom hook script can use arbitrary conditional logic based on the table name etc. The downside is your hook script will also become responsible for executing DDL properly, rather than relying on Skeema to execute DDL.

One solution for post-push SQL-based hooks is present in today's release of Skeema Premium v1.12, via one-time events. These can be used to execute arbitrary SQL, either a single time per server/environment (data migrations, imperative RENAMEs, etc) or on every single push (e.g. tracking schema change timing or running some proc after every push)
Pros: doesn't require any additional state tracking, such as a migration log table. The database server manages event metadata directly in its data dictionary.
Cons: requires the server event scheduler to be enabled; events can be difficult to debug because they only log failure messages to the server error log.
Keeping this issue open, since more direct support for hooks (separate from events) is still likely in the future.


Layout and initial values

Bonjour,
It's more a question than an issue, but there is not tag for liform on stackoverflow... So here is my question : http://stackoverflow.com/q/43979726/2470352
Merci

Answer in the slack no "schema.title" => use null in renderField

Thanks for reporting this. I have updated the docs accordingly.


tab content not focusable

introduction of preventdefaulting in other PRs broke the focusability of tab content as the tab key is now prevented

closed by #28


Corrupted image in pdf file on chrome.

Environment:
Browser: Chrome Version 49.0.2623.112 (64-bit)
MAC OS X - 10.10.5
Are you using the latest version of jsPDF?
Version 1.2.61-git Built on 2016-03-01T16:15    CommitID 04343b54b0
Have you tried using jspdf.debug.js?
Yes, no error in console.
Steps to reproduce
Click on "Download"
http://jsbin.com/balulopexo/edit?html,js,console,output
What I saw
report_corrupted.pdf
What I expected
Out from http://mrrio.github.io/jsPDF/
Images.pdf
Other Observations

Working correctly in Safari.
Preview image correctly shown in chrome from data URI(http://dopiaza.org/tools/datauri/index.php)

Hello!
You should add this script in your page, then it will works.

Then it will works.

No, even after using the script, image is still corrupted.
Check the green patch and text in the logo.
Steps to reproduce(with suggested changes)
Click on "Download"
http://jsbin.com/jusuxubame/edit?html,js,console,output

There was not enouht free space on pdf page to place whole your image.
http://jsbin.com/mufixetuqa/1/edit?html,js,console,output
But i still see problem with colors in it. I'll try to resolve it

Thanks for support.
I have tried only with Logo image and still see the corruption. Will send the jsbin.com soon.
As I mentioned earlier, with http://mrrio.github.io/jsPDF/  it is working fine with same image.
Just replace imgData with my image data in (http://jsbin.com/mufixetuqa/1/edit?html,js,console,output) and you can see the result.
No corruption at all.
Same code is working correctly on Safari browser also.

It looks like a PNG is being added as a JPEG, try changing formats?

Yes converting PNG to JPEG format does solves the issue. Not sure if the original format of the image file was JPEG. Will check.
Any ideas how it is working in your tool(http://mrrio.github.io/jsPDF/ ) or Safari?

I checked it, it was actually a PNG which you tried to process as an JPG.
In my chrome with the latest code it works fine.


Android Notifications Reappears After Being Dismissed

Steps to replicate issue:

Play then pause an audio file in my app
Dismiss audio player from notifications
Switch to another app (BeyondPod)
Play audio from podcast app
Audio player from my app reappears in notifications after being dismissed already

I have this same problem. You don't need to switch to another app. Simply just dismiss the notification, and it will reappear after a short while.

It seems like this has been fixed in newer versions of the package. I've posted instructions on how to update to newest version in this issue: https://github.com/martijn00/XamarinMediaManager/issues/234

@Inrego Thanks!  I will give that a try.

Update! No, it doesn't seem to be fixed in newer versions.


Unable to use on Mac M1 (Aarch64)

Hi,
I used the plugin on my previous mac and loved it, but since switching to the M1 family of Macs I have this error:

(error detail below)
Would you consider an update that supports M1, or let me know how to go about resolving this?
I did a little further research and I believe the Xerial snappy library does support M1, but maybe the plugin needs to be updated to a more recent version?
https://github.com/xerial/snappy-java/issues/279
Stack trace:
Unable to process file /private/var/folders/8n/5c3_bt153s94pgl3gfdwqx3r0000gn/T/1636083936286-0/weather.parquet/part-00000-f79c3105-d729-4006-9b0c-3a3c2bdcb98f-c000.snappy.parquet

org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64
	at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:361)
	at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:195)
	at org.xerial.snappy.SnappyLoader.loadSnappyApi(SnappyLoader.java:167)
	at org.xerial.snappy.Snappy.init(Snappy.java:69)
	at org.xerial.snappy.Snappy.<clinit>(Snappy.java:46)
	at org.apache.parquet.hadoop.codec.SnappyDecompressor.decompress(SnappyDecompressor.java:62)
	at org.apache.parquet.hadoop.codec.NonBlockedDecompressorStream.read(NonBlockedDecompressorStream.java:51)
	at java.base/java.io.DataInputStream.readFully(DataInputStream.java:200)
	at java.base/java.io.DataInputStream.readFully(DataInputStream.java:170)
	at org.apache.parquet.bytes.BytesInput$StreamBytesInput.toByteArray(BytesInput.java:286)
	at org.apache.parquet.bytes.BytesInput.toByteBuffer(BytesInput.java:237)
	at org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary.<init>(PlainValuesDictionary.java:91)
	at org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainBinaryDictionary.<init>(PlainValuesDictionary.java:74)
	at org.apache.parquet.column.Encoding$1.initDictionary(Encoding.java:90)
	at org.apache.parquet.column.Encoding$5.initDictionary(Encoding.java:163)
	at org.apache.parquet.column.impl.ColumnReaderBase.<init>(ColumnReaderBase.java:413)
	at org.apache.parquet.column.impl.ColumnReaderImpl.<init>(ColumnReaderImpl.java:46)
	at org.apache.parquet.column.impl.ColumnReadStoreImpl.getColumnReader(ColumnReadStoreImpl.java:82)
	at org.apache.parquet.io.RecordReaderImplementation.<init>(RecordReaderImplementation.java:271)
	at org.apache.parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:147)
	at org.apache.parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:109)
	at org.apache.parquet.filter2.compat.FilterCompat$NoOpFilter.accept(FilterCompat.java:177)
	at org.apache.parquet.io.MessageColumnIO.getRecordReader(MessageColumnIO.java:109)
	at org.apache.parquet.hadoop.InternalParquetRecordReader.checkRead(InternalParquetRecordReader.java:136)
	at org.apache.parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:225)
	at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:132)
	at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:136)
	at uk.co.hadoopathome.intellij.viewer.fileformat.ParquetFileReader.getRecords(ParquetFileReader.java:99)
	at uk.co.hadoopathome.intellij.viewer.FileViewerToolWindow$2.doInBackground(FileViewerToolWindow.java:193)
	at uk.co.hadoopathome.intellij.viewer.FileViewerToolWindow$2.doInBackground(FileViewerToolWindow.java:184)
	at java.desktop/javax.swing.SwingWorker$1.call(SwingWorker.java:304)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.desktop/javax.swing.SwingWorker.run(SwingWorker.java:343)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Thanks for bringing it to my attention. I don't have an M1 Mac to test this on but the plugin is due a version bump across all dependencies so hopefully that fixes it. Would you be able to test it for me once I have please? I'll upload the zip here sometime this weekend and let you know.

Awesome, thanks! Yes I will be happy to try it. I haven’t tested a pre-release plugin before - I look forward to giving it a go!

I have updated just Snappy to see if that helps. I can't attach 60MB files to GitHub so you can download the file from my Google Drive at https://drive.google.com/file/d/1DzV7SuzRChkHAF50QRlsn4t0ttYj9xZ6/view?usp=sharing.
If you would rather build it yourself then checkout https://github.com/benwatson528/intellij-avro-parquet-plugin/tree/Updating-Xerial and follow the instructions at https://github.com/benwatson528/intellij-avro-parquet-plugin/tree/Updating-Xerial#building-locally to build it.
To install it, uninstall the plugin from the marketplace, restart your IDE and then install the zip from the plugin installation page.

Hi Ben! It worked!
I uninstalled the marketplace plugin, downloaded the zip from your Google Drive and installed it from disk:

Thanks Luke I'll try and get an update out this week. I'll let you know when it's on the marketplace.

Hi @luketn, the update is live on the marketplace so you can uninstall the zip and re-install from the marketplace.


Strange HomeKit behavior if light supports both color and temperature

I am experiencing exactly this "bug" / behaviour of HomeKit:
https://github.com/home-assistant/core/pull/30756
This problem causes the ambient light control to be not reliable. It forgets the presets etc.
A solution would be to just disable the ColorTemperature characteristic for the background light. I think this is the best solution because you typically only use the color feature and not the temperature feature of the background light.

I can't get it working with ceiling20 (GuangCan). It has a background light and a moon mode. The moon mode is working perfectly but the background light colour presets will be forgotten and defaulted to CT.
I tried with and without the beta function (Allow CT to be set for RGB lights)


Rojo Sourcemap Issue

When I run "pesde install", sourcemap_generator.luau regenerates itself with a missing parenthesis. See below:

https://github.com/daimond113/pesde-scripts/pull/2


nextcloud-client: 3.9.0 -> 3.9.1

Diff: https://github.com/nextcloud/desktop/compare/v3.9.0...v3.9.1
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Maybe upstream replace the commit of the tag?


Use a new prefix definition for info:fedoraconfig/

Addresses: https://jira.duraspace.org/browse/FCREPO-2247
The goal here is to avoid making an earlier change non-breaking for users who had previously made use of the fedoraconfig prefix.

@acoburn, What are your thoughts on the risk of the new namespace prefix config clashing with existing use of that same prefix by current Fedora installations?

I am not at all concerned about this. If, however, I were concerned, the only sensible approach would be to remove that namespace declaration altogether, which has other implications.
In fact making any change at all to the namespaces in the CND file has a possibility of conflicting with some existing repository somewhere. But I do not believe that is a legitimate reason to not ever make a change.

Resolved with: https://github.com/fcrepo4/fcrepo4/commit/f4f06ffe37f4229ec3c9414410907ca85c3d3401


Feature-#64: Add links to PR titles

What's Changed
Closes #64
Technical Description

Edittted the function signature of render_pull_request_table such that org/repo information is available such that the
link to PR can be formed.
Added link to pr using f-strings

(It's my first time contributing to a project, and during testing I can't seem to be able to click onto anything from within the terminal -- is there a specific terminal that I need to use?)

First of all, thank you for your contribution @yushao2!
As for terminals, I have been using iTerm2 on mac and Windows Terminal (with WSL2 on Windows) but I'm not aware of any issues with othersterminals. Which terminal are you using?

@yushao2 I can confirm the link works for me when using `control + click to open the link:

@yushao2 I can confirm the link works for me (on Windows Terminal) when using `control + click to open the link:

Unable to get docker working right, but managed to perform the linting steps anyway -- this PR should be good for merger if it's all good with you :)

Looks good. Thank you once again for your contribution :)


How to keep a node_group with size = 0

I have issues

We don't want to delete a node_group
We just want to set the size to 0 to clean resources

I'm submitting a...

[x] bug report
[ ] feature request
[ ] support request - read the FAQ first!
[ ] kudos, thank you, warm fuzzy

What is the current behavior?
Error: expected scaling_config.0.desired_size to be at least (1), got 0

  on .terraform/modules/eks/modules/node_groups/node_groups.tf line 11, in resource "aws_eks_node_group" "workers":
  11:     desired_size = each.value["desired_capacity"]

Error: expected scaling_config.0.desired_size to be at least (1), got 0

  on .terraform/modules/eks/modules/node_groups/node_groups.tf line 11, in resource "aws_eks_node_group" "workers":
  11:     desired_size = each.value["desired_capacity"]

If this is a bug, how to reproduce? Please include a code sample if relevant.

Setting the desired_capacity to 0

What's the expected behavior?

I can have a node_group of size 0

Are you able to fix this problem and submit a PR? Link here if you have already.
No
Environment details

Affected module version:
OS:
Terraform version:

Any other relevant info

@marcellodesales if you are using managed node groups, this is an EKS feature that has not be released yet but they are working on it. See https://github.com/aws/containers-roadmap/issues/724

Hi,
can we relook at this pls?
I understand the AWS API feature for min_size = 0 and desired_size = 0 has just been released (about 2-3 days ago).
So that should be inline with this.
We can create cluster nodegroup with 0 size in eksctl in ui and also in aws-cli. terraform should be the same.
one of the use-case is to "initially" create a 0-sized nodegroup, then customize the CNI, so we dont have to end-up scaling the nodegroups 2 times, (to 0 and back) to essentially restart the nodes.

I have checked and current release of module is able to create such node groups

  # module.eks_cluster.module.node_groups.aws_eks_node_group.workers["node2"] will be created
  + resource "aws_eks_node_group" "workers" {
      + ami_type               = "AL2_x86_64"
      + arn                    = (known after apply)
      + capacity_type          = "ON_DEMAND"
      + cluster_name           = "eks"
      + disk_size              = 50
      + id                     = (known after apply)
      + instance_types         = [
          + "m5.xlarge",
        ]
      + node_group_name        = (known after apply)
      + node_group_name_prefix = "eksnode2"
      + node_role_arn          = "arn:aws:iam::*******:role/eks20210825103410662700000003"
      + release_version        = (known after apply)
      + resources              = (known after apply)
      + status                 = (known after apply)
      + subnet_ids             = [
          + "subnet-07a78a7e8184e3702",
        ]
      + tags                   = {
          + "Name" = "eks"
        }
      + tags_all               = {
          + "Name"    = "eks"
        }
      + version                = (known after apply)

      + scaling_config {
          + desired_size = 0
          + max_size     = 10
          + min_size     = 0
        }

      + timeouts {}

      + update_config {
          + max_unavailable            = (known after apply)
          + max_unavailable_percentage = (known after apply)
        }
    }


setup.json for the AiiDA registry

Hi @abelcarreras,
could you move the kwargs of the call to setup() in setup.py into a setup.json, so that setup.py only contains
#!/usr/bin/env python

from __future__ import absolute_import
from setuptools import setup, find_packages
import json

if __name__ == '__main__':
    # Provide static information in setup.json
    # such that it can be discovered automatically
    with open('setup.json', 'r') as info:
        kwargs = json.load(info)
    setup(
        packages=find_packages(),
        long_description=open('README.md').read(),
        long_description_content_type='text/markdown',
        **kwargs)

and the rest in in the json? See e.g. https://github.com/ltalirz/aiida-zeopp/blob/master/setup.py
Note also you will need to create a new MANIFEST.in file in the root directory with this content:
https://github.com/ltalirz/aiida-zeopp/blob/master/MANIFEST.in
This will allow to have the correct information in the AiiDA registry: https://aiidateam.github.io/aiida-registry/plugins/lammps.html

Thanks!


Wrong title being used when playing video

The title being displayed when a video starts is different from the title being used on the rest of the screen.
Can we use the same the title?
See Dances at a Gathering (2020).

We had a chat today with James and Maksym Opanasenko and verified the titles pulling from Prismic.
We have found out, that our event titles are pulling correctly - the logic is the following :

if there is a title in digital event details, then we show it.
if there is no data from digital events details, then we pull the title from the event details page
the titles, that we show when the player is showing a video are also correct - they are different from the Event Title by design, as there might be several videos linked to one event.
The same logic is present on the web, so we have replicated it in the TV app. We have also shown the code and confirmed, that the logic is correct. So I guess we do not need extra development efforts on this topic.


Toggle off activity spans

Checked for duplicates
No - I haven't checked
Alternatives considered
Yes - and alternatives don't suffice
Related problems
Sometimes you want to hide activity spans instead of the directives.  The names of the directives can be edited, so they can provide useful information that the activities do not.  If you don't care about the spans, they can introduce a lot of clutter.
If a simulation dataset could be deselected, then the activity spans would disappear, but so would the plots, so this doesn't quite satisfy the need.
Describe the feature request
Add a button beside the directive toggle to toggle activity spans.
Better would be to also(?) include this (and the directive toggle) as part of the timeline editor.

@bradNASA while we don't have a global timeline button for this, in the latest Aerie version you are able to right-click on an Activities row in the timeline and select "Show Activities", "Show Activity Directives" or "Both". Does this satisfy the need or do you think we also need a button for this at the top? If no button is necessary, this can be closed.

Existing context menu implementation is sufficient for closing


Thematize CircleProgress component

Description
Yamada UI components are tied to a theme. However, theCircleProgress component does not have a theme.
Problem Statement/Justification
Themes have defaultProps, which can be utilized to share props throughout the project. However, the FileButton component does not have a theme, so it cannot take advantage of them.
Proposed Solution or API
No response
Alternatives
No response
Are you willing to participate in implementing this feature and create a pull request include the implementation?
None
Additional Information
No response

I'll try it.


Add checkConsistency API call

Adds support fo the checkConsistency API command which is used to determine whether to promote or reattach a transaction.

Fixed that, thank you.

Thanks!


Deprecate Beta network policy support

Beta network policy support should now be fully deprecated in favor of GA NetworkPolicies (since Kube1.8)

implemented by #155


Gulp doesn't exit after last task exited

When I run the coverage task in gulp with mocha and istanbul, gulp reports the task as done. However, it doesn't exit.
I would have expected gulp to exit once the task is over.
gulpfile.js:
'use strict';

const gulp = require('gulp');
const mocha = require('gulp-spawn-mocha');
// istanbul requires the non-threaded gulp-mocha version
const mocha_istanbul = require('gulp-mocha');
const istanbul = require('gulp-istanbul');
const eslint = require('gulp-eslint');
const runSequence = require('run-sequence');

const test_files = 'test/**/*_test.js';
const test_helper_file = 'test/test_helper.js';
const lib_files = 'lib/**/*.js';
const app_file = 'index.js';
const min_coverage = 90;

// ...
// other unrelated tasks
// ...

gulp.task('pre-cov', () => {
  return gulp.src([lib_files])
    .pipe(istanbul({ includeUntested: true }))
    .pipe(istanbul.hookRequire());
});

gulp.task('cov', ['pre-cov'], () => {
  process.env.NOCK_BACK_MODE = 'lockdown';
  return gulp.src([test_files], { read: false })
    .pipe(mocha_istanbul({ reporter: 'dot' }))
    .pipe(istanbul.writeReports())
    .pipe(istanbul.enforceThresholds({ thresholds: { global: min_coverage } }));
});

Output when running the task (the ^C is me exiting gulp since it doesn't do it itself)
$ gulp cov
[13:26:20] Using gulpfile ~/project/gulpfile.js
[13:26:20] Starting 'pre-cov'...
[13:26:21] Finished 'pre-cov' after 125 ms
[13:26:21] Starting 'cov'...

  ․․․․․․․․․․․․․․․․․․․․․․․

  23 passing (249ms)

---------------------|----------|----------|----------|----------|----------------|
File                 |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
---------------------|----------|----------|----------|----------|----------------|
 lib/                |      100 |      100 |      100 |      100 |                |
  my_lib.js          |      100 |      100 |      100 |      100 |                |
---------------------|----------|----------|----------|----------|----------------|
All files            |      100 |      100 |      100 |      100 |                |
---------------------|----------|----------|----------|----------|----------------|

=============================== Coverage summary ===============================
Statements   : 100% ( 136/136 )
Branches     : 100% ( 25/25 )
Functions    : 100% ( 20/20 )
Lines        : 100% ( 135/135 )
================================================================================
[13:26:21] Finished 'cov' after 921 ms
^C

All other gulp tasks running fine and exiting properly.
$ gulp -v
[13:30:28] CLI version 3.9.1
[13:30:28] Local version 3.9.1
$ node -v
v4.4.3

One of your dependencies is keeping node open - use https://github.com/mafintosh/why-is-node-running to figure out why.  Not a gulp issue.


Updating database scripts and added two columns

Adding this new columns to held the appointment for a client.

If you want to leave the pick_up_date as null in seed_database.sql, then I guess it's all good.

Hi Alessandro thanks for your review, yes is necessary to leave that false according to the acceptance criteria.

It could later help to have some example dates, but it's no big deal.


Implement AttestationCallProxyV1

Spec is here: https://github.com/celo-org/celo-monorepo/issues/7057

@m-chrzan to split this up into smaller tickets and assign parts to someone else


Get it working :)

Find all necessary packages to be imported such that the flask is running and all functions are working.
You can test this by signing up after the code is run and you can check if the details are entered being collected in the database file by running check.py

Can i be assigned to this

!assign @Git-Cat-21 30

!deassign

i have fixed the issue why has it been deassigned i am about to make a PR

!assign @Git-Cat-21  30

!bounty 10


Do backwards compatible Dataset related types improvements

Like #65 but fully backwards compatible and merging to master branch.

@rad-cord ping


Submit Button not working in Safari Private Mode

I run iOS 8.1 Safari. The submit button does not create an new comment.
When I use the Chrome browser app it works as expected. So my server config is not broken.

I have a similiar issue with the iPad1. I can edit files, but the commit button is disabled. Anyone know a solution?

This has been fixed!


Failed: Update autorest version to prerelease 3.0.0-alpha.20240507.14

Triggered from pull request: https://github.com/Azure/autorest.csharp/pull/4432

API change check
API changes are not detected in this pull request.


Raise exception when injecting into modules

To me it happened that I accidentally injected my dependencies into a namespace module instead of into a class, like this:
module Operations
  module Articles
    include Import["repositories.articles"]

    class Create
      # ...
    end
  end
end

When I did that, I got an error originating from that line, but happening somewhere in dry-auto_inject's internals (a NoMethodError for NilClass). This led me to believe that I've somehow incorrectly passed parameters to Import[], and it took me a while to realize what was the error.
The error occurred because the downstream code assumes the #initialize method will be defined, which is not the case for modules. To improve the developer experience, we detect that we're attempting to inject into a module and raise an explicit exception.

I won't go into details right now, but I want to note that this is strategy-dependent. For example, effect-based DI works with modules or whatever. And I believe we can make built-in strategies to work with modules too. However, this would require some hacking.

Hmm, you're right, I haven't considered that. I've also often seen people using modules instead of classes for stateless operations.
I'll keep this open in case you still want to merge it until support for modules is added.

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 3

Complexity increasing per file
==============================
- lib/dry/auto_inject/strategies/constructor.rb  2

Clones added
============
- spec/dry/auto_inject_spec.rb  3

See the complete overview on Codacy


proxy: Add logging

Signed-off-by: Ryan Northey<EMAIL_ADDRESS>

Codecov Report

Merging #176 (e6b7cc5) into main (63615b5) will decrease coverage by 2.10%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##             main     #176      +/-   ##
==========================================
- Coverage   43.81%   41.71%   -2.11%
==========================================
  Files          91       84       -7
  Lines        2433     2347      -86
==========================================
- Hits         1066      979      -87
- Misses       1367     1368       +1

Impacted Files
Coverage Δ

playground/control/connectors/docker/proxies.py
34.78% <50.00%> (+0.69%)
:arrow_up:

tests/py/command.py

tests/py/connectors/docker/base.py

tests/py/event.py

tests/py/run.py

tests/py/runner.py

tests/py/validators.py

tests/py/request.py

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 63615b5...e6b7cc5. Read the comment docs.


fix: path for new summary_filtered and old summary api endpoints

What new features does this PR implement?
new /projects/summary api endpoint implementation was not backward compatible, so previous implementation of /projects/summary was restored and the new functionality is exposed at /projects/summary_filtered endpoint.
This change was partially performed in previous PR #4691, but the frontend api pointed to the /projects/summary endpoint rather than the /projects/summary_filtered.  This PR points to the appropriate endpoint from the frontend.
What bugs does this PR fix?
The frontend was calling the wrong endpoint which has a different signature, so it could not retrieve the projects.
Does this PR introduce any additional changes?
No.
How have you tested this PR?
Tested locally against a locally served api backend.  Unfortunately, testing of previous PR #4691 was flawed as it connected to a deployed backend rather than local backend.
Additional information
None.

:tada: This PR is included in version 9.28.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Try from err typos

Fixes minor typographical errors for the following conversions and adds corresponding integration tests:

TryFrom for Vec was Error::ExpectedInteger and is not Error::ExpectedReal
From<HashMap<String, Robj>> was missing, though &str impl existed

I did not fix the misspelling 'Enviroment' throughout extendr-api since it does not actually produce errors.
R documentation quashed for the conversion test functions.

Apologies for being picky. I think the hashmap code needs to be its own separate PR, and it probably needs some discussion first in an issue. It's not clear to me that we want to always convert a hashmap into a list, as opposed to an environment.
And for a PR that fixes spelling and other minor issues (such as @noRd tags) I think it should also fix the spelling of environment.
Finally, the integration tests are failing because R check notices that there are no man pages. I think @usage NULL will fix this. (But not entirely certain. I just know somehow it's possible to suppress documentation.)

Your comments make sense and aren't too picky. I agree discussing hashmap is reasonable. The lack of From HashMap<String, Robj> seemed to someone new on the project to be simply an oversight given existing From,TryFrom implementations for HashMap with &str and String keys.
Will implement your suggestions and resubmit. Thanks

I've opened an issue about hasmaps: #230

recent upstream commits here, and

removal of HashMap<String, Robj> impl fix from previous push
Rd files suppressed with @usage NULL for all conversion test functions
Enviroment typo fixed throughout extendr-api
maintains fix for typo in TryFrom error type in double_vec

It's starting to look good, but there are some changes that don't seem to belong (the change in the changelog, the hashmap code). Overall, from experience, I'd say once you have more than a handful of commits chances are something went wrong and you'd be better off starting over. Also, never rebase, just merge in the main tree.
The tests fail because there's a formatting error. I'll flag it.

It's possible that a lot of things will be fixed if you just merge in master.

hmm, so i did not make these changes or most of the others below --- they come from the previous commits made yesterday (re: #226) and which i pulled this afternoon to avoid merge conflicts. the same is true from the below.
to your main point: it would have been cleaner to start over.
nonetheless, thank you for the time here, and please do let me know the preference in situations like this.

It's possible that a lot of things will be fixed if you just merge in master.

yes, will do

@clauswilke To clean up this git history mess a bit my plan is to merge to master on my fork and have the PR point there. I've not done that before as I typically am a maintainer on projects I work on. If doing so, with some loss of the commit history, is undesirable please do let me know. Trying to reduce the amount of nonsense I produce for review here going forward. Thanks again

Yes, all you have to do is merge master into your current branch and then push. The PR should update and GitHub should then only register the changes since master.
The history doesn't matter because when we merge the PR we squash. It's another reason why each PR should be one single issue only; it's going to be only one commit on the master branch.

I think this looks good now. Could you try to merge in master so we no longer see the changes that aren't actually part of your PR?

great, thanks for the comments all.
what follows is rather tedious. TLDR summary is that i can reopen a PR to reset history, switch the PR base branch and back to trick github, or do nothing. the PR merge shouldn't associate the unrelated commits to this PR, as the history displayed here is a cosmetic feature of github apparently.
on the commit history from upstream/master that isn't mine but appears here:
i did indeed merge my branch into origin/master (my fork master), but the issue of github showing commits from upstream/master appears to be a github issue. see this long-standing thread, still active:
https://stackoverflow.com/questions/16306012/github-pull-request-showing-commits-that-are-already-in-target-branch#26986320
annoying, though it seems this issue is mostly cosmetic since you will squash this PR's commits anyway.
i see a few options and am happy to do what is easiest or best from your perspective:

try one of the hacks listed above, such as switching PR base branch to something else and back. one person suggested this adds more history, so i did not just do it.

open a new PR and close this one, which should reset the history.

rebase to squash all commits into one and re-push. i am hesitant to do this as I am unsure whether this will make it appear as though i have done someone else's work. i don't believe so --- again, this commit history is a github display issue not a git issue ---- but am unsure.

I think we should try to get this right, most importantly so you've got the workflow down in the future, and also because if I merge now I'm quite certain it'll look like @yutannihilation co-authored the PR.
In brief, I think this is where you went wrong:

i did indeed merge my branch into origin/master (my fork master),

You need to merge upstream master into your branch, but you should not touch your fork master.
In general, the workflow that works is:

Make a fork of the project
Check out the fork and make a branch for development
Make commits, then push (the first time with git push --set-upstream origin <branch>)
Then make a PR
If changes are needed, make more commits and push
If the PR can't be merged because of a conflict, fetch upstream, merge upstream/master into your branch, fix conflicts, and push
Repeat 5 and 6 until ready for merge

looks like a new PR is the cleanest solution i think. if you would prefer, i could squash the history into one commit, with me as author.

In brief, I think this is where you went wrong:

in case this is a reference to my future self or others: i should have been more clear. i did that only after the previous commit history from origin/master had shown up in this PR, so it cannot be the cause, at least not this time.
but your comment is still valid, i think, in broad terms: these errant commits, which were updates to origin/master while this PR was ongoing, showed up after i merged upstream/master into origin/master then into my branch, rather than merging upstream/master directly into my branch. otherwise, i followed your steps 1-7. step 6 was the key here, in other words. noted.

looks like a new PR is the cleanest solution i think. if you would prefer, i could squash the history into one commit, with me as author.

Whichever is easier for you. I don't particularly care. We squash when we merge into master.

ok, this should fix things. thanks for the back and forth.

CI failures on R-devel runner are probably irrelevant to this pull request, so please ignore (probably due to the recent API change on R-devel, which itself is a bit worrisome...).
  error[E0425]: cannot find function, tuple struct or tuple variant `SET_PRSEEN` in this scope
      --> extendr-api/src/wrapper/promise.rs:27:13
       |
  27   |             SET_PRSEEN(sexp, 0);
       |             ^^^^^^^^^^ help: a function with a similar name exists: `SET_PRENV`
       |
      ::: /home/runner/work/extendr/extendr/target/debug/build/libR-sys-fe61ba3184160aea/out/bindings.rs:4969:5
       |
  4969 |     pub fn SET_PRENV(x: SEXP, v: SEXP);
       |     ----------------------------------- similarly named function `SET_PRENV` defined here

I'm still confused by the hashmap situation but I'm going to merge since this contains many useful fixes. We can sort out the hashmap in future PRs.


autovot.py outputs double negative confidences

Using autovot (both via ISCAN and normally) with pos_only flagged inside VotDecode returns double negative confidences (e.g., --1196.96). This still completes in normal autovot, but inside the ISCAN environment it returns a ValueError (since it's a string that can't be converted to a float).

This can be closed as this is no longer reproducible.


feat (desktop): upgrade from wry 37 to 41

In my playing around with dioxus to resolve an issue with wry, I toyed with upgrading to the latest version: v0.41.0
I'm not sure if it changed anything for me yet, but I figured it might be useful for the project so I've made a PR for it.
Hopefully its helpful somehow :)

For the sake of fun metrics, after more testing, jumping from v37 to v41 made about a 400ms improvement in time to the initial render after repeated testing using my own personal project.


Will wasmtime support threaded wasm?

Hi there,
I am a PhD student working on parallelism & concurrency. We are hoping to do some work on parallel wasm, and are looking for a performant nonweb runtime. Are there plans for wasmtime to support the Threads Proposal?
How much work do you think it would be to extend/modify wasmtime to support the semantics proposed by the threads extension? What kind of changes would it entail? If there is interest, I would love to try and contribute.
Thanks in advance!
--Gautam

Looks like we have a few of these issues open in the issue tracker so I'm going to close this in favor of a dedicated tracking issue -- https://github.com/bytecodealliance/wasmtime/issues/888


await waitForNavigation times-out?

Using waitForNavigation after a click always times-out.
Here's an example:
my $pw = Playwright->new();
my $browser = $pw->launch( headless => 1, type => 'chrome' );
my $page = $browser->newPage({ viewport => { width => 2048, height => 1024 }});

$page->goto('https://github.com/teodesian/playwright-perl', { waitUntil => 'networkidle' });
$page->click('text="Go to file"');

$pw->await( $page->waitForNavigation({ waitUntil => "domcontentloaded" }));

This is based on a JS playwright test script which does the following:
await Promise.all([
            this.page.click(sel),
            this.page.waitForNavigation({ waitUntil: "domcontentloaded" }),
        ]);

Note that you have done your waiting in JS in a Promise.all, which fires both requests simultaneously.
In the perl you are clicking the link that executes navigation, and then waiting.
I suspect if you do your wait before the click, and then await the promise like so:
my $promise = $page->waitForNavigation({ waitUntil => "domcontentloaded" }));
$page->click('text="Go to file"');
$pw->await($promise);

the code will work.  See the example here:
https://github.com/teodesian/playwright-perl/blob/main/example.pl#L96

In summary, because perl is not an event-driven language like javascript, you have to handle things slightly differently than you would expect.  From a technical point of view, this is not actually any different than what is going on in JS -- you have to set up the relevant listener before the required event happens, or a timeout will occur.  Promise.All simply allows you to not feel that pain.

I can on the other hand say with confidence that it would be valuable and a good user experience to implement a similar capability to promise.All which may be used to abstract away such akwardness.  Reopening as a feature request along those lines.


Client closed connection before receiving entire response

unable to download file
getting this error "Client closed connection before receiving entire response"
the file which i am trying to download is just 266KB
     for (NSString *downloadURL in arr) {
   // NSString *downloadURL = [d objectForKey:@"url"];
    dispatch_async(dispatch_get_main_queue(), ^{
        NSLog(@"URL for downloading : %@",downloadURL);
        NSArray *pathArray = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask,YES);
        NSString *documentsDirectory = [pathArray objectAtIndex:0];
        NSString *downloadedZipPath = [documentsDirectory stringByAppendingPathComponent:TEMP_DICTIONARY];
        NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:downloadURL]];
       // [request addValue:@"bytes=x-" forHTTPHeaderField:@"Range"];
       // [request setTimeoutInterval:20];
        AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];

        operation.outputStream = [NSOutputStream outputStreamToFileAtPath:downloadedZipPath append:NO];

        [operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
            NSLog(@"Success");
        } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
            NSLog(@"Error: %@", error);
        }];

        [operation start];
    });

tried to increase api request time out but still not working
http://stackoverflow.com/questions/33608425/client-closed-connection-before-receiving-entire-response-afnetworking

This typically has something to do with your server configuration, and not something specific to AFNetworking. You could try migrating to AFHTTPSessionManager, which uses NSURLSession under the hood. The NSURLConnection based API's you are using are no longer supported in the 3.0.0 branch.
:beers:


improve migrate command

running migrations from different locations (modules)

more or less like this one from @cebe https://github.com/yiiext/migrate-command

I'll leave this post here to not create the same one.

This issue is almost 2 years.  I am waiting for this feature from milestone to milestone. What is the problem with solving it? So, we could discuss it...
PS: Is there any tool/extension to automate the process of running migrations of plugged in modules?

@vladim1 Check out our extension https://github.com/dmstr/yii2-migrate-command

I think #8202 is a good way to fix this issue for modules, extensions, external libraries and all the other cases discused here.

Since I include migrations in my test routines - ie migrations should be able to migrate all the way down and all the way up at any time - I am looking forward to a built-in solution. :+1:
The https://github.com/dmstr/yii2-migrate-command is part of my standard tool-kit already, but it would make Yii even more cool if it was built in. Especially since newcomers wouldn't have to discover it themselves. :)

I whould solve this matter using namespaces instead of paths, just I have proposed here:
https://github.com/yiisoft/yii2/issues/9698#issuecomment-147696833

@klimov-paul We're currently using aliases, which is essentially the same, isn't it? https://github.com/yiisoft/yii2/issues/384#issuecomment-63638791

I discovered this issue before two years. Well the eta was RC. Any chance that this feature will be implemented in the next release or should we stick to some extensions that can handle this approach?

Solution proposed at #12511

Resolved by commit 8aa0e85


update client-c to disable grpc client keepalive (#4655)

What problem does this PR solve?
cherry-pick https://github.com/pingcap/tiflash/pull/4655 to release-5.3 branch
close https://github.com/pingcap/tiflash/issues/4192, close https://github.com/pingcap/tiflash/issues/4662
Problem Summary:
What is changed and how it works?
Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

Fix bug that TiFlash query will meet keepalive timeout error randomly.

/merge


Report

WIP / Test

See https://github.com/mne-tools/mne-python/pull/9749 which I think will be an easier route


Automatically refresh the graph as needed

The graph periodically checks the status of the jobs that need attention (i.e. the ones that are in waiting, queued, or running) and updates the graph as need.
In this example we see a job going from queued, to running, to error:

Codecov Report

Merging #2413 into dev will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##              dev    #2413   +/-   ##
=======================================
  Coverage   93.79%   93.79%
=======================================
  Files         161      161
  Lines       18983    18983
=======================================
  Hits        17805    17805
  Misses       1178     1178

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66df7ba...b49e769. Read the comment docs.