Error: Could not start container

Hello everyone,
I'm using testcontainers for E2E testing our Spring application. Everything was working, but one day on my PC (my colleague don't have any problem) it started crashing. docker-compose up is working, I cleaned the whole docker cache, volumes, images, containers..., I cleaned all the gradle build, but I'm still receiving the following error:
2018-11-05 16:30:10.479 DEBUG 12682 --- [           main] o.t.utility.TestcontainersConfiguration  : Testcontainers configuration overrides will be loaded from file:/home/peter/.testcontainers.properties
2018-11-05 16:30:10.479 DEBUG 12682 --- [           main] o.t.utility.TestcontainersConfiguration  : Testcontainers configuration overrides loaded from TestcontainersConfiguration(properties={docker.client.strategy=org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy})
2018-11-05 16:30:10.497  INFO 12682 --- [           main] o.t.d.DockerClientProviderStrategy       : Loaded org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy from ~/.testcontainers.properties, will try it first
2018-11-05 16:30:10.514  INFO 12682 --- [           main] o.t.d.DockerClientProviderStrategy       : Will use 'okhttp' transport
2018-11-05 16:30:10.637 DEBUG 12682 --- [     ducttape-0] o.t.d.DockerClientProviderStrategy       : Pinging docker daemon...
2018-11-05 16:30:10.807  INFO 12682 --- [           main] tAndSystemPropertyClientProviderStrategy : Found docker client settings from environment
2018-11-05 16:30:10.819  INFO 12682 --- [           main] o.t.d.DockerClientProviderStrategy       : Found Docker environment with Environment variables, system properties and defaults. Resolved:
    dockerHost=unix:///var/run/docker.sock
    apiVersion='{UNKNOWN_VERSION}'
    registryUrl='https://index.docker.io/v1/'
    registryUsername='peter'
    registryPassword='null'
    registryEmail='null'
    dockerConfig='DefaultDockerClientConfig[dockerHost=unix:///var/run/docker.sock,registryUsername=peter,registryPassword=<null>,registryEmail=<null>,registryUrl=https://index.docker.io/v1/,dockerConfigPath=/home/peter/.docker,sslConfig=<null>,apiVersion={UNKNOWN_VERSION},dockerConfig=<null>]'

2018-11-05 16:30:10.820  INFO 12682 --- [           main] org.testcontainers.DockerClientFactory   : Docker host IP address is localhost
2018-11-05 16:30:10.932  INFO 12682 --- [           main] org.testcontainers.DockerClientFactory   : Connected to docker:
  Server Version: 18.06.1-ce
  API Version: 1.38
  Operating System: Arch Linux
  Total Memory: 7850 MB
2018-11-05 16:30:10.950 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : Looking up auth config for image: quay.io/testcontainers/ryuk:0.2.2
2018-11-05 16:30:10.950 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : RegistryAuthLocator has configFile: /home/peter/.docker/config.json (exists) and commandPathPrefix:
2018-11-05 16:30:10.953 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : registryName [quay.io] for dockerImageName [quay.io/testcontainers/ryuk:0.2.2]
2018-11-05 16:30:10.953 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : no matching Auth Configs - falling back to defaultAuthConfig [null]
2018-11-05 16:30:10.953 DEBUG 12682 --- [           main] o.t.d.a.AuthDelegatingDockerClientConfig : Effective auth config [null]
2018-11-05 16:30:14.422 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : Looking up auth config for image: quay.io/testcontainers/ryuk:0.2.2
2018-11-05 16:30:14.422 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : RegistryAuthLocator has configFile: /home/peter/.docker/config.json (exists) and commandPathPrefix:
2018-11-05 16:30:14.423 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : registryName [quay.io] for dockerImageName [quay.io/testcontainers/ryuk:0.2.2]
2018-11-05 16:30:14.423 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : no matching Auth Configs - falling back to defaultAuthConfig [null]
2018-11-05 16:30:14.423 DEBUG 12682 --- [           main] o.t.d.a.AuthDelegatingDockerClientConfig : Effective auth config [null]
2018-11-05 16:30:14.977 DEBUG 12682 --- [containers-ryuk] o.testcontainers.utility.ResourceReaper  : Sending 'label=org.testcontainers%3Dtrue&label=org.testcontainers.sessionId%3D164abc25-0a92-4e4d-b6d4-c448d98e9a29' to Ryuk
2018-11-05 16:30:14.977 DEBUG 12682 --- [containers-ryuk] o.testcontainers.utility.ResourceReaper  : Received 'ACK' from Ryuk
2018-11-05 16:30:14.977  INFO 12682 --- [           main] org.testcontainers.DockerClientFactory   : Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
        ℹ Checking the system...
        ✔ Docker version should be at least 1.6.0
        ✔ Docker environment should have more than 2GB free disk space
2018-11-05 16:30:15.039  INFO 12682 --- [           main] 🐳 [alpine/socat:latest]                 : Pulling docker image: alpine/socat:latest. Please be patient; this may take some time but only needs to be done once.
2018-11-05 16:30:15.039 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : Looking up auth config for image: alpine/socat:latest
2018-11-05 16:30:15.039 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : RegistryAuthLocator has configFile: /home/peter/.docker/config.json (exists) and commandPathPrefix:
2018-11-05 16:30:15.039 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : registryName [index.docker.io] for dockerImageName [alpine/socat:latest]
2018-11-05 16:30:15.039 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : no matching Auth Configs - falling back to defaultAuthConfig [null]
2018-11-05 16:30:15.039 DEBUG 12682 --- [           main] o.t.d.a.AuthDelegatingDockerClientConfig : Effective auth config [null]
2018-11-05 16:30:17.356 DEBUG 12682 --- [containers-ryuk] o.testcontainers.utility.ResourceReaper  : Sending 'label=com.docker.compose.project%3Dyt7kd71uwxpq' to Ryuk
2018-11-05 16:30:17.358  INFO 12682 --- [           main] 🐳 [docker-compose]                      : Local Docker Compose is running command: pull
2018-11-05 16:30:17.369 DEBUG 12682 --- [           main] o.t.s.o.z.exec.ProcessExecutor           : Executing [docker-compose, pull] in /home/peter/Workspace/rds/rds-backend/out/test/resources with environment {COMPOSE_PROJECT_NAME=yt7kd71uwxpq, COMPOSE_FILE=/home/peter/Workspace/rds/rds-backend/out/test/resources/docker-compose.yml}.
2018-11-05 16:30:17.371 DEBUG 12682 --- [           main] o.t.s.o.z.exec.ProcessExecutor           : Started java.lang.UNIXProcess@48b2dbc4
2018-11-05 16:30:17.398 DEBUG 12682 --- [containers-ryuk] o.testcontainers.utility.ResourceReaper  : Received 'ACK' from Ryuk
2018-11-05 16:30:17.771 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling db        ...
2018-11-05 16:30:17.771 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling uptime-db ...
2018-11-05 16:30:19.149 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.150 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling uptime-db ... pulling from library/mysql
2018-11-05 16:30:19.182 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.183 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling db        ... pulling from timescale/timescaledb
2018-11-05 16:30:19.183 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.183 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling db        ... digest: sha256:48eac4ba55f6338015...
2018-11-05 16:30:19.183 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.183 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling db        ... status: image is up to date for t...
2018-11-05 16:30:19.184 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.185 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling db        ... done
2018-11-05 16:30:19.634 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.635 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling uptime-db ... digest: sha256:42bab37eda993e417c...
2018-11-05 16:30:19.635 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.635 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling uptime-db ... status: image is up to date for m...
2018-11-05 16:30:19.636 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.636 ERROR 12682 --- [       Thread-6] 🐳 [docker-compose]                      : Pulling uptime-db ... done
2018-11-05 16:30:19.685 DEBUG 12682 --- [           main] o.t.s.o.z.exec.WaitForProcess            : java.lang.UNIXProcess@48b2dbc4 stopped with exit code 0
2018-11-05 16:30:19.686 ERROR 12682 --- [           main] 🐳 [docker-compose]                      :
2018-11-05 16:30:19.687  INFO 12682 --- [           main] 🐳 [docker-compose]                      : Docker Compose has finished running
2018-11-05 16:30:19.687  INFO 12682 --- [           main] 🐳 [docker-compose]                      : Local Docker Compose is running command: up -d
2018-11-05 16:30:19.687 DEBUG 12682 --- [           main] o.t.s.o.z.exec.ProcessExecutor           : Executing [docker-compose, up, -d] in /home/peter/Workspace/rds/rds-backend/out/test/resources with environment {COMPOSE_PROJECT_NAME=yt7kd71uwxpq, COMPOSE_FILE=/home/peter/Workspace/rds/rds-backend/out/test/resources/docker-compose.yml}.
2018-11-05 16:30:19.688 DEBUG 12682 --- [           main] o.t.s.o.z.exec.ProcessExecutor           : Started java.lang.UNIXProcess@66e1b2a
2018-11-05 16:30:19.943 ERROR 12682 --- [       Thread-8] 🐳 [docker-compose]                      : Creating network "yt7kd71uwxpq_default" with the default driver
2018-11-05 16:30:20.050 ERROR 12682 --- [       Thread-8] 🐳 [docker-compose]                      : Creating yt7kd71uwxpq_uptime-db_1_bee56e916398 ...
2018-11-05 16:30:20.050 ERROR 12682 --- [       Thread-8] 🐳 [docker-compose]                      : Creating yt7kd71uwxpq_db_1_ed9e0d2e72c6        ...
2018-11-05 16:30:20.488 ERROR 12682 --- [       Thread-8] 🐳 [docker-compose]                      :
2018-11-05 16:30:20.488 ERROR 12682 --- [       Thread-8] 🐳 [docker-compose]                      : Creating yt7kd71uwxpq_uptime-db_1_bee56e916398 ... done
2018-11-05 16:30:20.521 ERROR 12682 --- [       Thread-8] 🐳 [docker-compose]                      :
2018-11-05 16:30:20.521 ERROR 12682 --- [       Thread-8] 🐳 [docker-compose]                      : Creating yt7kd71uwxpq_db_1_ed9e0d2e72c6        ... done
2018-11-05 16:30:20.560 DEBUG 12682 --- [           main] o.t.s.o.z.exec.WaitForProcess            : java.lang.UNIXProcess@66e1b2a stopped with exit code 0
2018-11-05 16:30:20.561 ERROR 12682 --- [           main] 🐳 [docker-compose]                      :
2018-11-05 16:30:20.561  INFO 12682 --- [           main] 🐳 [docker-compose]                      : Docker Compose has finished running
2018-11-05 16:30:20.563  INFO 12682 --- [           main] 🐳 [alpine/socat:latest]                 : Creating container for image: alpine/socat:latest
2018-11-05 16:30:20.563 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : Looking up auth config for image: alpine/socat:latest
2018-11-05 16:30:20.563 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : RegistryAuthLocator has configFile: /home/peter/.docker/config.json (exists) and commandPathPrefix:
2018-11-05 16:30:20.563 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : registryName [index.docker.io] for dockerImageName [alpine/socat:latest]
2018-11-05 16:30:20.563 DEBUG 12682 --- [           main] o.t.utility.RegistryAuthLocator          : no matching Auth Configs - falling back to defaultAuthConfig [null]
2018-11-05 16:30:20.563 DEBUG 12682 --- [           main] o.t.d.a.AuthDelegatingDockerClientConfig : Effective auth config [null]
2018-11-05 16:30:20.598 ERROR 12682 --- [           main] 🐳 [alpine/socat:latest]                 : Could not start container

org.testcontainers.containers.ContainerLaunchException: Aborting attempt to link to container yt7kd71uwxpq_uptime-db_1 as it is not running
	at org.testcontainers.containers.GenericContainer.applyConfiguration(GenericContainer.java:492) [testcontainers-1.9.1.jar:na]
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:256) [testcontainers-1.9.1.jar:na]
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:237) [testcontainers-1.9.1.jar:na]
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76) ~[duct-tape-1.0.7.jar:na]
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:235) [testcontainers-1.9.1.jar:na]
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:220) [testcontainers-1.9.1.jar:na]
	at org.testcontainers.containers.DockerComposeContainer.startAmbassadorContainers(DockerComposeContainer.java:248) ~[testcontainers-1.9.1.jar:na]
	at org.testcontainers.containers.DockerComposeContainer.start(DockerComposeContainer.java:158) ~[testcontainers-1.9.1.jar:na]
	at com.rieter.rds.util.DockerEnv.start(DockerUtil.kt:33) ~[classes/:na]
	at com.rieter.rds.util.DockerExtension.beforeAll(DockerUtil.kt:13) ~[classes/:na]
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.lambda$invokeBeforeAllCallbacks$7(ClassTestDescriptor.java:358) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.invokeBeforeAllCallbacks(ClassTestDescriptor.java:358) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.before(ClassTestDescriptor.java:197) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.before(ClassTestDescriptor.java:74) ~[junit-jupiter-engine-5.3.1.jar:5.3.1]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:102) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at java.util.ArrayList.forEach(ArrayList.java:1257) ~[na:1.8.0_192]
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:110) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51) ~[junit-platform-engine-1.3.1.jar:1.3.1]
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220) ~[junit-platform-launcher-1.3.1.jar:1.3.1]
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188) ~[junit-platform-launcher-1.3.1.jar:1.3.1]
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202) ~[junit-platform-launcher-1.3.1.jar:1.3.1]
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181) ~[junit-platform-launcher-1.3.1.jar:1.3.1]
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128) ~[junit-platform-launcher-1.3.1.jar:1.3.1]
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:74) ~[junit5-rt.jar:na]
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) ~[junit-rt.jar:na]
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) ~[junit-rt.jar:na]
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) ~[junit-rt.jar:na]

Can anybody help me?
Thanks

Problem seems to be in this line:
2018-11-05 16:30:20.598 ERROR 12682 --- [           main] 🐳 [alpine/socat:latest]                 : Could not start container
Can you please run this image manually with docker run and see what happens?

Problem seems to be in this line:
2018-11-05 16:30:20.598 ERROR 12682 --- [ main] 🐳 [alpine/socat:latest] : Could not start container
Can you please run this image manually with docker run and see what happens?
Thank you for the quick response. I tried the example from the docker hub

docker run --restart=always     -p <IP_ADDRESS>:2376:2375     -v /var/run/docker.sock:/var/run/docker.sock     alpine/socat     tcp-listen:2375,fork,reuseaddr unix-connect:/var/run/docker.sock
Unable to find image 'alpine/socat:latest' locally
latest: Pulling from alpine/socat
ff3a5c916c92: Pull complete
abb964a97c4c: Pull complete
Digest: sha256:5f245d7a2d63fccdb098834d00d9fb04c404a3f1423eb2f84045fc00e93d7c32
Status: Downloaded newer image for alpine/socat:latest
^C

Sorry, I don't have any real clue here.
You're colleague is also using Arch?
Best debugging idea for me is looking into the container logs while they are starting and before the exception appears (also Docker daemon logs maybe).

They are using Mint. For now I ended with using multiple GenericContainers. Thank you for help, I will try to investigate the docker daemon :slightly_smiling_face:

I actually prefer using multiple container objects over docker-compose for
most Testcontainers use cases, so no objection here ;)
javorka<EMAIL_ADDRESS>schrieb am Mi., 7. Nov. 2018, 10:39:

They are using Mint. For now I ended with using multiple
GenericContainers. Thank you for help, I will try to investigate the docker
daemon 🙂
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/testcontainers/testcontainers-java/issues/955#issuecomment-436563474,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2jaEOiZ7FoopbF6C60nss3D4t8AcYFks5usqo0gaJpZM4YOtO8
.

Hi
It's seems I'm having the similar problem. Could not start alipne:socat/latest internally KafkaContainer is suing SocatContainer. My integration test uses KafkaContainer and while loading the ApplicationContext it ends up in
`
2018-11-15 15:38:46.098  INFO 24220 --- [           main] 🐳 [alpine/socat:latest]                 : Creating container for image: alpine/socat:latest
2018-11-15 15:38:46.098  WARN 24220 --- [           main] o.t.utility.RegistryAuthLocator          : Failure when attempting to lookup auth config (dockerImageName: alpine/socat:latest, configFile: /home/fabdul/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /home/fabdul/.docker/config.json (No such file or directory)
2018-11-15 15:38:46.113 ERROR 24220 --- [           main] 🐳 [alpine/socat:latest]                 : Could not start container
java.lang.reflect.UndeclaredThrowableException: null
at com.sun.proxy.$Proxy95.exec(Unknown Source) ~[na:na]
at org.testcontainers.containers.Network$NetworkImpl.create(Network.java:88) ~[testcontainers-1.10.0.jar:na]
at org.testcontainers.containers.Network$NetworkImpl.getId(Network.java:59) ~[testcontainers-1.10.0.jar:na]
at org.testcontainers.containers.GenericContainer.applyConfiguration(GenericContainer.java:526) ~[testcontainers-1.10.0.jar:na]
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:256) ~[testcontainers-1.10.0.jar:na]
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:237) ~[testcontainers-1.10.0.jar:na]
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76) ~[duct-tape-1.0.7.jar:na]
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:235) ~[testcontainers-1.10.0.jar:na]
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:220) ~[testcontainers-1.10.0.jar:na]
at org.testcontainers.containers.KafkaContainer.doStart(KafkaContainer.java:68) ~[kafka-1.10.0.jar:na]
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:220) ~[testcontainers-1.10.0.jar:na]
at org.testcontainers.lifecycle.Startable$start.call(Unknown Source) ~[na:na]
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-2.5.4.jar:2.5.4]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115) ~[groovy-2.5.4.jar:2.5.4]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:119) ~[groovy-2.5.4.jar:2.5.4]
at ...................................MyTestClass.groovy.
at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:649) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:373) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:127) ~[spring-boot-test-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99) ~[spring-test-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117) ~[spring-test-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:108) ~[spring-test-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:118) ~[spring-test-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83) ~[spring-test-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener.prepareTestInstance(SpringBootDependencyInjectionTestExecutionListener.java:44) ~[spring-boot-test-autoconfigure-2.1.0.RELEASE.jar:2.1.0.RELEASE]
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:246) ~[spring-test-5.1.2.RELEASE.jar:5.1.2.RELEASE]
at org.spockframework.spring.SpringTestContextManager.prepareTestInstance(SpringTestContextManager.java:56) ~[spock-spring-1.2-groovy-2.5.jar:1.2]
at org.spockframework.spring.SpringInterceptor.interceptInitializerMethod(SpringInterceptor.java:43) ~[spock-spring-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.extension.AbstractMethodInterceptor.intercept(AbstractMethodInterceptor.java:24) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:97) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:475) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.runInitializer(BaseSpecRunner.java:341) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.runInitializer(BaseSpecRunner.java:336) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.initializeAndRunIteration(BaseSpecRunner.java:274) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.runSimpleFeature(BaseSpecRunner.java:266) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.doRunFeature(BaseSpecRunner.java:260) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner$5.invoke(BaseSpecRunner.java:243) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:484) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:467) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.runFeature(BaseSpecRunner.java:235) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.runFeatures(BaseSpecRunner.java:185) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.doRunSpec(BaseSpecRunner.java:95) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner$1.invoke(BaseSpecRunner.java:81) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:484) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:467) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.runSpec(BaseSpecRunner.java:73) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.BaseSpecRunner.run(BaseSpecRunner.java:64) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.spockframework.runtime.Sputnik.run(Sputnik.java:63) ~[spock-core-1.2-groovy-2.5.jar:1.2]
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) ~[surefire-junit4-2.22.1.jar:2.22.1]
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) ~[surefire-junit4-2.22.1.jar:2.22.1]
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) ~[surefire-junit4-2.22.1.jar:2.22.1]
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) ~[surefire-junit4-2.22.1.jar:2.22.1]
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) ~[surefire-booter-2.22.1.jar:2.22.1]
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) ~[surefire-booter-2.22.1.jar:2.22.1]
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) ~[surefire-booter-2.22.1.jar:2.22.1]
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) ~[surefire-booter-2.22.1.jar:2.22.1]
Caused by: java.lang.reflect.InvocationTargetException: null
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.testcontainers.dockerclient.AuditLoggingDockerClient.lambda$wrappedCommand$14(AuditLoggingDockerClient.java:98) ~[testcontainers-1.10.0.jar:na]
... 57 common frames omitted
Caused by: com.github.dockerjava.api.exception.NotFoundException: {"message":"could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network"}
    at org.testcontainers.dockerclient.transport.okhttp.OkHttpInvocationBuilder.execute(OkHttpInvocationBuilder.java:273) ~[testcontainers-1.10.0.jar:na]
    at org.testcontainers.dockerclient.transport.okhttp.OkHttpInvocationBuilder.execute(OkHttpInvocationBuilder.java:257) ~[testcontainers-1.10.0.jar:na]
    at org.testcontainers.dockerclient.transport.okhttp.OkHttpInvocationBuilder.post(OkHttpInvocationBuilder.java:128) ~[testcontainers-1.10.0.jar:na]
    at com.github.dockerjava.core.exec.CreateNetworkCmdExec.execute(CreateNetworkCmdExec.java:27) ~[testcontainers-1.10.0.jar:na]
    at com.github.dockerjava.core.exec.CreateNetworkCmdExec.execute(CreateNetworkCmdExec.java:12) ~[testcontainers-1.10.0.jar:na]
    at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21) ~[testcontainers-1.10.0.jar:na]
    at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35) ~[testcontainers-1.10.0.jar:na]
    ... 62 common frames omitted

`
Versions used:
java 11
testcontainers/kafka: 1.10.1
ubuntu 16.04

The problem which I posted yesterday got solved by cleaning up iptables rules, bridge network devices, and routing table entries created by docker containers.
docker network prune --filter "until=24h"

Having the same issue as @faris-git - any progress on this?

Pruned the networks, stopped and removed all Docker containers, restarted the Docker engine as well as MacOS, upgraded testcontainers-mysql to latest version and then downgraded it again and magically my test works now.

@faris-git the cleanup of containers (and networks) should be tackled automatically, so something is not right here... Do you see any logs from ResourceReaper or ‘Ryuk’ during test runs? In particular, any errors from these components?
Sent with GitHawk

@faris-git the cleanup of containers (and networks) should be tackled automatically, so something is not right here... Do you see any logs from ResourceReaper or ‘Ryuk’ during test runs? In particular, any errors from these components?
Sent with GitHawk

I ran into this issue when I run many tests in one job and if more tests got interrupted(e.g. due to ApplicationContext could not start). I see that when the tests are interrupted then testcontainer couldn't stop the container properly(especially socat container). As in error stack above

Encountering this issue with networks not being pruned properly leading eventually to the error. Pruning the networks does resolve it as a workaround, but could definitely use a permanent fix. Some co-workers running into this 1/day

Are you sure this is the same issue?
Which version of Testcontainers are you using?
Networks should be cleaned by the Ryuk container worst case after JVM exit.

Fairly certain it's the same issue, I can prune the networks, run tests that invoke testcontainers, and the network is still showing when I run docker network ls. I'm running 1.11.1

@nateha1984 are you running Docker Compose or "normal" containers?

We're using Docker Compose

Just hit what I believe is the same issue - and docker network prune (Deleted Networks: ... x21) fixed it.

There seems to be a bug in Docker Compose, I reported it here:
https://github.com/docker/compose/issues/6636

It looks like the workaround suggested in the Docker Compose issue won't work with testcontainers at the moment as testcontainers is using DC 1.8.0 and v2.1 compose files are compatible with DC 1.9.0+. Any plans to upgrade the DC version?

@nateha1984 can you use the local compose mode?

@bsideup said:
There seems to be a bug in Docker Compose, I reported it here:
docker/compose#6636

Then @shin-, the maintainer from Docker Compose, said in that thread:

@shin- said:
Hey @chris-crone @bsideup -- The reason for this is the v2.0 file format is mapped to an API version that didn't support labels on networks and volumes (see the code here). Using 2.1 or above will solve this issue.

The solution to this issue is to use Docker Compose file version 2.1, where TestContainers only supports version 2.0 (I just tried). So the networks aren't going to get automatically pruned because they're not being tagged properly.
Docker Compose will either have to support tagging of networks with file version 2.0, or TestContainers will have to support file version 2.1.
@rnorth, @bsideup:  Are you guys going to support Compose file version 2.1?

In my case the issue went the same scenario as in the original report:

Everything was working, but one day on my PC (my colleague don't have any problem) it started crashing.

I realized that this one day was exactly the day I added some lightweight unit tests alongside with the integration tests with Testcontainers' docker-compose. I am using static singleton container pattern.
My suspicion is that the container is run twice: once in surefire maven plugin execution (unit tests), and once in failsafe execution (integration tests).
I'm going to experiment with some options. The first thing I thought of is:

parameterize container instantiation somehow (probably implies making it non-static, lazy kinda, to not run with unit tests)

Maybe this gives others some ideas on how to resolve it.

We have a very similar issue with a docker-compose based tests (using Testcontainers) hanging on Jenkins (ubuntu) but run fine locally (mac, windows). The same test runs successfully sometimes on the Jenkins then fails 10 times in a row (sporadic). Network seems ok - the slave is a freshly installed ubuntu slave without anything else running (physical pc, not a vm or circleCI env etc.). Pruning everything and restarting the whole machine has little effect.
The PC running jenkins slave has some strange issues with the DNS though - sometimes not pingable by name. The problem might be unrelated but i thought i mention it as it could be input for someone trying to fix this.
No obvious errors can be seen - it just hangs there and fails after waiting for at least 10minutes. Which is also strange because timeout is actually 5 minutes ... anyway - would be nice to have docker-compose stuff running correctly. Normal testcontainer tests run perfectly. For now we just convert all docker-compose based tests to multiple testcontainers. So far this is working for us. But maybe you could deprecate docker-compose tests until this is fixed in a stable manner as it is time consuming to switch testing styles.

@Schwaller we do recommend using GenericContainers and not Docker Compose as they are more reliable and provide better features. That said, there are no known issues with Docker Compose, not to mention blockers that would make us deprecate it any time soon. But you should always remember that Docker Compose integration delegates most of the things the Docker Compose binary which may behave weirdly. Last but not least, the DNS issue you mentioned suggests that there is some issue with the network on the machine which may affect Docker Compose.

@bsideup thx for the quick response! I understand that its not ideal that DNS seems to be not perfect but this never stopped any other process from working correctly. I can run the docker-compose file on that machine manually without any issues. Just as soon as testcontainers&docker-compose is involed chances are around 80% that within a month it will develop sporadic issues.
If you think it works for others I guess I have to take your word for it ;-)

BTW thx for creating/maintaining testcontainers! Great stuff!

encounter this issue too. after disable the docker-compose v2. the issue is solved.


Upgraded to 1.2.6

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Rework sample limit to work for 2.0

Correctly update reported series.
Increment prometheus_target_scrapes_exceeded_sample_limit_total.
Add back unittests.
Ignore stale markers when calculating sample limit.
Fixes #2770
This ended up being more work than expected.

:+1:


add tx_succeeded?

Adding a method to check if transaction has succeeded.
Also took the opportunity to change the name of is_mined_tx? -> tx_mined? to be closer to Ruby conventions (boolean methods with ? without is_)

Codecov Report

Merging #173 (38c3f14) into main (85e2283) will decrease coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #173      +/-   ##
==========================================
- Coverage   99.75%   99.70%   -0.05%
==========================================
  Files          70       70
  Lines        4016     4061      +45
==========================================
+ Hits         4006     4049      +43
- Misses         10       12       +2

Impacted Files
Coverage Δ

lib/eth/client.rb
98.69% <100.00%> (-1.31%)
:arrow_down:

spec/eth/client_spec.rb
100.00% <0.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

hey @kurotaky, the build in Ubuntu is failing because it lacks libyaml I guess, but idk why this error started now :thinking:

Thanks, I'll fix the CI.


Update vcenter-access-identity-description.md

Corrected VMware product names.

@rvandenbedem : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Update Eigen to version that includes scan op fix

This updates Eigen to a newer version that includes a fix needed to merge #2711.

Can one of the admins verify this patch?

https://github.com/tensorflow/tensorflow/blob/dfb71ea206eb9f61e5d97c9727caa1a6449e39cb/tensorflow/contrib/cmake/external/eigen.cmake#L10 can you update that too?

Yeah, sorry for forgetting about the cmake build. I've updated it too, now.

@tensorflow-jenkins test this please

@tensorflow-jenkins test this please (once more, I don't expect the result to change.  cmake is always broken anyway :/).  Will merge if everything else passes.

Cool, the cmake test build is a good idea.
The error
execvp: /usr/local/bin/protoc: Permission denied

seems a bit bizarre. Maybe the filesystem is mounted with noexec, or SELinux is interfering?

The protobuf installation is unpacked from a zip file here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/install/install_proto3.sh#L29
Maybe the executable permissions aren't preserved when unpacking. (Although it seems to have the right permissions when I unzip that file on my laptop)

Bah, I am just going to merge this anyway, benoit has another update that should hopefully fix this anyway.

Thanks! That means #2711 should work now after a rebase.


Image display problem

When I loaded image, this problem occurred

What's going on?

It's problem of lab2rgb_LUT_opencl function

Hi,
Are you loading a .png or .jpeg image?
Ted

Nothing calls lab2rgb_LUT_opencl() directly. It overrides lab2rgb_fast at the bottom of utils.py if the OpenCL module successfully loads. You can comment out the assignment there.

@yig Yes, that's what I mean.


Improve NeMo Curator Experience for Pytorch Models (with crossfit)

Is your feature request related to a problem? Please describe.
Based on user feedback we need to fix the following to make user experience better:

[x] Enable PyTorch to share same memory pool as RMM via cli #1392
PR in dask-cuda merged, will follow up for curator in next release
[x] Fix GPU-id logging of crossfit
[x] Enable  memory estimation of crossfit+PyTorch when RMM backend is used

At risk because not all items above will complete in this sprint.  Vibhu to break into two issues.

All subtasks are completed.


Extractor raywenderlich is not working

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

C:\ytdl>youtube-dl.exe --cookies rw.txt https://www.raywenderlich.com/32317111-advanced-networking-with-urlsession --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'rw.txt', 'https://www.raywenderlich.com/32317111-advanced-networking-with-urlsession', '--verbose']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 5.1.1-full_build-www.gyan.dev, ffprobe 5.1.1-full_build-www.gyan.dev
[debug] Proxy map: {}
[RayWenderlichCourse] 32317111-advanced-networking-with-urlsession: Downloading webpage
[download] Downloading playlist: Advanced Networking with URLSession
[RayWenderlichCourse] playlist Advanced Networking with URLSession: Collected 13 video ids (downloading 13 of them)
[download] Downloading video 1 of 13
[RayWenderlich] 32317111-advanced-networking-with-urlsession/1: Downloading webpage
ERROR: Unable to extract video id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 815, in wrapper
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 836, in __extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 534, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\raywenderlich.py", line 105, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 1012, in _search_regex
youtube_dl.utils.RegexNotFoundError: Unable to extract video id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

Hi everyone! The corresponding extractor for the site https://www.raywenderlich.com/ does not work. Please help me :[

At that time, some download managers (like IDM) can download videos from there.

At that time, some download managers (like IDM) can download videos from there.

https://youtube.com/watch?v=_K-L9uhsBLM&t=70s

https://youtube.com/watch?v=_K-L9uhsBLM&t=70s

yes, this too

but they don't solve the problem

... This is irrelevant; you aren't meant to open/submit a duplicate "new" issue just because the original "existing" one(s) remain open/unresolved...
https://github.com/ytdl-org/youtube-dl#is-the-issue-already-documented
You may comment on/bump an existing issue similar/identical to the one you were about to submit, but please first read:
https://github.com/ytdl-org/youtube-dl#how-can-i-speed-up-work-on-my-issue
... not failing to pay attention to:

If nobody is interested in solving your issue, you are welcome to take matters into your own hands and submit a pull request (or coerce/pay somebody else to do so).


HBASE-25065 WAL archival to be done by a separate thread

…separate thread

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 39s
Docker mode activated.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
5m 33s
the patch passed

+1 :green_heart:
compile
1m 36s
the patch passed

+1 :green_heart:
javac
1m 36s
the patch passed

+1 :green_heart:
shadedjars
8m 52s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 54s
hbase-server in the patch failed.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
5m 45s
the patch passed

+1 :green_heart:
compile
1m 36s
the patch passed

+1 :green_heart:
javac
1m 36s
the patch passed

+1 :green_heart:
shadedjars
8m 51s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 57s
hbase-server in the patch failed.

_ Other Tests _

+1 :green_heart:
unit
231m 25s
hbase-server in the patch passed.

269m 52s

Subsystem
Report/Notes

Docker
Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2501

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 61b7e96488ce 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 9f238bd79e

Default Java
2020-01-14

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/2/testReport/

Max. process+thread count
2808 (vs. ulimit of 30000)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/2/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

This is not batching to the NN? It is just doing archiving on new background thread.
Ya I did change the JIRA title. But not here. I will try to change the title if it allows. We are only doing it in separate thread now.
If we fail to complete an archiving because RS dies, will new RS pick up the archiving?
Actually speaking even if it fails it should be fine because if a WAL is not archived we can skip it while recovered edits are replayed assuming that the seqId has already been flushed to some hfile.
BTW the entire 'abortable' case was brought in just to ensure the current behaviour. Since if the current Logroller thread trying to do the archive fails to do the archival we throw the Exception (from the AbstractFSWAL) and the log roller which has reference to the 'server' makes the 'abort' call. Since we are moving to a seperate thread and that AbstractFSWAL and its impl does not have access to 'server' we need to pass 'Abortable' every where.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 28s
Docker mode activated.

Looks to be fine.  Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hbase/pull/2501

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/5/console

versions
git=2.17.1

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m  0s
Docker mode activated.

-1 :x:
patch
0m  4s
https://github.com/apache/hbase/pull/2501 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hbase/pull/2501

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/5/console

versions
git=2.17.1

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m  0s
Docker mode activated.

-1 :x:
patch
0m  3s
https://github.com/apache/hbase/pull/2501 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hbase/pull/2501

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2501/5/console

versions
git=2.17.1

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.


generate_model_yaml macro to take original yaml file as an additional argument

Looking at the generate_model_yaml macro, I can see that it takes the following arguments:

model_name
upstream_descriptions

As I understand it, running
dbt run-operation generate_model_yaml --args '{"model_name": "dim_customer"}'
will produce a yaml similar to the following
version: 2

models:
  - name: dim_customer
    description: ""
    columns:
      - name: dim_customer_id
        description: ""

      - name: customer_id
        description: ""

      - name: customer_first_name
        description: ""

...

But what if there is an already existing yml file for dim_customers such as...
...
  - name: dim_customer
    description: ""
    columns:
      - name: dim_customer_id
        description: "this is my description"
        meta:
           - something_else:
        tests_and_other_things...:
...

Is it possible we can take these original descriptions, tests and meta things too for the new output produced by the generate_model_yaml macro?
Why someone would want this: Most of the time you are not in fact generating a model yaml from scratch, there is an already existing one, and you have made changes to it. It's really a quality of life improvement, but it would greatly improve one's workflow and make it easier to keep documentation best practices.

@charlespicowski if there were some way to left or full outer join two different YAML files into a new one, would that fill the need you are looking for?
Practically speaking, the "join" described above would probably be more like "merge file a.yml into file b.yml" or vice versa.

I have a Github Action that uses yamlpath yaml-merge using a deep array of hash method that basically does the left join that @dbeatty10 is talking about - basically the Action runs generate_source and then merges the old one. I haven't quite gotten it ready to share publicly - it's not 100% pretty, but it works.
I'm also currently experimenting with changes to generate-source() to do much the same thing using the same methods as generate_model_yaml. That method reads only the descriptions from the graph nodes and includes those values when appending the description: line to the yaml output. To expand this to include the other four possible fields for columns (meta, data_type, quote, and tags) is possible, but you'd also probably want to exclude appending empty fields (don't append meta if there's no existing value, for instance). The code gets pretty repetitive to write yaml you've already got (in the existing file) - though I think it might be possible to figure out an elegant helper function or two that can handle all of it cleanly.
Another wrinkle is it is difficult, maybe not possible, to preserve multi-line descriptions block scalars (the > followed by text on multiple lines for readibility) using the graph traversal approach of generate_model_yaml. Again, you're trying to reverse engineer generating yaml you've already got, so I'm considering pivoting back to working on the yamlpath Github Action rather than continue to develop with dbt's macro ecosystem.

I'm imagining this as a separate macro.  update_yaml_files that would use a lot of the same functionality but would actually overwrite the file (or at least have the option to).
Caveat I have no idea about the inner workings of this package so don't know if this is possible given current base.  But it has the assumption that the yamls can be merged.
Describe the feature

args

file_names: the list of files you want to update (could we default None this and just do all of them)?
upstream_descriptions: the same as generate_model_yaml
overwrite: default false?  but to overwrite the file

It would be great if it could

read all the models from the files selected
in dag order, run generate_model_yaml against that single model

dag order because I'm imagining updating a source description and then chaining that description through multiple models

merge the generated with what is currently in the yml

for exact collisions, generated always wins? unless generated is blank and in place is not?


MarshallComponentException: ('Could not convert component args to JSON', TypeError('Object of type PickerType is not JSON serializable'))

Hi,
Thanks for this widget, I would like to try it. However,  paste the time range picker example and run it, will give error like the title.
MarshallComponentException: ('Could not convert component args to JSON', TypeError('Object of type PickerType is not JSON serializable'))
If change picker_type=PickerType.time.string_value and without other parameters , the widget can show up.  but it shows the invalid date time.   Both start=default_start  and start=str(default_start) or start=default_start.strftime("%Y-%m-%d %H:%M:%S") not work. BTW, python version is 3.10.12
What is the problem?

picker_type=PickerType.time.string_value has deprecated, please refer to latest usage from README

picker_type=PickerType.time.string_value has deprecated, please refer to latest usage from README

Sure. The problem is that using  picker_type=PickerType.time will report error as the title @imdreamer2018

I tried streamlit 1.32 again( stop app, install and run again), the widget works well. It seems not work with streamlit 1.34

It is wired, I shutdown the streamlit run command and rerun again, everything works fine.  Thanks a lot


Remove reth-provider dependency for reth-eth-types

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Major refactoring and improvements (now automerge is possible)

@tstephen @mskurz
The tests are now all working.
The reason for the failing tests was simple:

The reference C.1.1 file was changed.
The tools contain a copy of the C.1.1 which has a minor change injected.
With the reference being changed, a junit test detected too many changes between the (updated) reference C.1.1 and the (old) C.1.1 with injected changes.

Now the modified C.1.1 has been updated. Everything should work now.

Closing, as a new pull request with more enhancements will be opened.


Add invited state for TestFlight beta testers.

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context
I recently pushed a PR to add ConnectAPI support to Krause's awesome clean_testflight_testers plugin, and while doing so realized Spaceship doesn't include a state for when a tester has been invited, but not yet accepted the invitation.
Description
This PR adds the INVITED state, as returned by the App Store Connect API.

@googlebot I signed it!

Ping @max-ott - review pls

Looks like not much changed, but to be 100% sure this works, can you provide simple steps to reproduce with the playground?

Ok, it took me way more than I'd like to admit to get playground working, but here goes a set of commands that will successfully test this...
...provided you have at least one test flight user with a pending invitation 😅
# Get your app
app = Spaceship::ConnectAPI::App.find("your.app.bundle.identifier")

# Grab all testers. Right now (AFAIK) this seems to be the only way to be able to pass
# the `includes: "betaTesterMetrics"` parameter. So, if you have a lot of testers, this might
# take a few minutes...
testers = app.get_beta_testers(includes: "betaTesterMetrics", limit: 200)

# Filter the list and grab the first tester that has the state we want to validate
invited_tester = testers.select { |tester| tester.beta_tester_metrics.first.beta_tester_state == "INVITED" }.first

# Validate the tester's state matches the new constante introduced in this PR
invited_tester.beta_tester_metrics.first.beta_tester_state == Spaceship::ConnectAPI::BetaTesterMetric::BetaTesterState::INVITED
=> true


Fixed arguments in SpectrogramCube to match NDCube

Description
Fixes the interface of SpectrogramCube so that it works with recent versions of ndcube.

Thanks so much for this PRn @tiagopereira.  Will review now.

Thanks so much for this PRn @tiagopereira.  Will review now.

I would suggest we need a unit test of some sort to make sure this doesn't happen again.

I would suggest we need a unit test of some sort to make sure this doesn't happen again.

Agreed.  I'm looking into that.  @tiagopereira, could you also go through the SpectrogramCubes created at the top of the test file and make they are all instantiated with kwargs?  Then could you also add a mask to spectrogram_instrument_axes.  I think that should catch it.

Agreed.  I'm looking into that.  @tiagopereira, could you also go through the SpectrogramCubes created at the top of the test file and make they are all instantiated with kwargs?  Then could you also add a mask to spectrogram_instrument_axes.  I think that should catch it.

Great!  Just realised this isn't quite enough to catch this error if it happens again.  We have a test for confirming the instrument_axes are sliced properly, but not one to check the other attributes are.  Can you add one more test that slices spectrogram_instrument_axes and then calls this ndcube function that checks all the attributes are as expected?  It will require instantiating the expected cube.  But it shouldn't take much time to do.
Also, can you replace this print statement?

Great!  Just realised this isn't quite enough to catch this error if it happens again.  We have a test for confirming the instrument_axes are sliced properly, but not one to check the other attributes are.  Can you add one more test that slices spectrogram_instrument_axes and then calls this ndcube function that checks all the attributes are as expected?  It will require instantiating the expected cube.  But it shouldn't take much time to do.
Also, can you replace this print statement?

I won't have time to work more on this for at least three weeks, but feel free to add the changes yourself if you want.

I won't have time to work more on this for at least three weeks, but feel free to add the changes yourself if you want.

@tiagopereira, I opened a PR to your branch that (I hope) fixes the outstanding issues on this PR.  I realise you don't have much time at the moment, but if you could quickly merge it on your end then hopefully I will be able to merge this PR and we can move ahead with another sunraster release.  Alternatively, I can close this PR and open another so I have merge rights.  Let me know which you prefer.

@tiagopereira, I opened a PR to your branch that (I hope) fixes the outstanding issues on this PR.  I realise you don't have much time at the moment, but if you could quickly merge it on your end then hopefully I will be able to merge this PR and we can move ahead with another sunraster release.  Alternatively, I can close this PR and open another so I have merge rights.  Let me know which you prefer.

Merged!  Thanks so much @tiagopereira for this PR.  I plan to release sunraster 0.2 in the coming weeks so this bugfix will be available.

Merged!  Thanks so much @tiagopereira for this PR.  I plan to release sunraster 0.2 in the coming weeks so this bugfix will be available.


Error when running poe lint with Podman

Describe the problem
When I run poe lint with Podman, I'm getting an error. Has anybody had this problem before?
Template
n/a
To Reproduce
poetry run poe lint

Logs
Poe => lint
Nix not found; fallback to a container
Trying to pull docker.io/nixos/nix:latest...
Getting image source signatures
Copying blob sha256:aab801e044ff3dc60056435ddad765c9c56955b2b2555c2f5b80077c540563b0
...
Copying config sha256:1e2dfe3a781d08b04fea1b00ffc96dec3881b025bf29c20cc878282e6803f69a
Writing manifest to image destination
Storing signatures
2d87cd6286ddf20d6fd7984e73bc8fee0c7d296948c193a932001d7e4fd7a677
copying path '/nix/store/p58d2j0ac7zvja5jl14xzbc19fakjxh2-source' from 'https://devenv.cachix.org'...
copying path '/nix/store/yj1wxm9hh8610iyzqnz75kvs6xl8j3my-source' from 'https://devenv.cachix.org'...
copying path '/nix/store/y9m0i9f9nk27kgiwwm0nnc72bzr1p55j-source' from 'https://devenv.cachix.org'...
copying path '/nix/store/hpmnfdwvrsdwmn9pswv0znswbb38wpvj-source' from 'https://cache.nixos.org'...
copying path '/nix/store/rlj4q2p1lax5zsvmmqzk6829c1l85r0a-source' from 'https://devenv.cachix.org'...
copying path '/nix/store/sk4ga2wy0b02k7pnzakwq4r3jdknda4g-source' from 'https://copier.cachix.org'...
error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:9:12:
            8|
            9|   strict = derivationStrict drvAttrs;
             |            ^
           10|

       … while evaluating derivation 'devenv-shell'
         whose name attribute is located at /nix/store/y9m0i9f9nk27kgiwwm0nnc72bzr1p55j-source/src/modules/mkNakedShell.nix:63:13

       … while evaluating attribute 'shellHook' of derivation 'devenv-shell'
         at /nix/store/y9m0i9f9nk27kgiwwm0nnc72bzr1p55j-source/src/modules/mkNakedShell.nix:78:5:
           77|     # requires that other trampoline.
           78|     shellHook = ''
             |     ^
           79|       ${lib.optionalString debug "set -x"}

       … while evaluating definitions from `/nix/store/y9m0i9f9nk27kgiwwm0nnc72bzr1p55j-source/src/modules/top-level.nix':

       … while evaluating definitions from `/nix/store/y9m0i9f9nk27kgiwwm0nnc72bzr1p55j-source/src/modules/top-level.nix':

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: devenv was not able to determine the current directory.
       Make sure Nix runs with the `--impure` flag.

       See https://devenv.sh/guides/using-with-flakes/
Traceback (most recent call last):
  File ".../copier/devtasks.py", line 61, in lint
    local["nix"].with_cwd(HERE)[args] & TEE
    ~~~~~^^^^^^^
  File ".../copier/.venv/lib/python3.12/site-packages/plumbum/machines/local.py", line 242, in __getitem__
    return LocalCommand(self.which(cmd))
                        ^^^^^^^^^^^^^^^
  File ".../copier/.venv/lib/python3.12/site-packages/plumbum/machines/local.py", line 203, in which
    raise CommandNotFound(progname, list(cls.env.path))
plumbum.commands.processes.CommandNotFound: (CommandNotFound(...), 'nix', [<LocalPath .../copier/.venv/bin>, <LocalPath $HOME/.pyenv/shims>, <LocalPath $HOME/.pyenv/bin>, <LocalPath $HOME/.local/pipx/bin>, <LocalPath $HOME/.local/bin>, <LocalPath $HOME/.local/bin>, <LocalPath /usr/local/sbin>, <LocalPath /usr/local/bin>, <LocalPath /usr/sbin>, <LocalPath /usr/bin>, <LocalPath /sbin>, <LocalPath /bin>, <LocalPath /usr/games>, <LocalPath /usr/local/games>, <LocalPath /snap/bin>, <LocalPath /snap/bin>, <LocalPath /opt/puppetlabs/bin>])

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File ".../copier/devtasks.py", line 84, in lint
    runner["container", "start", "--attach", "copier-lint-v1"] & TEE
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~
  File ".../copier/.venv/lib/python3.12/site-packages/plumbum/commands/modifiers.py", line 178, in __rand__
    with cmd.bgrun(
  File "$HOME/.pyenv/versions/3.12.2/lib/python3.12/contextlib.py", line 144, in __exit__
    next(self.gen)
  File ".../copier/.venv/lib/python3.12/site-packages/plumbum/commands/base.py", line 224, in bgrun
    runner()
  File ".../copier/.venv/lib/python3.12/site-packages/plumbum/commands/base.py", line 215, in runner
    return run_proc(p, retcode, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../copier/.venv/lib/python3.12/site-packages/plumbum/commands/processes.py", line 328, in run_proc
    return _check_process(proc, retcode, timeout, stdout, stderr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../copier/.venv/lib/python3.12/site-packages/plumbum/commands/processes.py", line 17, in _check_process
    proc.verify(retcode, timeout, stdout, stderr)
  File ".../copier/.venv/lib/python3.12/site-packages/plumbum/machines/base.py", line 27, in verify
    raise ProcessExecutionError(
plumbum.commands.processes.ProcessExecutionError: Unexpected exit code: 1
Command line: | /usr/bin/podman container start --attach copier-lint-v1

Expected behavior
This should work.
Screenshots/screencasts/logs
No response
Operating system
Linux
Operating system distribution and version
Ubuntu 22.04
Copier version
9.2.0
Python version
CPython 3.12
Installation method
pipx+pypi
Additional context
Copier and Python versions are irrelevant.
$ podman --version
podman version 3.4.4

Weird, the flag's there:
https://github.com/copier-org/copier/blob/d94fa6c7124ba30fbac74f3766c6de781039dfd0/devtasks.py#L25
I'll have to investigate.
As a workaround, you can let the CI do the formatting


Remove links to dcp-feedback repo

The feedback page has links to the dcp-feedback repo via the "bug report" and "suggest an idea" links.
These should be removed as this repo is no longer monitored - all user feedback should be channeled through the HCA ZenDesk.

@theathorn  this is on staging for review.
https://staging.data.humancellatlas.org/feedback

Looks good.


Question: Access to immutable database-path

Hi Simon
Is there anywhere in the app-context where one can access the hashed urlpath of the database? Currently it's included in the template-context (databases[0]["path") when rendering urls of the database (eg. /db-44b06v9/cases...), but where can I find the hashed url when rendering the index-page? I'm trying to avoid redirects. Thanks!

Thanks for the suggestion! I'll add this.

It's interesting to hear from someone who's using this feature - I'm considering moving it out into a plugin #647.

https://latest.datasette.io/.json currently returns:
{
  "fixtures": {
    "name": "fixtures",
    "hash": "87b3f2c55dfb81ff1452dd306c2623fa5550b90982cfa32bad404c4d8bbedde2",
    "color": "87b3f2",
    "path": "/fixtures",
    "tables_and_views_truncated": [

I published fixtures.db here like this:
datasette publish cloudrun fixtures.db --service datasette-hash-urls --extra-options '--config hash_urls:1'

https://datasette-hash-urls-j7hipcg4aq-uw.a.run.app/.json
{
  "fixtures": {
    "name": "fixtures",
    "hash": "bda7daa889c23f9a8f06e46d7d280dd423c76275e9593c4c1cad7c53b19032fe",
    "color": "bda7da",
    "path": "/fixtures-bda7daa",

But... https://datasette-hash-urls-j7hipcg4aq-uw.a.run.app/fixtures-bda7daa.json doesn't expose that hash:
{
  "database": "fixtures",
  "size": 258048,
  "tables": [
    {
      "name": "123_starts_with_digits",

Likewise https://datasette-hash-urls-j7hipcg4aq-uw.a.run.app/fixtures-bda7daa/complex_foreign_keys.json
{
  "database": "fixtures",
  "table": "complex_foreign_keys",
  "is_view": false,
  "human_description_en": "",
  "rows": [
    [
      "1",
      "1",
      "2",
      "1"
    ]
  ],

And https://datasette-hash-urls-j7hipcg4aq-uw.a.run.app/fixtures-bda7daa/complex_foreign_keys/1.json
{
  "database": "fixtures",
  "table": "complex_foreign_keys",
  "rows": [

@clausjuhl do you have any thoughts on what would be most useful for you in these JSON responses? The full /databasename-hash path, just the 7 character hash, or something else?

@simonw I would prefer just the 7 character hash. No need to make the urls any longer than they need to be :)


Using prefix gives an error

TypeError: Cannot read property 'sheet' of undefined
import Styletron from 'styletron-client';
import {StyletronProvider} from 'styletron-react';

const styletron = new Styletron({prefix: 'mypref'});

https://www.webpackbin.com/bins/-KoNumG8GMlbLKjevuYY

Try:
const styletron = new Styletron(void 0, {prefix: 'mypref'});

Both arguments to StyletronClient are optional, the first being the SSR style elements, and the second being an options object.
But this is indeed a bit confusing; I should make the styletron-client and styletron-server constructor interface the same.


Update protocols

Update protocols to have current versions available again

Hey,
We changed the way we pull out new protocol. The proper branch to use is the python_module one which we will merge into master.
There is a setup.py in there that does the dependency work for s2protocol.
Cheers (Schönes Grüße from the team :) )

Hey,
alright, will try that :) Maybe it's a good idea to update the README?
Schöne Grüße zurück und bis später :-D


Download chunks concurrently when in background download mode

What is the version of your Overlaybd
No response
What would you like to be added?
I would like overlaybd to download chunks concurrently when in background download mode, instead of waiting for one chunk to download before starting on the next.
Why is this needed for Overlaybd?
OverlayBD lazy pulling works amazingly if the latency between the backing OCI registry and overlaybd-tcmu is less than 2ms. Unfortunately, this isn't the case if the latency is high, for example 200-300ms when talking to quay.io. Downloading each 256Kb chunk is near instant, but it takes 200ms for a roundtrip to the registry, and containers boot slowly.
Increasing the refillSize to 10 Mb helps. Lazy pulling from quay.io works better, but still the link is nowhere near saturated, and boot times are high. Probably raising the chunk size to 100 Mb would help, but at such a high chunk size all benefits of lazy pulling are negated.
When background download is enabled the layer download speed of overlaybd should be indistinguishable from just pulling the layers ahead of time using a tool like skopeo or crane. Currently, that's not the case as downloading chunks is synchronous. Overlaybd waits for one chunk to finish downloading before starting on the next. I want to solve this by having a thread pool downloading chunks concurrently, similar to prefetch.
Are you willing to submit PRs to contribute to this feature?

[x] Yes, I am willing to implement it.

@shuaichang In my case I'm not using containerd or the overlaybd containerd snapshoter, I've integrated directly with overlaybd-tcmu which is used by the overlaybd containerd snapshotter and run qemu VM's using it. By looking at the code I think this is a fairly easy fix in overlaybd for background download to be as performant as pre downloaded blobs. Previously I've used skopeo to download all blobs ahead of time and recently implemented optional lazy pulling. I have a good setup to compare the difference after I make a PR.

The current implementation tries to avoid downloading too fast in background, in the hope that it would not influence foreground work.
Indeed the speed may be too slow for networks with high bandwidth-delay product. Welcome to make a PR to improve this case!


ksort errors under PHP 8 (with fix)

The functions loadNameIndex() and loadGroupsLightsNameIndex() in include/hueapi.php issue errors when ksort reaches a NULL value. PHP 8 has stricter type checking.
I fixed that locally by
    if (is_array($this->info[$idxname])) {
	    ksort($this->info[$idxname]);
    }

and
if (is_array($this->info['groups'][$gnum]['lightsnames'])) {
ksort($this->info['groups'][$gnum]['lightsnames']);
}
and it worked under PHP 8.0.0.

Hello,
Thank you for reporting the issue.
I don't have empty lists in my configuration and forgot to check the code against null value :(
I'll fix that asap.

Hello,
Thank you for reporting the issue.
I don't have empty lists in my configuration and forgot to check the code against null value :(
I'll fix that asap.

Corrections included into v1.7.4

Corrections included into v1.7.4


Error: Container action is only supported on Linux

would it be easy to make this compatible on Windows github CI runners...? 🙏🏻

I personally don’t use windows runners and don’t know what’s involved in writing a windows runner. But I’d imagine it’s quite different. Please feel free to make a PR if you’d like this functionality!

should be easily doable by switching to a composite action instead of docker action. 🙃


pmem-csi-operator: delete obsolete objects of a deployment

Different operator releases might result in obsolete sub-resources that were created for a deployment.  Operator shall detect these obsolete objects if any and delete them in the reconcile loop.
There is no direct API to detect all the objects owned by an object. So, we solve this by annotating the Deployment CR with the list of object types & names, and the version of the operator that reconciled. On operator upgrade/degrade we check if any pre-deployed objects are still valid.
FIXES #595

@pohly Now I rewrote the code such that instead of annotating the CR with the object list, generating the list dynamically by retrieving objects with a list of hard-coded types.

Given that all three commits are independent, wouldn't it be better to also submit them in separate PRs?


docs(documentation): fix text errors and supplement template information

Summary
Mainly modify the text and supplement the template code in Frontend Specs Templates
Details

🤖 Generated by Copilot at 695cbc6

Add pagination component to home and profile pages to enable navigation through multiple pages of articles (link, link)
Add edit profile settings button to profile page to allow updating profile information and settings (link)
Add error messages component to settings and create/edit article pages to display validation errors for user input (link, link)
Change placeholder text for password input in settings page to "New Password" to clarify password change functionality (link)
Update tag list component in create/edit article page to show an example of a tag with a delete icon to demonstrate tag management functionality (link)
Add edit article and delete article buttons to article page to allow author to modify or remove their article (link)
Remove edit icon from comment component in article page to simplify design and avoid confusion (link)
Correct text in templates.md to match code logic for header links visibility (link)

Great, thanks for the contribution!
My next step is to add more HTML fragments to separate the templates depending on the user (author/not logged in)


[Workaround] lirc (IR) support

What it the correct way to enable support of IR receiver in the system? I.e. to enable support of irm-2638 receiver connected to GPIO2_A2 pin of RK3328

So, for anyone who want to enable IR receiver on their devices (at least in 4.4 kernels):

Download the current kernel source from https://github.com/ayufan-rock64/linux-kernel/releases
Unpack the source, install all the needed tools for compilation
run make menuconfig in the kernel's source directory, enable the GPIO IR receiver driver under "Device drivers" -> "Multimedia support" -> "Remote controller devices" -> "GPIO IR remote control"
Compile the kernel and build an installation package if needed
Install the new kernel and reboot
Start dtedit from root
in ir-receiver section append the following string: gpios = <0x47 2 1>;m where 0x47 is a handle for GPIO bank 2 and 2 is the number of pin in the bank (A2 == 2)
Reboot
Load the gpio_ir_recv and create a proper file in /etc/modules-load.d/ directory for auto-load if needed
Make sure that the module was successfully initialized (see image below)
Enjoy!

See also:

How to build a kernel for Debian-based systems: https://wiki.debian.org/BuildADebianKernelPackage
GPIO Port mappings: https://github.com/Leapo/Rock64-R64.GPIO/blob/master/docs/Rock64_GPIO_Reference.xlsx
GPIO 2 bank has an 0x47 handle value, defined in gpio2@ff230000 section of device tree
pin numbering formula for user-space (used in /sys/class/gpio) = (n_bank - 1) * 32 + (pin_letter * 8 + pin_digit)
so in user-space programs you can use the following number: GPIO2_A2 = (2 - 1) * 32 + (A * 8 + 2) = 32 + (0 * 8 + 2) = pin 34 = 0x20
IR driver source code: https://github.com/rockchip-linux/kernel/blob/release-4.4/drivers/media/rc/gpio-ir-recv.c
Rock64 schematic with all connections depicted: http://files.pine64.org/doc/rock64/ROCK64_Schematic_v2.0_20171019.pdf

OK. Got a proper way to enable an IR receiver in device tree:
enable_dtoverlay gpio-ir-recv ir-receiver okay "gpios = <0x47 2 1>"

Additional info about device trees and overlays: https://www.raspberrypi.org/documentation/configuration/device-tree.md

So am I right in thinking that the main issue here is that the kernel hasn't been compiled with the necessary IR support? Meaning that if it were (compiled in as standard), it would just be a matter of turning on the right dt overlay if you want to use it?

So am I right in thinking that the main issue here is that the kernel hasn't been compiled with the necessary IR support? Meaning that if it were (compiled in as standard), it would just be a matter of turning on the right dt overlay if you want to use it?

As far as I remember - yes, you are right. The kernel must just to be built with ir-gpio support. Then the pin mappings may be set just with a proper dt overlay

I think that we just miss proper dts entries
On Sat, 9 Mar 2019 at 19:57, Sergey Kostyuk<EMAIL_ADDRESS>wrote:

So am I right in thinking that the main issue here is that the kernel
hasn't been compiled with the necessary IR support? Meaning that if it were
(compiled in as standard), it would just be a matter of turning on the
right dt overlay if you want to use it?
As far as I remember - yes, you are right. The kernel must just to be
built with ir-gpio support. Then the pin mappings may be set just with a
proper dt overlay
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ayufan-rock64/linux-build/issues/225#issuecomment-471211668,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACTpQZAJuX5FTCIqDgnNEEvHxTPcio_Yks5vVAQbgaJpZM4UuBJ6
.

I'm not so sure... I don't see any CONFIG_IR* or CONFIG_IR_GPIO_CIR in either the rockchip or mainline kernel configs... but I'll boot up a rock64 later to double check...

OK. Got a proper way to enable an IR receiver in device tree:
enable_dtoverlay gpio-ir-recv ir-receiver okay "gpios = <0x47 2 1>"

Did you mean gpios = "<0x48 2 1>"  in the  enable_dtoverlay command?  I didn't notice the hex address was different until I failed at this command, and was successful with the dtedit method.  Thank you for posting how to enable the IR.

Hey everyone. I am trying to enable the IR on RockPro64, but I need help with getting the device tree thing correct. I actually tried what was in this thread before I realized that this was for Rock64.
I found that they added it to the official linux git tree just a few months ago: https://github.com/torvalds/linux/commit/7c7f041309f73b05d21980f88f29305255e126ee
	ir-receiver {
		compatible = "gpio-ir-receiver";
		gpios = <&gpio0 RK_PA6 GPIO_ACTIVE_LOW>;
		pinctrl-0 = <&ir_int>;
		pinctrl-names = "default";
	};

// ...

	ir {
		ir_int: ir-int {
			rockchip,pins = <0 RK_PA6 RK_FUNC_GPIO &pcfg_pull_none>;
		};
	};

But the &gpio0 label does not work with ayufan's release, and I do not know how to find the numeric value. And I am guessing that pinctrl-0 is optional (but what does it do?).
Instead of RK_PA6 I can use the value 6 (defined here), and instead of GPIO_ACTIVE_LOW I can use 1 (defined here). So now I just need to figure out what &gpio0 is. I don't think it is 0x48 as in Rock64, but I am not sure.
I am afraid to try values at random since I don't know if it can be risky or not. Please help me. Thank you!

Hello @stefansundin,
For the gpio handle I do not remember how exactly I determined the value. I think you can try running dtedit to view what is already present in the device tree and which handle is assigned to gpio0. Probably the handle is assigned when dts gets compiled because the source does not mention it: https://github.com/ayufan-rock64/linux-kernel/blob/release-4.4.190/arch/arm64/boot/dts/rockchip/rk3399.dtsi#L2256. I'm not positive about both rock64 and rockpro64 using the same handles because they are based on different SoCs.
Considering that the ayufan's distribution does not get any updates nowadays, I would rather look into ArchLinux ARM and Armbian. I'm personally planning to install ArchLinux on my Rock64, but I'm not sure whether it will work for RockPro64.

Hi @s-kostyuk. Thank you for the reply.
That rk3399.dtsi file is interesting. I think this may be the answer:
		gpio0: gpio0@ff720000 {
			reg = <0x0 0xff720000 0x0 0x100>;

According to https://blog.modest-destiny.com/posts/linux-device-tree-pinctrl-tutorial/:

Here, let me explain about some magic numbers appearing in the device tree source.

gpio@48322000, reg = <0x48322000 0x1000>

The number 48322000 or 0x48322000 is the starting memory mapped address of GPIO5.

So if that is the case, then 0x0 may work instead of &gpio0. I may give it a try later.
I already tried official Debian (https://wiki.pine64.org/index.php/ROCKPro64_Software_Release#Official_Debian), but was disappointed that it didn't support video acceleration or wifi (at least I wasn't able to figure it out). ayufan's release does everything I want (besides IR, so far). There are some minor issues and the software is a bit dated, but it's the best option that I have tried so far.


Cannot install unlocked packages because code coverage check is now required

Describe the bug
Since Winter '21, unlocked packages require minimal code coverage as described here :
https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_unlocked_pkg_code_coverage.htm
The task create_package_version does not offer an option to pass the --codecoverage parameter to the force:package:version:create command, so how can we ensure that the code coverage of tests is correct during package version creation ?
Reproduction steps

cci task info create_package_versionoffers no code coverage option

Your CumulusCI and Python versions
CumulusCI version: 3.71.0 (/usr/local/bin/cci)
Python version: 3.9.12<EMAIL_ADDRESS>Operating System
macOS 11.7.3
Windows environment
No response
CumulusCI installation method
pipx
Error Gist
No response
Additional information
No response

Looking in the code, I found that when --skip_validation=False, then the --codecoverage option is passed to package version creation. This should be documented.


Django Application Fails to Connect to Cloud SQL Proxy in Docker

Bug Description
I'm trying to connect to my cloudsql database in Docker via the cloud proxy, but Django keeps throwing a connectionerror.
Example code (or command)
docker-compose.yml:
version: '3.8'

services:
    web:
        build: .
        depends_on:
            - tom_postgres_db
        command: python3 manage.py runserver <IP_ADDRESS>:8080
        volumes:
            - .:/app
        ports:
        - 8080:8000
        - 8000:8000
        env_file:
        - .env
    tom_postgres_db:
        image: gcr.io/cloudsql-docker/gce-proxy:1.16
        ports:
            - 5432:5432
        command: ./cloud_sql_proxy -instances=${INSTANCE_CONNECTION_NAME}=tcp:<IP_ADDRESS>:5432 -credential_file=/config
        volumes:
        - ${GOOGLE_APPLICATION_CREDENTIALS}:/config
        restart: always

.env:
DEBUG=1
DJANGO_SETTINGS_MODULE=my-settings
SECRET_KEY=my-secret-key
DJANGO_HOST=<IP_ADDRESS>
GOOGLE_APPLICATION_CREDENTIALS=path-to-credentials
INSTANCE_CONNECTION_NAME=my-connection-name
SQL_ENGINE=django.db.backends.postgresql
SQL_DATABASE=my-db-name
SQL_USER=my-db-user
SQL_PASSWORD=my-db-pw
SQL_PORT=5432

Stacktrace
django.db.utils.OperationalError: could not connect to server: Connection timed out
 	Is the server running on host "<IP_ADDRESS>" and accepting
        TCP/IP connections on port 5432?

How to reproduce

Run docker-compose up from project root.
See error.

Environment

OS type and version: Ubuntu 20.04
Cloud SQL Proxy version (./cloud_sql_proxy -version): 1.16

Hi @jlibermann, I'm happy to help you troubleshoot this. Can you verify that the Cloud SQL Proxy is running and ready for connections?  My suspicion is that the proxy isn't running, possibly because of an authentication issue. From looking at your 'docker-compose.yaml' file, the issue could have something to do with volume mapping the credential file,  but since I don't know what the actual value of $GOOGLE_APPLICATION_CREDENTIALS is, I can't say for sure. Can you post the logs for your tom_postgres_db service?
Could you also post your settings.py just in case it's an issue with how you've configured the database in Django?

Shubha's suspicion seems like a good one to me, I would definitely start with the things mentioned in the preceding comment.
Another thing I just happened to notice, but I'm not sure whether it's relevant/important:
The error message from Django says it was trying to connect to <IP_ADDRESS>, but the docker-compose.yml configured the proxy to listen on <IP_ADDRESS> (which means: accept connections from outside the server). If the Django app is not on the same virtual server(/host) as the Proxy, then connecting to <IP_ADDRESS> (aka localhost) will not work since the "local" host is not the same host as where the Proxy is running; you'll have to instead provide the address of where the Proxy is running somehow.  (And if the Proxy and Django app are indeed on the same host, then the Proxy's config probably doesn't need to specify that it wants to listen on <IP_ADDRESS>, though it's not "wrong".)
From these docs I stumbled upon, it seems like you should be having Django try to access the proxy via a hostname, i.e. tom_postgres_db; but I admit I've never used Docker Compose before so this is a shot in the dark :)

From looking at those docs, I think @Carrotman42 is right that using the hostname could fix the issue. A possibility is that you didn't specify the host in your Django database settings, and your application defaulted to using localhost. Try adding
{'HOST': 'tom_postgres_db'} to your database settings in settings.py and see if that fixes it.

Hi, after adding {'HOST': 'tom_postgres_db'}  and fixing the ports, I was able to run my development server. Thank you for the help!


Increase ngrok request timeout

I've noticed ngrok will return a 504 gateway timeout after about 7 minutes in to a long running HTTP request. Is this a bug or a configuration setting I have control over?

Wrong project, sorry.


A few 4.14 updates

In relation to #329 this PR

updates the CI installation job to also check 4.14 (and 5.1) opam package installation
documents the 4.14 availability in the README

To avoid triggering a entire separate CI-run for it, I include an update to set the found Flexdll issue to 'fixed'.

CI summary

a 32 bit Linux failure due to a web-server error
a Windows bytecode trunk threadomain crash #203
a Windows trunk failure to trigger STM int ref test parallel asymmetric
a Cygwin 5.1 part1 timeout due to excessive Bigarray1/Domain-spawn-join/Atomic/Hashtbl/Bytes/Ephemeron tests
a Windows bytecode 5.1 timeout due to threadomain livelock
a Windows trunk timeout due to threadomain livelock

Again, none of these are related to this PR so I'll merge


Fix: Let the user of the library pass an AsRef

Thank you for your PR! This is a great addition. :)


Please fix the href for "Elise Hawkins Nutritional Therapy" under "Services" on the merchants list

Hello!
The href mentioned is:
Elise Hawkins Nutritional Therapy
Which makes the click redirect to "https://getmonero.org/community/merchants/www.elisehawkinsnutritionaltherapy.com" – a 404 page – instead of the proper "https://www.elisehawkinsnutritionaltherapy.com" (this one will redirect to "https://www.shastanutritionaltherapy.com/", by the way).
So perhaps fixing the href to this last one would be even better?
I don't even know whose site is that, just noticed the link wasn't working properly.
Thanks!

Nice catch! Should be fixed whenever #640 gets merged.

+in progress


How to understand the training stage of videoVAE?

Hello, i found there are three stages in the training of videoVAE from report v1.1.0,
#Similar to v1.0.0, we initialized from the Latent Diffusion's VAE and used tail initialization. For CausalVideoVAE, we trained for 100k steps in the first stage with a video shape of 9×256×256. Subsequently, we increased the frame count from 9 to 25 and found that this significantly improved the model's performance. It is important to clarify that we enabled the mixed factor during both the first and second stages, with a value of a (sigmoid(mixed factor)) reaching 0.88 at the end of training, indicating the model's tendency to retain low-frequency information. In the third stage, we reinitialized the mixed factor to 0.5 (sigmoid(0.5) = 0.6225), which further enhanced the model's capabilities.
So what these three stages means? can you describe it in details? thank you!

In fact, based on our current experience, there is no need to start training from frame 9. Training at 25 or 33 frames is a good start.

In fact, based on our current experience, there is no need to start training from frame 9. Training at 25 or 33 frames is a good start.

Thanks for your reply, i have a another question, what is the initial value of mix_factor (used in time down and up sampling) during training?

Hello, i found there are three stages in the training of videoVAE from report v1.1.0,
#Similar to v1.0.0, we initialized from the Latent Diffusion's VAE and used tail initialization. For CausalVideoVAE, we trained for 100k steps in the first stage with a video shape of 9×256×256. Subsequently, we increased the frame count from 9 to 25 and found that this significantly improved the model's performance. It is important to clarify that we enabled the mixed factor during both the first and second stages, with a value of a (sigmoid(mixed factor)) reaching 0.88 at the end of training, indicating the model's tendency to retain low-frequency information. In the third stage, we reinitialized the mixed factor to 0.5 (sigmoid(0.5) = 0.6225), which further enhanced the model's capabilities.
So what these three stages means? can you describe it in details? thank you!

I couldn't find the initialization code in the source code. Could you please share the relevant initialization code，thank you.


Implement Hash and [Partial]Eq for binary types

I have a concrete use-case for this PR: using an OwnedBinary as the key in caches/maps. I'm currently forced to wrap the binary in a clumsy wrapper struct.
For completeness sake, this PR implements Hash, PartialEq, and Eq for Binary in addition to implementing PartialEq between Binary ⟷ OwnedBinary.

requested a review from yesterday

That's weird. From who? A GitHub bug?

That's weird. From who? A GitHub bug?

:arrow_up: That's what it should look like :)

LGTM, I wonder if we should have a unit test for this?

I think this is good as-is, we don't necessarily need a unit test for something like this.


setAutocorrectionType_

The setAutocorrectionType_ method call of SUITextView crashes current Pythonista 2.0 release. The performSelectorWithObject workaround does not seem to have any effect.
The bug is fixed in beta. Code needs to be modified accordingly when beta is over.

A proper workaround is added with commit 0c19309da78128dffd65a0156bbea8ec8f44a0d2


#159089798-logout

Pull Request Test Coverage Report for Build 45

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 43:
0.0%

Covered Lines:
15

Relevant Lines:
15

💛  - Coveralls


Allow records to be imported where the pk field name contains spaces.

Currently this fails with the error:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) syntax error at or near "Id"
LINE 1: SELECT id record_id, r.data->>'Display Id' Display Id
                                                           ^

[SQL: SELECT id record_id, r.data->>'Display Id' Display Id
FROM record r WHERE project_id = 'b457cf86-62e0-4049-92cd-1ddb68814a72']
(Background on this error at: https://sqlalche.me/e/14/f405)

I think there may still be issues with the record import, will raise a separate issue for this.

@simonepstein thanks a lot, this will be included in the next release :)


this.context.createScriptProcessor is not a function

Using Tone.js for my virtualpiano project
after apple ios update decided to use new version of tone (14.7.39) coz old one (13.8.25) stopped working on ios devices
im using webaudio recorder (https://github.com/higuma/web-audio-recorder-js) to record audio context of how users playing piano to let the download how they play a songs
after updating tone js recorder stopped working with error comming in console // Uncaught TypeError: this.context.createScriptProcessor is not a function//
any ideas how to deal with it?

Hi @DaniilFawkes,
Tone.js v14 uses standardized-audio-context for cross browser support. It doesn't have a ScriptProcessorNode anymore. @tambien rewrote the internal code to use an AudioWorklet instead. You basically need to do the same with your ScriptProcessor code. #712 contains an example which shows how that could be done.
I hope this helps.

thanks for chiming in @chrisguttandin
Like Christoph said ScriptProcessorNode is deprecated and won't be available in future versions. hopefully your functionality isn't too hard to reproduce in the AudioWorklet


Photography cog added

fixed #10
Basically what it does is... listens to two commands only from #hangout-zone of Photography club.
?postraw CAPTION
<Attatchment>

attachment can be image and video as well...
and it will delete the author's message (to reduce clutter) from the hangout zone and put the image as an embed with a heart reaction on #raw-pictures.
the caption is referred to as description in the code...
same for the ?postedited
working is very similar to our suggestion system...

<@515097702057508882>


Some buttons break when camera undocked and disabled

When "No Camera Behavior" is set to "Default" or "Reduced" and the camera is Popped Out/Undocked the buttons for "Send to Dock" and "Configure Settings" move and become impossible to click until camera is re-enabled.
Also, when camera is Docked and Disabled and "No Camera Behavior" is set to "Default", the same two buttons from above move into an more inconvenient location (over the nameplate of the avatar) than where they move when the "No Camera Behavior" setting is set to "Reduced".
Attached pictures to show issues with specific configurations.
Minimal UI is the only active module while taking these images.

Thanks for reporting it @Killasaurus-Warbear -
At this point, my best bet is to disable this feature... Foundry VTT keeps changing the way it works every release, and it is more pain than it's worth it to keep fixing it everytime :)

That's too bad. The feature is awesome, and my players with smaller monitors/laptops definitely appreciate it. But, I definitely understand disabling it at least until Foundry stops tweaking it so frequently. It still amazes me that this isn't a standard feature of Foundry considering how much screen real estate those boxes take up by default.

Hi @Killasaurus-Warbear - I hope I got it right this time. Fixed in .24 - Could you please check? Thanks

It looks like all the buttons are back where they belong. I'm not sure if this is the intent or not, but the avatars don't become "Reduced" down to nameplates when Popped-Out/Undocked, but everything else appears to be functioning as expected. Thanks!

It looks like all the buttons are back where they belong. I'm not sure if this is the intent or not, but the avatars don't become "Reduced" down to nameplates when Popped-Out/Undocked, but everything else appears to be functioning as expected. Thanks!

It is intended, that popout mode does not get reduced. I am mostly sure it was like this in the previous Foundry VTT version.
Floating windows with reduced mode would be weird, by just showing the name bar.
You're welcome


update milestone-2 PR

This PR updates #34

remove changes to README and milestone1
update sequence diagrams to use supported mermaid syntax, and remove execution_client calls

Thanks @metachris!
I think we should merge this one as is, close #34, and start discussing individual concerns as issues.


chore: bump nodes versions

What type of PR is this? (check all applicable)

[x] Chore

Description
Bump all nodes in prep for v4.0.0.
Merge Plan
This PR can be merged when approved

Really hope this was a script lol

Heh nope! Brute forced in a late night node frenzy


Revert "Enable static validation of the EVENT log macros (#33067)"

Commit Message:
This reverts commit 6d1363a40824ee31dd6643e03e17568c8fa502df.
Additional Description:
Broke mobile builds.
bazel build --config=remote-clang-libc++ //library/jni/types:jni_exception_lib
Risk Level: Low
Testing: Unit Tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

I think this needs to be synced past the raven PR revert
https://github.com/envoyproxy/envoy/pull/33124