internal: Container killed by YARN for exceeding memory limits. 116.8 GB of 60 GB virtual memory used. Consider boosting spark.yarn.executor.memoryOverhead.

Environment:

Python version 2.7
Spark version 2.3.1
TensorFlow version 1.13.0]
TensorFlowOnSpark version 1.3.2
Cluster version [e.g. Standalone, Hadoop 2.8, CDH5]

Describe the bug:
gs: Namespace(batch_size=100, cluster_size=2, epochs=1, format='csv', images='hdfs://ec2-54-84-34-119.compute-1.amazonaws.com:9000/user/ubuntu/mnist/csv/train/images', labels='hdfs://ec2-54-84-34-119.compute-1.amazonaws.com:9000/user/ubuntu/mnist/csv/train/labels', mode='train', model='mnist_model', output='predictions', rdma=False, readers=1, steps=1000, tensorboard=False)
2019-05-31T10:46:37.631270 ===== Start
zipping images and labels
2019-05-31 10:46:38,228 INFO (MainThread-23006) Reserving TFSparkNodes
2019-05-31 10:46:38,228 INFO (MainThread-23006) cluster_template: {'ps': [0], 'worker': [1]}
2019-05-31 10:46:38,230 INFO (MainThread-23006) listening for reservations at ('<IP_ADDRESS>', 39469)
2019-05-31 10:46:38,230 INFO (MainThread-23006) Starting TensorFlow on executors
2019-05-31 10:46:38,237 INFO (MainThread-23006) Waiting for TFSparkNodes to start
2019-05-31 10:46:38,238 INFO (MainThread-23006) waiting for 2 reservations
2019-05-31 10:46:39,239 INFO (MainThread-23006) waiting for 2 reservations
2019-05-31 10:46:40,240 INFO (MainThread-23006) waiting for 2 reservations
2019-05-31 10:46:41,242 INFO (MainThread-23006) waiting for 2 reservations
2019-05-31 10:46:42,243 INFO (MainThread-23006) waiting for 2 reservations
2019-05-31 10:46:43,244 INFO (MainThread-23006) waiting for 2 reservations
2019-05-31 10:46:44,246 INFO (MainThread-23006) waiting for 2 reservations
2019-05-31 10:46:45,247 INFO (MainThread-23006) all reservations completed
2019-05-31 10:46:45,247 INFO (MainThread-23006) All TFSparkNodes started
2019-05-31 10:46:45,247 INFO (MainThread-23006) {'executor_id': 1, 'addr': '/tmp/pymp-h7C3zs/listener-24snEb', 'task_index': 0, 'job_name': 'worker', 'authkey': '\xd7\xd2G\x19N\xb3H\xdd\xa2)Q\x99q\xdd\xbfd', 'host': '<IP_ADDRESS>', 'port': 43995, 'tb_pid': 0, 'tb_port': 0}
2019-05-31 10:46:45,247 INFO (MainThread-23006) {'executor_id': 0, 'addr': ('<IP_ADDRESS>', 46080), 'task_index': 0, 'job_name': 'ps', 'authkey': 'fj]\xffU\tL\x95\xb3\x03\r\xc1r\xc3\x13\x92', 'host': '<IP_ADDRESS>', 'port': 41702, 'tb_pid': 0, 'tb_port': 0}
2019-05-31 10:46:45,247 INFO (MainThread-23006) Feeding training data
19/05/31 10:46:58 ERROR YarnScheduler: Lost executor 1 on ip-172-16-7-225.ec2.internal: Container killed by YARN for exceeding memory limits. 98.8 GB of 60 GB virtual memory used. Consider boosting spark.yarn.executor.memoryOverhead.
19/05/31 10:46:58 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, ip-172-16-7-225.ec2.internal, executor 1): ExecutorLostFailure (executor 1 exited caused by one of the running tasks) Reason: Container killed by YARN for exceeding memory limits. 98.8 GB of 60 GB virtual memory used. Consider boosting spark.yarn.executor.memoryOverhead.
19/05/31 10:46:58 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Requesting driver to remove executor 1 for reason Container killed by YARN for exceeding memory limits. 98.8 GB of 60 GB virtual memory used. Consider boosting spark.yarn.executor.memoryOverhead.
19/05/31 10:46:58 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Requesting driver to remove executor 2 for reason Container killed by YARN for exceeding memory limits. 116.4 GB of 60 GB virtual memory used. Consider boosting spark.yarn.executor.memoryOverhead.
19/05/31 10:46:58 ERROR YarnScheduler: Lost executor 2 on ip-172-16-7-225.ec2.internal: Container killed by YARN for exceeding memory limits. 116.4 GB of 60 GB virtual memory used. Consider boosting spark.yarn.executor.memoryOverhead.
19/05/31 10:46:58 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 2, ip-172-16-7-225.ec2.internal, executor 2): ExecutorLostFailure (executor 2 exited caused by one of the running tasks) Reason: Container killed by YARN for exceeding memory limits. 116.4 GB of 60 GB virtual memory used. Consider boosting spark.yarn.executor.memoryOverhead.
19/05/31 10:47:10 WARN TaskSetManager: Lost task 0.1 in stage 1.0 (TID 4, ip-172-16-7-225.ec2.internal, executor 4): org.apache.spark.api.python.PythonException: Traceback (most recent call last):
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 229, in main
process()
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 224, in process
serializer.dump_stream(func(split_index, iterator), outfile)
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 362, in func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 809, in func
File "/home/ubuntu/TensorFlowOnSpark/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 384, in _train
File "./tfspark.zip/tensorflowonspark/util.py", line 74, in read_executor_id
with open("executor_id", "r") as f:
IOError: [Errno 2] No such file or directory: 'executor_id'
at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.handlePythonException(PythonRunner.scala:298)
at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:438)
at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:421)
at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:252)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at org.apache.spark.InterruptibleIterator.to(InterruptibleIterator.scala:28)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at org.apache.spark.InterruptibleIterator.toBuffer(InterruptibleIterator.scala:28)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at org.apache.spark.InterruptibleIterator.toArray(InterruptibleIterator.scala:28)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:939)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:939)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:109)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/05/31 10:47:11 ERROR TaskSetManager: Task 0 in stage 1.0 failed 4 times; aborting job
Traceback (most recent call last):
File "/home/ubuntu/TensorFlowOnSpark/examples/mnist/spark/mnist_spark.py", line 66, in
cluster.train(dataRDD, args.epochs)
File "/home/ubuntu/TensorFlowOnSpark/tfspark.zip/tensorflowonspark/TFCluster.py", line 92, in train
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 814, in foreachPartition
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 1056, in count
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 1047, in sum
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 921, in fold
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 824, in collect
File "/home/ubuntu/spark/python/lib/py4j-0.10.6-src.zip/py4j/java_gateway.py", line 1160, in call
File "/home/ubuntu/spark/python/lib/py4j-0.10.6-src.zip/py4j/protocol.py", line 320, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 8, ip-172-16-7-225.ec2.internal, executor 4): org.apache.spark.api.python.PythonException: Traceback (most recent call last):
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 229, in main
process()
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 224, in process
serializer.dump_stream(func(split_index, iterator), outfile)
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 362, in func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 809, in func
File "/home/ubuntu/TensorFlowOnSpark/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 384, in _train
File "./tfspark.zip/tensorflowonspark/util.py", line 74, in read_executor_id
with open("executor_id", "r") as f:
IOError: [Errno 2] No such file or directory: 'executor_id'
at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.handlePythonException(PythonRunner.scala:298)
at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:438)
at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:421)
at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:252)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at org.apache.spark.InterruptibleIterator.to(InterruptibleIterator.scala:28)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at org.apache.spark.InterruptibleIterator.toBuffer(InterruptibleIterator.scala:28)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at org.apache.spark.InterruptibleIterator.toArray(InterruptibleIterator.scala:28)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:939)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:939)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:109)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1599)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1587)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1586)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1586)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:831)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1820)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1769)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1758)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:642)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2027)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2067)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2092)
at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:939)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
at org.apache.spark.rdd.RDD.collect(RDD.scala:938)
at org.apache.spark.api.python.PythonRDD$.collectAndServe(PythonRDD.scala:153)
at org.apache.spark.api.python.PythonRDD.collectAndServe(PythonRDD.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.spark.api.python.PythonException: Traceback (most recent call last):
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 229, in main
process()
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/worker.py", line 224, in process
serializer.dump_stream(func(split_index, iterator), outfile)
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 2438, in pipeline_func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 362, in func
File "/home/ubuntu/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 809, in func
File "/home/ubuntu/TensorFlowOnSpark/tfspark.zip/tensorflowonspark/TFSparkNode.py", line 384, in _train
File "./tfspark.zip/tensorflowonspark/util.py", line 74, in read_executor_id
with open("executor_id", "r") as f:
IOError: [Errno 2] No such file or directory: 'executor_id'
at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.handlePythonException(PythonRunner.scala:298)
at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:438)
at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:421)
at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:252)
at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at org.apache.spark.InterruptibleIterator.to(InterruptibleIterator.scala:28)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at org.apache.spark.InterruptibleIterator.toBuffer(InterruptibleIterator.scala:28)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at org.apache.spark.InterruptibleIterator.toArray(InterruptibleIterator.scala:28)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:939)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:939)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:109)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
19/05/31 10:47:11 WARN TaskSetManager: Lost task 1.2 in stage 1.0 (TID 9, ip-172-16-7-225.ec2.internal, executor 4): TaskKilled (Stage cancelled)
Logs:

Install Cuda 10.0:
sudo apt-get install gnupg-curl
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_10.0.130-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1604_10.0.130-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
sudo apt-get update
wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb
sudo apt install ./nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb
sudo apt-get update

Install NVIDIA driver
Issue with driver install requires creating /usr/lib/nvidia
sudo mkdir /usr/lib/nvidia
sudo apt-get install --no-install-recommends nvidia-410
Reboot. Check that GPUs are visible using the command: nvidia-smi
Install development and runtime libraries (~4GB)
sudo apt-get install --no-install-recommends
cuda-10-0
libcudnn7=<IP_ADDRESS>-1+cuda10.0
libcudnn7-dev=<IP_ADDRESS>-1+cuda10.0
Ref: https://www.tensorflow.org/install/gpu

Steps in all instances:
sudo pip install numpy
sudo pip install tensorflow-gpu tensorflow on spark
sudo apt-get install zip

Spark Submit Command Line:
${SPARK_HOME}/bin/spark-submit
--master yarn
--deploy-mode client
--queue ${QUEUE}
--num-executors 2
--executor-memory 10G
--py-files TensorFlowOnSpark/tfspark.zip,TensorFlowOnSpark/examples/mnist/spark/mnist_dist.py
--conf spark.dynamicAllocation.enabled=false
--conf spark.memory.storageFraction=0.2
--conf spark.yarn.maxAppAttempts=1
--conf spark.driver.memoryOverhead=1G
--conf spark.yarn.executor.memoryOverhead=2G
--conf spark.executorEnv.LD_LIBRARY_PATH=$LIB_CUDA:$LIB_JVM:$LIB_HDFS:/usr/local/cuda-10.0/extras/CUPTI/lib64
--driver-library-path=$LIB_CUDA
TensorFlowOnSpark/examples/mnist/spark/mnist_spark.py
--images hdfs://ec2-54-84-34-119.compute-1.amazonaws.com:9000/user/ubuntu/mnist/csv/train/images
--labels hdfs://ec2-54-84-34-119.compute-1.amazonaws.com:9000/user/ubuntu/mnist/csv/train/labels
--mode train
--model mnist_model
Can you please help me

As the error message says, you're most likely running out of memory on the executor:
19/05/31 10:46:58 ERROR YarnScheduler: Lost executor 1 on ip-172-16-7-225.ec2.internal: Container killed by YARN for exceeding memory limits. 98.8 GB of 60 GB virtual memory used. Consider boosting spark.yarn.executor.memoryOverhead.

I'm not sure why your job is requiring so much memory.  Are you running the example as-is, or have you modified it?  i.e. are you loading any datasets into memory first?  Also note that there was an old issue #71 where someone was seeing the executor memory usage blowing up.  Unfortunately, there wasn't a conclusion on why it was occurring... I'm assuming there was something in the environment/setup.

Thanks for your reply @leewyang I have not modified the mnist example and followed GetStarted_YARN guide. But I had to use Cuda 10 and tensorflow 1.12 . --ec2-instance-type 'p2.xlarge'
Hadoop 2.7.3 . Yes I am also surprise why its using so much of yarn memory. The CPU version doesn't use so much memory. The CPU version runs successfully with the following config
--deploy-mode cluster
--num-executors 2
--executor-memory 4G \
I just realised I am using --deploy-mode client is tha a problem? but in CPU mode I was using --deploy-mode client
Please let em know if you have any further suggestion.

@biswasingh just had another thought.  If you're able, can you run a single-node GPU version of your code on just one of those EC2 instances (no Spark and no TFoS) and monitor the memory usage?
As for --deploy-mode client, I don't think that would affect it... that's just running the driver on your grid gateway... but in TFoS, the driver is mostly just orchestrating the launch of the TF nodes on the executors.

Closing due to inactivity.  Feel free to re-open if still an issue.

I met this same problem, it was virtual memory overhead while many others met physical memory overhead. And boosting spark.yarn.executor.memoryOverhead did not work. So where is the problem, spark or tensorflow?

@cheungwoonming unfortunately, it's difficult to say... the simplest way to tell is to run a single-node variant of your TF app (to remove spark and TFoS as possibilities and help isolate the issue).

@cheungwoonming unfortunately, it's difficult to say... the simplest way to tell is to run a single-node variant of your TF app (to remove spark and TFoS as possibilities and help isolate the issue).

Thanks for your reply. I will have a try.

@cheungwoonming unfortunately, it's difficult to say... the simplest way to tell is to run a single-node variant of your TF app (to remove spark and TFoS as possibilities and help isolate the issue).

Thanks for your reply. I will have a try.
Excuse me, did you solve the problem of virtual memory?


feat(energy): update "best" array to use SI prefixed units

See my comment here for additional context: https://github.com/jonahsnider/convert/pull/619#issuecomment-1804327574

Thanks for taking the time to add this in!


Can't star a repository from iOS device

Question
Response

Version?
v1.6.0

Devices tested?
iPhone 7

What Currently Happens?
If you navigate to a repository and try to Star the repository you will get an error lie below.

What Do You Expect To Happen?
The user should be able to star the repository without an error.
My Reproduction Steps

Navigate to any github repository
Press the options button in the top right corner
Press Star

Fixed by #855 but not release.

Closing the issue since the solution has been found already


feat(compression): include da compressed blocks into the block importer

chore(fuel-core): bump to v0.38
feat(compression): include da compressed blocks into the block importer

Incrementally fetches blocks instead of fetching by range, if a da compressed block exists, it inserts that into storage, if not, it fetches the full block and inserts that into storage.

closing in favor of: https://github.com/FuelLabs/fuel-block-committer/pull/142


Add umbrella reminder feature

Umbrella Reminder
Included an umbrella reminder that reminds you to carry an umbrella on rainy days. I was able to use the weather id from the weather api in order to determine the day's conditions.

Add umbrella reminder feature

@bosnian Thanks for approving! Do you mind merging it in?


Suo/chat engine

Summary

Introduce ChatEngine concept (which is basically a stateful version of QueryEngine) that exposes chat, achat, rest and a chat_repl (for ease of use in CLI and notebook)
Hooks into our main high-level API via index.as_chat_engine(chat_mode=...), configurable via ChatMode
Create 3 basic ChatEngine implementations:

condense_question: first create a condensed question from chat history and user question, then pass condensed question into query engine
react: use react agent (based on LangChain)
simple: no retrieval augmentation, just chat with LLM

Initial demo notebooks

Todos

[ ] Refine docstrings & notebooks, add to documentation
[ ] Add more implementations, e.g. inspired by semantic kernel, transformers agent, etc.

Notes

Our LLMPredictor abstraction limits our ability to properly interact with OpenAI chat API
We use a simplistic chat history representation for now.

I don't think I have any comments beyond what Jerry said, but I'm super pumped for this. It also leaves the door open for any chat+agent implementations of our own ✌🏻


VR 3D liquid not display.

When c*ming, the 3D semen is not displayed in VR mode. I have adjusted the juice to 3D detailed in the settings.

By the way, is there any way to fix my head on male character's head in VR?

Not in current version.

My KKS_VR version is 1.4, KKS version is 1.1.4, using quest3.
Milk and juice display normally, but sperm not.

Not in current version.

My KKS_VR version is 1.4, KKS version is 1.1.4, using quest3.
Milk and juice display normally, but sperm not.


[BUG] OnUserEnableLocalVideo callback event does not get triggered properly

I've been trying to subscribe a function to the OnUserEnableLocalVideo callback to show or hide each user's canvas when they start or stop streaming their webcam, but the event doesn't seem to be registering properly.
I tried subscribing this next function to the callback event to test if it actually worked, but the console only outputs the Debug.Log message once, as soon as a new user joins the room... Plus, the value of the var "enabled" prints as true, even though I set EnableLocalVideo to false on each user's Awake().
void OnUserEnableLocalVideo(uint uid, bool enabled) { Debug.Log($"{uid} - {enabled}"); }
rtcEngine.OnUserEnableLocalVideo += OnUserEnableLocalVideo;

After the user joins the room, enabling/disabling their local video doesn't trigger the callback function or print anything else on the console anymore.

Full code:
`
public class AgoraManager : MonoBehaviourPunCallbacks {
string appID = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";

public static AgoraManager Instance;

private IRtcEngine rtcEngine;

private static bool isMuted = false;
public static bool isDisplayingWebcam = false;

// A list of dimensions for switching
VideoDimensions[] dimensions = new VideoDimensions[]{
    new VideoDimensions { width = 640, height = 480 },
    new VideoDimensions { width = 480, height = 480 },
    new VideoDimensions { width = 480, height = 240 }
};
int dim = 0;
private const float Offset = 100;

private void Awake() {
    // Singleton
    if (Instance != null)
        Destroy(gameObject);
    else {
        Instance = this;
        DontDestroyOnLoad(gameObject);
    }

    // Inits the RTC Engine and subscribes some functions to the RTC Agora callbacks
    InitEngine();
}

private void Update() { RequestPerms(); }

#region Initialize
// Function that inits the RTC Engine
void InitEngine() {
    rtcEngine = IRtcEngine.GetEngine(appID);

    rtcEngine.SetChannelProfile(CHANNEL_PROFILE.CHANNEL_PROFILE_LIVE_BROADCASTING);
    rtcEngine.SetClientRole(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);

    rtcEngine.EnableAudio();
    rtcEngine.EnableVideo();
    rtcEngine.EnableLocalVideo(false);
    rtcEngine.EnableVideoObserver();

    rtcEngine.OnJoinChannelSuccess += OnJoinChannelSuccess;
    rtcEngine.OnUserJoined += OnUserJoinedHandler;
    rtcEngine.OnLeaveChannel += OnLeaveChannel;
    rtcEngine.OnUserEnableLocalVideo += OnUserEnableLocalVideo;
    rtcEngine.OnError += OnError;
    rtcEngine.OnConnectionLost += OnConnectionLostHandler;

    SetVideoEncoderConfiguration(0);
}

// Returns the RTC Engine
public IRtcEngine GetRtcEngine() {
    return rtcEngine;
}

// Requests Microphone perms until the user accepts them
private void RequestPerms() {
    PermissionHelper.RequestMicrophontPermission();
    PermissionHelper.RequestCameraPermission();
}

public void SetVideoEncoderConfiguration(int dim = 0) {
    if (dim >= dimensions.Length) {
        Debug.LogError("Invalid dimension choice!");
        return;
    }

    VideoEncoderConfiguration config = new VideoEncoderConfiguration {
        dimensions = dimensions[dim],
        frameRate = FRAME_RATE.FRAME_RATE_FPS_15,
        minFrameRate = -1,
        bitrate = 0,
        minBitrate = 1,
        orientationMode = ORIENTATION_MODE.ORIENTATION_MODE_ADAPTIVE,
        degradationPreference = DEGRADATION_PREFERENCE.MAINTAIN_FRAMERATE,
        mirrorMode = VIDEO_MIRROR_MODE_TYPE.VIDEO_MIRROR_MODE_AUTO
    };
    rtcEngine.SetVideoEncoderConfiguration(config);
}
#endregion

#region MicSwitchState
// Function that reverts the mute state of the user's audio input source
public void Mute() {
    isMuted = !isMuted;
    UIManager.Instance.ToggleUIButtonColor("mic", isMuted);
    rtcEngine.MuteLocalAudioStream(isMuted);
    Debug.Log($"You are now {(isMuted ? "muted" : "unmuted")}!");
}

// Function that sets the mute state of the user's audio input source
public void MicSwitchState(bool _isMuted) {
    isMuted = !_isMuted;
    this.Mute();
}
#endregion

#region CamSwitchState
public void OnButtonCameraSwitchActive() {
    if (!isDisplayingWebcam)
        MakeVideoView(0);
    else
        DestroyVideoView(0);
}

public void CamSwitchState(bool isDisplay) {
    isDisplayingWebcam = !isDisplay;
    OnButtonCameraSwitchActive();
}
#endregion

#region Webcam Video Streaming
public void MakeVideoView(uint uid) {
    GameObject go = GameObject.Find(uid.ToString());
    if (!ReferenceEquals(go, null))
        return; // Reuse

    // Create a GameObject and assign to this new user
    VideoSurface videoSurface = MakeImageSurface(uid.ToString());
    if (!ReferenceEquals(videoSurface, null)) {
        // Configure videoSurface
        videoSurface.SetForUser(uid);
        videoSurface.SetEnable(true);
        videoSurface.SetVideoSurfaceType(uid.ToString() != "0" ? AgoraVideoSurfaceType.Renderer : AgoraVideoSurfaceType.RawImage);
    }
}

private VideoSurface MakeImageSurface(string goName) {
    GameObject go;

    if (goName != "0") {
        go = GameObject.CreatePrimitive(PrimitiveType.Plane);
        go.transform.name = goName;
        Destroy(go.GetComponent<MeshCollider>());

        GameObject userGO = SpatialAudio.GetPlayerFromUID(goName);
        go.transform.parent = userGO.GetComponent<SpatialAudio>().webcamParent;
        go.transform.localPosition = Vector3.zero;
        go.transform.localRotation = Quaternion.Euler(Vector3.zero);
        go.transform.localScale = Vector3.one;

        MeshRenderer mr = go.GetComponent<MeshRenderer>();
        mr.shadowCastingMode = UnityEngine.Rendering.ShadowCastingMode.Off;
        Material mat = new Material(Shader.Find("Unlit/Texture"));
        mat.color = Color.white;
        mr.material = mat;
        return go.AddComponent<VideoSurface>();
    }
    else {
        if (!isDisplayingWebcam) {
            rtcEngine.EnableLocalVideo(true);
            Debug.Log($"Webcam data stream started!");

            isDisplayingWebcam = true;
            UIManager.Instance.ToggleUIButtonColor("cam", isDisplayingWebcam);
            UIManager.Instance.canvasProxy.cameraImage.transform.parent.gameObject.SetActive(true);
            return UIManager.Instance.canvasProxy.cameraImage.GetComponent<VideoSurface>();
        }
        else return null;
    }
}

public void DestroyVideoView(uint uid) {
    if (uid != 0) {
        GameObject go = GameObject.Find(uid.ToString());
        Destroy(go);
    }
    else {
        rtcEngine.EnableLocalVideo(false);
        Debug.Log($"Webcam data stream stopped!");

        isDisplayingWebcam = false;
        UIManager.Instance.ToggleUIButtonColor("cam", isDisplayingWebcam);
        UIManager.Instance.canvasProxy.cameraImage.transform.parent.gameObject.SetActive(false);
    }
}
#endregion

#region Callbacks [Photon] (Syncs Photon and Agora rooms)
// On Error callback [PHOTON]
public override void OnJoinedRoom()  {
    if (PhotonNetwork.CurrentRoom.Name != "Main")
        rtcEngine.JoinChannel(PhotonNetwork.CurrentRoom.Name);
}

// On Left Room callback [PHOTON]
public override void OnLeftRoom() {
    rtcEngine.LeaveChannel();
}
#endregion

#region Callbacks [Agora]
// On Join Channel Success callback [AGORA]
void OnJoinChannelSuccess(string channelName, uint uid, int elapsed) {
    Debug.Log($"Channel {channelName} joined!");

    // Creates Player's custom properties as hashtable
    Hashtable hash = new Hashtable();
    hash.Add("agoraID", uid.ToString());
    PhotonNetwork.SetPlayerCustomProperties(hash);

    // Refreshes the player last scene's mic and cam state to the new one
    this.MicSwitchState(isMuted);
    this.CamSwitchState(isDisplayingWebcam);
}

void OnUserJoinedHandler(uint uid, int elapsed) {
    Debug.Log($"User {uid} joined!");
    //MakeVideoView(uid);
}

// On Leave Channel callback [AGORA]
void OnLeaveChannel(RtcStats stats) {
    Debug.Log($"Left channel with duration {stats.duration}s...");
}

void OnUserEnableLocalVideo(uint uid, bool enabled) {
    Debug.Log($"{uid} - {enabled}");
}

// On Error callback [AGORA]
void OnError(int error, string msg) {
    Debug.Log($"Error with agora:{(msg != "" ? $" {msg}\n" : " ")}(ERROR ID: {error})");
}

void OnConnectionLostHandler() {
    Debug.Log(string.Format("Connection lost..."));
}
#endregion

void OnApplicationQuit() {
    Debug.Log("Application quitted...");
    if (rtcEngine != null) {
        rtcEngine.LeaveChannel();
        rtcEngine.DisableVideoObserver();
        IRtcEngine.Destroy();
    }
}
}

`

Try subscribing to OnUserMuteVideo instead.

I've tried it and, to be honest, video-related callbacks are being super inconsistent right now.
I've tried using "OnUserEnableLocalVideo", "OnUserMuteVideo" and "OnRemoteVideoStateChanged" to check if a user has enabled their video or not, and there have been times where those callbacks have been fired all the time correctly and other times they just fire once on startup.
I'm at the point where they just stop working completely, while I'm absolutely certain that video is being enabled/disabled by other users.
I really don't know what makes it work sometimes but most of the time they won't work at all.

@hoomoongoos  our test show the callbacks work properly except it could be too much if all these events are registered.  Underneath they corresponds to whether the user's stream is published or not.  There isn't a really a one-to-one functional match to muteVideo vs. disableLocalVideo.  Please bear in mind that as a wrapper, the WebGL SDK is micmicing the native does but they are not 1-to-1 in parity.  I would suggest to use OnUserMuteVideo if you are using the RtcEngine instance or OnRemoteVideoStateChanged if you are using multichannel in your implementation, but not all of them.   That said, we will try to implement OnUserEnableLocalVideo as the direct response to this issue.

I agree with @icywind 's previous comment. OnUserEnableLocalVideo hasn't been properly implemented and will need some work before it's functional. But OnUserMuteVideo does seem to work fine with the RtcEngine. I've been testing it myself.
Here's how I'm using the callback:
mRtcEngine.OnUserMuteVideo = userVideoMutedHandler; void userVideoMutedHandler(uint uid, bool muted)     {         logger.UpdateLog(string.Format("onUserMuteHandler uid: {0}, muted: {1}", uid, muted));     }
I'll be pushing the adjustments I made to the repo. But copy and paste over the following functions in clientManager.js to fix a bug where it's being called too many times:
handleUserPublished():
async handleUserPublished(user, mediaType) {     const id = user.uid;     if (this.audioSubscribing && mediaType == "audio" && (mediaType == "audio" && this.screenShareClient == null       || mediaType == "audio" && this.screenShareClient != null       && id != this.screenShareClient.uid)) {       await this.subscribe_remoteuser(user, mediaType);     } else if(this.videoSubscribing && mediaType == "video" && remoteUsers) {       await this.subscribe_remoteuser(user, mediaType);       this.getRemoteVideoStats(id);     }     remoteUsers[id] = user;   }
handleUserUnpublished():
handleUserUnpublished(user, mediaType) {     const id = user.uid;     // delete remoteUsers[id];     // $(#player-wrapper-${id}).remove();     var strUID = id.toString();   }
this will remove the unnecessary event callback functions for muting when publishing and unpublishing. Let us know how it goes for you and if you're still having trouble with OnUserMuteVideo still.


Lab 11 - Minor issue with Function App

Lab 11 - Azure Functions
Exercise 2 - Create Azure Function
Task 2.1 - Azure Resources
Step: 3
Description of issue
Part of the instructions in Step 3 are to pick .NET as the Runtime stack and "6 (LTS)" for the version.   In the Azure portal there is some confusion because there are two choices for version 6...  One is "6 (LTS) in-process" and the 2nd is "6 (LTS) Isolated."   It would be better to specify exactly which one the student should choose.

Should be able to choose either, but would suggest Isolated as that will be the only option in the future

Thank you @tylerfarmer1. We will narrow it down in our next update.

FYI - the "Isolated" one caused us issues in class.   Part of some of the labs have you edit some directly in the Azure portal.  One of the options - I think it was the in-process version - would show you the button to edit directly....but the other option would NOT show that button.


Port "Emitter: remove unused code in prepareTagPrefix"

https://bitbucket.org/snakeyaml/snakeyaml-engine/commits/62238f0804db8c499fccfb9df789b50eb35810b7

This piece of code doesn't exist in the current revision under main:
https://github.com/krzema12/snakeyaml-engine-kmp/blob/d8160db9e364357c9de413f85622ca885516180c/src/commonMain/kotlin/it/krzeminski/snakeyaml/engine/kmp/emitter/Emitter.kt#L287-L291
Perhaps depends on another commit to port.

@aSemy I know it's been a long time ago, but do you happen to recall why we removed this piece of code when converting to Multiplatform?
https://github.com/krzema12/snakeyaml-engine-kmp/pull/48/files#r1890452116

I noticed it when trying to port a change from snakeyaml-engine. I thought that we somehow split this method, but I cannot find an equivalent functionality in another place of the Emitter. It also doesn't seem related to porting to Multiplatform. If we fail to recall why we made this change, I think we should reinstate this piece.

My speculative explanation is that SnakeYAML was originally converted from PyYAML. Maybe SnakeYAML introduced a bug during this conversion, or PyYAML had to make a copy of the string because strings (or whatever datatype held the string data) in Python wasn't immutable. So this function made a copy for safety.


Duplicate Symbols for architecture x86_64

I am using fbsdk-next version 4.3 , with Admost
for using admost i should install the required pod which one of them is
  pod 'AMRAdapterFacebook', '~> 6.5'
after installing , i got the following buildtime error.

which is addressing to /FBSDKCoreKit_Basics/libFBSDKCoreKit_Basics.a(FBSDKUserDataStore.o)

Please search, then post: https://github.com/thebergamo/react-native-fbsdk-next/issues?q=duplicate+symbols+x86_64


FaceRecognition

In 26 lines of code, one can do face recognition it uses opencv library

Ok! Could you add your project name (and maybe a screenshot of it as well) to the main readme file so that I can merge your PR?

done
On Mon, Oct 3, 2022 at 5:38 PM prajaktasathe @.***>
wrote:

Ok! Could you add your project name (and maybe a screenshot of it as well)
to the main readme file so that I can merge your PR?
—
Reply to this email directly, view it on GitHub
https://github.com/PrajaktaSathe/Python/pull/92#issuecomment-1265346084,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APKSMGHFA4DYGU2E2DVC4RLWBLEDXANCNFSM6AAAAAAQ25AT6U
.
You are receiving this because you authored the thread.Message ID:
@.***>

Merged!


Remove Example

Example used here was irrelevant as it was for the previous question.

Hi @alexhagiopol,
Thanks for your response!
I have updated the example on both Python and C++ version of the problem. Hope this will suffice.

@Vipul97 Thanks for the contribution. I will merge it.


Clarify how maintenance_info objects should be compared.

MaintenanceInfoConflict error should only be produced when
maintenance_info.version doesn't match. Other fields of Maintenance
Info object should be ignored in such checks.
Adjust wording in the request/response body definitions to reflect the
above better.

This change is needed for the following reasons:

The typo fix in the description of the Maintenance Info object
should not cause new maintenance_info version to be created.
Platforms don't have to store or send back to the Service Broker all
the fields of the maintenance_info.
In general, the maintenance_info.version is what matters from
perspective of checking whether two objects are same or not.

#165995791
What is the problem this PR solves?
A description of what the problem is, or a link to an issue which this is related to.
Checklist:

[x] The swagger.yaml doc has been updated with any required changes (no changes required)
[x] The openapi.yaml doc has been updated with any required changes (no changes required)

@mattmcneeney We’ve found out that there was an * (asterisk) symbol missing on the required property version of the maintenance_info object, so I pushed a new commit to fix it here.
And GitHub decided to dismiss your review :(

Hey @fmui @jberkhahn @mattmcneeney
I’ve fixed merge conflicts (because of this was merged #669). Could you please review this change again?


duqtools - cleanup command

Remove any IDS generated by duqtools and clean run directories.

This should read runs.yaml to figure out which ids have been written for this directory. It also contains a list of the run directories, but it may make more sense to delete all subdirectories.

[ ] Delete ids data (check runs.yaml)
[ ] Delete runs.yaml (check Runs in create.py for BaseModel)
[ ] Delete subdirectories defined in runs.yaml
[ ] Delete other subdirectories matching our structure (i.e. cfg.workspace.cwd.glob('run_*'))


Remove "under construction" for "Developing with Angular"

What does this PR do?
Add Resource(s) | Remove Resource(s) | Add info | Improve Repo
Remove "under construction"  for "Developing with Angular" book in Angular section
For resources
Description
Why is this valuable (or not)
How do we know it's really free?
For book lists, is it a book?
Checklist:

[x] Not a duplicate
[ ] Included author(s) if appropriate
[ ] Lists are in alphabetical order
[ ] Needed indications added (PDF, access notes, under construction)

Congratulations!


ISPN-14786 Clean up leftovers from the WF modules removal

https://issues.redhat.com/browse/ISPN-14786

Integrated into main, thanks @tristantarrant !


KafkaAdmin clusterId configuration is ignored when observability is enabled and bootstrap supplier is not set

Backport of gh-3402

Fixed via c783ad6289f95f36ed3e034de2f2e7c80bfad1f1


Ticket description shouldn't appear in "follow-ups" section?

It seems that the ticket description is always the first follow-up and this bothers me for two reasons... First, the original description is not a follow-up. Second, the needless redundancy can be annoying, especially when the description is long as tends to be the case on my site.

My guess is that this needs to be done as-is because there's no way to attach a file to the ticket description. If an incoming email generates a new report, and that email has an attachment, it would be lost unless it is attached to a FollowUp. Maybe it could leave out the FollowUp unless an attachment is detected (and even then, only add the attachment to a blank FollowUp)? I do agree that reducing redundancy is a good goal.

In hindsight, I think FollowUp is a poor naming choice here.
If you think of each Ticket as a conversation heading from request through to resolution, the initial report is the first part of the discussion thread. It is then followed by a series of follow-ups, comments, questions, and clarifications. In this context, it makes sense (to me at least 😄) that the Ticket description has equal prominence to the FollowUp items that follow.
Do you agree? If so, should we rename FollowUp (at least in the UI) and remove the description from the ticket display?
I'm not saying I'm right - this is just my line of thinking - and I'm open to further thoughts :)

Removing description from the ticket display and renaming follow-ups in the UI works for me! I agree that it's good to keep the entire conversation together.

I don't think Ticket description is used anywhere else, is it? I like keeping conversation together and removing description.

I agree with @adrianboyko's suggestion to remain Follow-ups to Discussion. I initially have challenge with that choice of word but ignored it

Thank you for the input @adrianboyko and @Pauldic. A change is clearly needed, I just wish we could float the options out to a larger audience -- where's that django-polls app when you need it? 😉
I'll try out the replacement suggested here on my local guinea pigs and gauge their reaction, and collect any other possibilities that come up here in this ticket. It might not happen before 0.2.0 gets sent off to press at PyPI, but, barring any sudden revolt by current users, should be feasible for 0.2.1.


Anchors not working as expected with case-insensitive regex

Bug report
System info: InfluxDB 1.2.0 on OS X 10.12.2 (installed with Homebrew)
Steps to reproduce:

Create a test database

Populate with test data
 h2o_feet,location=santa_monica water_level=10 0
 h2o_feet,location=santa_monica water_level=12<PHONE_NUMBER>0
 h2o_feet,location=santa_monica water_level=11<PHONE_NUMBER>0
 h2o_feet,location=santa_monica water_level=13<PHONE_NUMBER>0

Attempt to use a case-insensitive regular expression with anchors:
 SELECT * FROM h2o_feet WHERE location =~ /(?i)^SANTA_MONICA$/

Expected behavior: Should return four rows
Actual behavior: Returns no rows
Additional info: Note that leaving off EITHER anchor returns the expected rows. Replacing either anchor with \b also returns expected rows.

@sethm thanks for this report. I think I know where the issue is. As a workaround for the moment you could, as you suggest, use =~ /(?i)^SANTA_MONICA/ or =~ /(?i)SANTA_MONICA$/ or =~ /(?i)SANTA_MONICA/ if any of those are precise enough for you.

Addendum: I just discovered that the anchors work properly when in a subquery, but not when in a main query. I hope that helps narrow it down.


Fix some errors when update profiles.

Coverage decreased (-0.07%) to 38.227% when pulling 77b8e67b403f7c872efc42f010aa7c224a8eb266 on wisererik:development into 16831ba0c286987676ae2b6447f8747c0d413627 on opensds:development.


Plugin continues operating with stale N1MM+ rig state once RadioInfo data stops arriving

The plugin receives all its awareness via RadioInfo XML documents sent over UDP, on a cadence determined by N1MM+.  The plugin uses these documents to update its internal state representations of RadioNr 1, and also 2 if sent to the plugin.
When N1MM+ changes configs, is crashed, exited, or otherwise inhibited from sending RadioInfo documents, the plugin will be none the wiser, and continue retaining the stale RadioInfo message.  This can manifest in a number of ways:

RadioNr 2 is deconfigured by user reverting to SO1V setting (or kill and restart SO2V/R N1MM+ with a different config for SO1V
Disable RadioInfo messages from being sent
Change the RadioInfo IP/port list to exclude the plugin from receipt
Set N1MM+ to send messages to plugin on an IP address or network interface that is subject to disturbance, disconnect or re-IP
Exit or kill N1MM+ and don't restart it
Designated UDP IP and port for RadioInfo receipt by plugin is pre-empted by another application that opens its UDP socket in exclusive mode (loopback address is always exclusive-use)

In any of these cases, the plugin will continue operating with the stale active radio information.
One issue that can arise during normal usage if the actual configuration changes from SO2V or SO2R to a single rig SO1V state. In this case, the RadioNr 2 messages simply stop arriving, and the plugin continues operating fine with RadioNr 1, but there is a small amount of wasted memory and a lost performance optimization because the plugin is elsewhere still checking which radio is in use.
These issues could be remedied by agreeing with the N1MM+ developers on a staleness timeout period, in which case the stale radio(s) will be deconfigured.  In the case that both rigs are lost, the plugin will go back to its startup no-op state of discarding all actions while awaiting RadioInfo status.  Special care is needed to ensure we don't start operating with RadioNr 2 info until after we also get RadioNr 1 info, as this might have unexpected anomalous behavior.

N1MM+ developers recommended a 15 second timeout, which is now implemented to eventually detect:

transition from multi-VFO/rig back to SO1R.
loss of radio 1
loss of N1MM+ connectivity.
These statuses will be surfaced in the N1MM+ graphical button on all sample pages, as documented in the readme.


Cross compile section could be improved

I just fell across the guidelines for cross compiling this project from the retired project (plietar/librespot) on https://awesomeopensource.com/project/plietar/librespot and they still works. Shouldn't those cross compiling guidelines also be part of the current compile guidelines

Have you looked at the wiki section on cross compiling?

Now I have 👍


[widgets] DraggableScrollableSheet's listView (scrollController) is getting detached from sheet

Steps to Reproduce
DraggableScrollableSheet's listView (scroll controller) is getting detached from the sheet when the keyboard appears on the screen.

Execute flutter run on the code sample
Drag the BottomSheet down to min extent.
Click on TextField (Observation: Actual result point 1).
Scroll the bottom sheet upwards (Observation: Actual result point 2).

Expected results:

Sheet should adjust itself to the top of the List and the text mentioned in the sheet should be visible

Actual results:

The text in the sheet will be detached from the sheet and gets hidden under the sheet.
Now if you scroll the list it will behave as shown in the video

Code sample

import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(),
    );
  }
}

class MyHomePage extends StatelessWidget {
  const MyHomePage({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('Flutter Demo Home Page')),
      body: Stack(
        children: [
          Container(color: Colors.amber),
          DraggableScrollableSheet(
            builder: (context, scrollController) => Card(
              child: ListView(
                controller: scrollController,
                children: const [
                  Text("Hello Observer this text, "
                      "it will get ditched from the Bottom sheet"
                      " when the keyboard appears"),
                  Divider(thickness: 6),
                  SizedBox(height: 32),
                  TextField(
                      decoration:
                          InputDecoration(border: OutlineInputBorder())),
                ],
              ),
            ),
          )
        ],
      ),
    );
  }
}

flutter doctor
[✓] Flutter (Channel stable, 3.3.9, on macOS 13.0.1 22A400 darwin-arm, locale
    en-IN)
    • Flutter version 3.3.9 on channel stable at /Users/padya/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision b8f7f1f986 (8 days ago), 2022-11-23 06:43:51 +0900
    • Engine revision 8f2221fbef
    • Dart version 2.18.5
    • DevTools version 2.15.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/padya/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14B47b
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)

[✓] IntelliJ IDEA Community Edition (version 2022.1.3)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.74.0-insider)
    • VS Code at /Applications/Visual Studio Code - Insiders.app/Contents
    • Flutter extension version 3.52.0

[✓] Connected device (3 available)
    • iPhone SE (3rd generation) (mobile) • 7648258C-33E0-4267-BB5C-DBE321AA34D9
      • ios            • com.apple.CoreSimulator.SimRuntime.iOS-16-1 (simulator)
    • macOS (desktop)                     • macos
      • darwin-arm64   • macOS 13.0.1 22A400 darwin-arm
    • Chrome (web)                        • chrome
      • web-javascript • Google Chrome 107.0.5304.121

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

 Video
https://user-images.githubusercontent.com/58727124/204785694-5539c58c-02e1-4978-899a-cd7c0059c13d.mov

Hi @DattatreyaReddy
I see there's an open issue addressing the case you described https://github.com/flutter/flutter/issues/94942
Please follow up on that issue, I'm closing the current one as a duplicate.
If you disagree, please write in the comments and I will reopen it.
Thank you


WFLY-17561 Added test bom dependency in microprofile

Issue: https://issues.redhat.com/browse/WFLY-17561
WFCORE issue integration jobs are failing https://issues.redhat.com/browse/WFCORE-6203 so need to add these dependencies in WFLY.
Integration jobs are passing after adding these changes.

Thanks @parsharma


FileNotFoundError: No such file or no access: ‘image_in_RPI_norm.nii’ with kernel 3D

This bug happens when using sct_deepseg_sc with -kernel 3D.
Here's the syntax and error:

sct_example_data/t2 $ sct_deepseg_sc -i t2.nii.gz -c t2 -brain 1 -kernel 3d

Method:
	Centerline algorithm: svm
	Assumes brain section included in the image: True
	Dimension of the segmentation kernel convolutions: 3d

Create temporary folder (/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417130541.576773-_NcN1m)...
Reorient the image to RPI, if necessary...
Finding the spinal cord centerline...
Remove temporary files...
rm -rf /var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417130544.886136-6pIv9s
Resample the image to 0.5 mm isotropic resolution...
Cropping the image around the spinal cord...
Normalizing the intensity...
Traceback (most recent call last):
  File "/Users/julien/code/sct/scripts/sct_deepseg_sc.py", line 185, in <module>
    main()
  File "/Users/julien/code/sct/scripts/sct_deepseg_sc.py", line 159, in main
    brain_bool=brain_bool, kernel_size=kernel_size, remove_temp_files=remove_temp_files, verbose=verbose)
  File "/Users/julien/code/sct/spinalcordtoolbox/deepseg_sc/core.py", line 640, in deep_segmentation_spinalcord
    'mm', 'linear', verbose=0)
  File "/Users/julien/code/sct/spinalcordtoolbox/resampling.py", line 138, in resample_file
    nii = nipy.load_image(fname_data)
  File "/Users/julien/code/sct/python/lib/python2.7/site-packages/nipy/io/files.py", line 57, in load
    img = nib.load(filename)
  File "/Users/julien/code/sct/python/lib/python2.7/site-packages/nibabel/loadsave.py", line 42, in load
    raise FileNotFoundError("No such file or no access: '%s'" % filename)
nibabel.py3k.FileNotFoundError: No such file or no access: 'image_in_RPI_norm.nii'

In addition to fixing the bug, I suggest to increase the sensitivity of the unit test to catch that error.
Also see: http://forum.spinalcordmri.org/t/sct-deepseg-sc-error-using-3d-kernel-v4-0-0-beta-2/56

Fixed in b5ea13a4689903b08546b9a10f03c6fb7b60fcef


fzf: 0.11.1 -> 0.11.4

Things done:

[x] Tested using sandboxing (nix-build --option build-use-chroot true or nix.useChroot on NixOS)
[x] Built on platform(s): NixOS
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Plus a little style fix

Is the inclusion of the second commit intentional? Seems sensible, just making sure.

Yes, it's just a bit of cleanup. I agree that it's not directly related to the PR


In example not work button «Войти по Сбер ID» and «SberPay»

I see in log, the try open some url and sber:// , but nothing happens on iOS.
Can you help how this must work ?

Hello, please attach the logs so we can look into the problem and help you.
sber:// is a deeplink, it should open the Sber app if it is installed and continue paying there.


[ko] 1st Korean localization work for v1.26

This PR is 1st Korean localization work for v1.26

[x] #38458
[x] #41518
[x] New Korean documents

Thanks to the contributors !
closes #38413

/hold
WIP. We need to complete #38458

Hi @jihoon-seo
Could you open a PR to resolve the merge conflicts ? (open a PR targeting to dev-1.26-ko.1)
It is related with https://github.com/kubernetes/website/pull/39787.
I guess data/i18n/en/en.toml layouts/shortcodes/cve-feed.html have been changed after #39787

/hold cancel

/assign

/approve

/approve cancel

/lgtm

/approve

Hi @natalisucks @a-mccarthy :)
This PR is to merge Korean localization work for v1.26 into the main.
This PR needs additional approval from English approvers since the PR includes changes to data/i18n/en/en.toml and layouts/shortcodes/cve-feed.html which Korean approvers cannot approve. The changes here are just to sync the Korean development branch with the main branch in order to resolve merge conflicts in PR: [ko] 1st Korean localization work for v1.26. The changes to data/i18n/en/en.toml and layouts/shortcodes/cve-feed.html will be identical with the original in the latest main branch, and effective to dev-1.26-ko.1 branch only.
The commit related with the merge conflict in this PR was from https://github.com/kubernetes/website/pull/41322#issuecomment-1567657012 directly.

Happy to do so @seokho-son! 🚀
/approve


Pass upstream client's user agent through to registry on image pulls

Summary:  pass through to registry an upstream user-agent on image pull.  Closes issue #20001.
What I did:  People running Registry servers should be able to see what client initiated an image pull per issue #20001.  For example, if Compose instructs Engine to pull from a local Registry, then the User-Agent header sent to that Registry server should be the following RFC 7231 compliant [1] string concatenation:
User-Agent:  [Docker Compose UA] upstream-ua/end; [Docker Engine UA]

upstream-ua/end; is just a delimiter.  (Open to suggestions for a better one.)
How I did it:  When Engine receives an image pull API request, I set a golang.org/x/net/context context inside the closure returned by NewUserAgentMiddleware() in api/server/middleware/user_agent.go.  This context includes the upstream client's UA string (e.g., Compose's UA string).  From there, I just plumbed the context variable all the way through the request processing chain until DockerUserAgent() is called.  DockerUserAgent() was modified to accept an optional string argument, which is read back from the context by the caller.
How to verify it:

Run a Registry server on port 5000 of localhost:

docker run -d -p 5000:5000 --restart=always --name registry registry:2

See [2] for more information.

In a separate terminal, tail the registry server's logs:

docker logs -f [REGISTRY CONTAINER ID]

Put an image in your empty registry like this:

docker pull hello-world
docker tag hello-world localhost:5000/hello-world
docker push localhost:5000/hello-world

Do an image pull from your registry via Engine but using a custom User-Agent header:

curl -X POST -A "Goelzer/99.9 (Goelzer; en-us)" "http://<IP_ADDRESS>:2375/images/create?fromImage=localhost:5000/hello-world" -H "X-Registry-Auth: e30="

Observe this User-Agent string in the registry log:

<IP_ADDRESS> - - [14/Mar/2016:23:09:09 +0000] "GET /v2/hello-world/manifests/latest HTTP/1.1" 200 708 "" "Goelzer/99.9 (Goelzer; en-us) upstream-ua/end; docker/1.11.0-dev go/go1.6 git-commit/xxxxxx-unsupported kernel/4.2.0-30-generic os/linux arch/amd64"

Refs:
[1] RFC 7231: http://tools.ietf.org/html/rfc7231#section-5.5.3
[2] https://github.com/docker/distribution/blob/master/docs/deploying.md

Looks like there's some golint errors;
23:20:32 ---> Making bundle: validate-lint (in bundles/1.11.0-dev/validate-lint)
23:20:32 Errors from golint:
23:20:32 api/server/middleware/user_agent.go:13:6: exported type UAStringKey should have comment or be unexported
23:20:32 api/server/middleware/user_agent.go:21:24: should drop = 0 from declaration of var k; it is the zero value
23:20:32 distribution/registry.go:42:33: should drop = 0 from declaration of var k; it is the zero value
23:20:32
23:20:32 Please fix the above errors. You can test via "golint" and commit the result.

Also, can you update the title of the PR, and update the commit message to be slightly more descriptive?

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "issue20001" git@github.com:mgoelzer/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "issue20001" git@github.com:mgoelzer/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "issue20001" git@github.com:mgoelzer/docker.git somewhere
$ cd somewhere
$ git rebase -i HEAD~2
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "issue20001" git@github.com:mgoelzer/docker.git somewhere
$ cd somewhere
$ git rebase -i HEAD~2
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "issue20001" git@github.com:mgoelzer/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "issue20001" git@github.com:mgoelzer/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Hi @thaJeztah, I fixed those initial hiccups with the commit message, title, golint issues, failing CI, etc.  I have also now made some additional code changes in response to @aaronlehmann 's feedback.
Once I get this PR passing CI again, I will ping you and some others for re-review.  Probably later today.

Right now windowsTP4 test is failing like this:
15:36:42 ---> Making bundle: validate-dco (in bundles/1.11.0-dev/validate-dco)
15:36:43 fatal: unable to access 'https://github.com/docker/docker.git/': Couldn't resolve host 'github.com'
15:36:56 + ec=128
15:36:56 + set +x

"Couldn't resolve host" sounds like a network issue on azure, so I'm going to wait a couple hours and try to push again.

@mgoelzer WindowsTP4 is probably due to bugs in TP4. Those servers need a reboot once in a while

Thanks for making the changes I requested. This is looking good. I added a few more minor comments.
 My main concern is that having a ...string argument makes it look like the function can accept multiple arguments, but in this case, it only looks at the first one. It would help if the function godoc comment clearly stated that only the first argument was used, and perhaps also if the function panicked if passed multiple arguments.
I think it would be worth adding a test for this PR. It shouldn't be too hard to write one using the newTestRegistry function, which creates a mock registry HTTP server. Have a look at integration-cli/docker_cli_v2_only_test.go, which uses this to check that the daemon doesn't hit /v1/ URLs when the v1 protocol is disabled. Instead of looking at the URLs, you would check the request headers for an appropriate User-Agent string.

Thanks, @aaronlehmann .  I'm making all the changes you suggest now, including eliminating the optional argument stuff.  Your feedback has been really helpful here.

@aaronlehmann Do you think I should put my test in integration-cli/docker_cli_v2_only_test.go or create a new test file in that directory?

@mgoelzer: Please create a new file. You can either call it docker_cli_registry_test.go to keep it generic for future tests related to the registry protocol but not push/pull specifically, or docker_cli_registry_user_agent_test.go.

Still working -- need to write test, need to get CI passing again...

Looks like we would not be able to merge this PR because of merge conflicts. Please rebase, fix conflicts, and force push to your branch.

@calavera @icecrime I have this passing CI and updated for all the changes suggested by maintainers here.  The one open issue is your (Dave's) comment on api/server/middleware/user_agent.go.
I actually think the custom type thing for net/context is good and avoids collisions (see the blog post I mentioned for details).  However, I am guessing that what you are concerned about is the fact that I'm introducing a new dependency on dockerversion in api/server/middleware/user_agent.go.  If so, I agree that that is bad.
I feel ambivalent.  I do agree that your method simplifies the dependency graph by removing one edge, but it is also less collision-proof that what I'm currently doing.  Not sure how to evaluate that tradeoff.  What do you think?

@thaJeztah This PR passed all the CI tests except windowsTP4.  But I think this is a case of a flaky test.  The failing test is below.  I have studied this test and I just don't see any way it could be related to my code changes.  (The same test passes when run on my local Ubuntu machine.  And it's not a Windows-specific test.)
02:36:52 ----------------------------------------------------------------------
02:36:52 FAIL: docker_cli_run_test.go:4223: DockerSuite.TestRunAttachFailedNoLeak
02:36:52
02:36:52 docker_cli_run_test.go:4230:
02:36:52     c.Assert(err, checker.NotNil)
02:36:52 ... value = nil
02:36:52
02:37:18
02:37:18 ----------------------------------------------------------------------

(full output here)
I'm going to push the same commit again now and see if I can get it to pass by chance...

WindowsTP4 failing because ``Couldn't resolve host 'github.com'`.  Kicking off another build...
13:51:54 INFO: Running validation tests...
13:51:54 + docker run --rm docker sh -c 'cd /c/go/src/github.com/docker/docker; \
13:51:54  	 sleep 5; \
13:51:54 	 hack/make.sh validate-dco validate-gofmt validate-pkg'
13:52:11
13:52:13 ---> Making bundle: validate-dco (in bundles/1.11.0-dev/validate-dco)
13:52:13 fatal: unable to access 'https://github.com/docker/docker.git/': Couldn't resolve host 'github.com'
13:52:26 + ec=128
13:52:26 + set +x
13:52:26
13:52:26 -------------------------
13:52:26 ERROR: Validation failed.
13:52:26 -------------------------

(full output here)

Closing in favor of #21306.


sensor read with mraa

I want to make an opc-ua server on galileo gen 2, with a yocto built version of linux.
I've made, with some help from git, a complete version of that kind of server for raspberry pi 3, but i have some problems with porting it to galileo. I'm using for the server dht11 and ds18b20 sensors, but i can't manage to make it work on galileo.
Here is my code:
var opcua = require("node-opcua");
var w1bus = require("node-w1bus");
var bus = w1bus.create();
var rpiDhtSensor = require("rpi-dht-sensor");
function read_sensor(sensor,callback) {
bus.getValueFrom(sensor)
.then(function(res){  callback(null,res.result); })
.catch(callback);
};
function start_server() {
    var server = new opcua.OPCUAServer({
            port: 1234
    });

    server.buildInfo.productName ="Rapsberry OPCUA Server"

    function post_initialize() {

            var addressSpace = server.engine.addressSpace;
            var devices = addressSpace.addFolder("ObjectsFolder",{ browseName: "Devices"});
            var temperatureSensor = "28-041460c2a3ff";
            install_sensor(temperatureSensor);
    };

    var tempValue = -20.0;
    var tempValue1 = -20.0;
    var humValue = -20.0;

    function install_sensor(sensor) {
            var addressSpace = server.engine.addressSpace;
            var sensorName = "ds18b20";
            var sensorName1 = "dht11";
            var devices = addressSpace.rootFolder.objects.devices;
            var sensor  = addressSpace.addObject({
                    organizedBy: devices,
                    browseName:  sensorName
            });

            var dht = new rpiDhtSensor.DHT11(27);

            setInterval(function() {
                    var temperatureSensor = "28-041460c2a3ff";
                    read_sensor(temperatureSensor,function(err,result) {
                            tempValue = result.value;
                    });
                    var readout = dht.read();
                    tempValue1 = readout.temperature.toFixed(2);
                    humValue = readout.humidity.toFixed(2);
            },3000);

            addressSpace.addVariable({
                    propertyOf: sensor,
                    browseName: "temperature1",
                    dataType:   "Double",
                    value: {
                            get: function () {
                                    return new opcua.Variant({dataType: opcua.DataType.Double, value: tempValue });
                            }
                    }
            });

            addressSpace.addVariable({
                    propertyOf: sensor,
                    browseName: "temperature2",
                    dataType:   "Double",
                    value: {
                            get: function () {
                                    return new opcua.Variant({dataType: opcua.DataType.Double, value: tempValue1 });
                            }
                    }
            });

            addressSpace.addVariable({
                    propertyOf: sensor,
                    browseName: "humidity",
                    dataType: "Double",
                    value: {
                            get: function() {
                                    return new opcua.Variant({dataType: opcua.DataType.Double, value: humValue });
                            }
                    }
            });
    };

    server.initialize(post_initialize);

    server.start(function() {
            console.log("\n\nTemperature - OPCUAServer");
            var endpointUrl = server.endpoints[0].endpointDescriptions()[0].endpointUrl;
            console.log("Server endpoint url is ", endpointUrl );
            console.log("\n");
    });

}
start_server();
I want to read info from one of these sensors, or others, if easier,with mraa, in order to use those info for my opcua server..
How can I read the temperature from sensors?? Pleasee, help..

What galileo image are you using? It's pretty unsupported at this stage so I can't recommend it. Generally we do not support the use of the DHT11 sensor as it's essentially bit banged and very hard to use reliably.

Yocto 3.8.7..
I don't need to do that just with dht11 sensor.. ds18b20 or any other example would be just fine..

I think you mean linux kernel 3.8.7 - FYI that's really really old.
The DS18B20 is a 1-wire sensor which we do support via uart emulation. But again that's not the kind of sensor we'd recommend. If you want cheap use an analog sensor, if you want something good there are a number of good i2c & spi sensors available readily.
For the DS18B20 (or any other sensor where you want a premade lib) use UPM, see the documentation there: http://upm.mraa.io/sensorDetail.html?name=DS18B20

@cristina21r, I see you've opened a UPM issue for further discussion about ds18b20 communication, so I'll close this one.


Correction for raw affiliation CIRAD, UPR BioWooEB, 34398 Montpellier, France

Correction needed for raw affiliation CIRAD, UPR BioWooEB, 34398 Montpellier, France
raw_affiliation_name: CIRAD, UPR BioWooEB, 34398 Montpellier, France
new_rors: 05kpkpg04;054ktxw16
previous_rors: 05kpkpg04
works_examples: W4388807448
contact: c5c9bdba99a6738279e7f2288e508a68:c8dc411028ceadf54ec0f256a516 @ cirad.fr

This issue was accepted and ingested by the OpenAlex team on 2024-12-19. The new affiliations should be visible within the next 7 days.


Scaling out

Hi @siddontang, I have a question about scaling out of xcodis.
Currently, xcodis relies on the database separation.
So, scaling out is limited to the number of it.
Do you have any plan to support tons of databases?

Well, ledisdb uses 1 byte for db index, so now it may only support maximum 256 dbs (my fault). Supporting larger (using 2 byte for db index) must upgrade the backend data store,  it is dangerous to do this.
btw, I think 256 dbs may be OK too.

@kouhate , now xcodis and ledisdb support configuring databases, (slot_num in xcodis config, databases in ledisdb config), you can decide how many databases you want to use.
Because of my old fault, ledisdb only supports maximum 256, and I don't want to change this now to prevent massive backend data upgrade.
But I still think 256 is ok, or even 64 too. If you use 64 machines (one runs a single ledisdb server for a database), I think the whole data storage is very very big.

@siddontang , thank you!
When I set databases = 256, I got this error.

Type mismatch for 'config.Config.databases': Value '256' is out of range for uint8.

So your config message of Maximum databases should be 255 not 256.
After changing databases = 255, I couldn't switch to index 255.
It's ok because you mentioned in config dbindex must be in [0, databases - 1].

localhost:6380> select 255
(error) ERR invalid db index 255
localhost:6380> select 254
OK

This is a trivial thing, but ledisdb support [0, 254], not [0, 255].
Btw, I think 256 dbs are not yet enough because distributed slot is still limited to 256.
It is true the whole storage become bigger, but the chance which some busy keys sit on the same slot may at least 1/256.
Redis cluster(not stable) support 16384, so probably those chances are 1/16384.
What I worry about is not the storage size but the equalization of the distribution feature.

Hi @kouhate, sorry the bug for 256 config.
the chance for busy keys on the same slot may be big, I think we can customize the routing rule, e.g, detect the busy key and set their own routing rule, not just mode 256.
btw, the default codis slot num is 1024, so the change is 1/1024, a little big too.

Hi @siddontang, sorry for my long silence.
Custom routing for busy key looks good, but this may bring some difficulty for maintenance in the future.
Is it possible to automate the busy key detection and distribute them equally?
If we always have to detect and distribute manually, the bigger keymap may get out of hand.
I think the same is true for big slot. If some keys are not busy but have too big values, the probability of which those keys sit on the same slot is also 1/256 at least.
As we know, custom routing needs a routing table, and always have to check whether the key is in it every time we send a command.
There is few matter while the table is small, but the bigger will be harmful, I think.
Where will you intend to put the table?
Only on zookeeper may decrease performance, but caching on xcodis will be also problematic.
Perhaps, without direct checking or every sec snapshot, xcodis have to keep the cache of not existing keys in the table.

Hi @kouhate, there is no perfect solution, so we have to weigh the cost of different ways and find a acceptable one.  A NoSQL can not fit all needs in the world. :-)
I don't want people to store huge big data in one key, migrating this key is a horrible thing and I think the architect is responsible for  designing a proper data structure or data usage scenario.
Customizing busy key routing rule may not be a good way, I still want to use a big slot instead, of course, 256 is too small, codis uses 1024, and I heard that elasticsearch also uses 1024 (maybe wrong), but redis uses 16384 which I think too big, :smile:
At the same time, even some busy keys mapped into same slot, they may not kill the performance badly, the backend rocksdb has a awesome performance, and we can read data from slaves if we don't consider data consistency too much.  Even we can use cache to solve it.
If we want to use a big routing table, saving it into zookeeper is not a wise decision, but we can save it in a LedisDB which only serves xcodis, and use zookeeper to coordinate them.


Add new Entries

Hello, I would like to take this up

Hey, it says that this project isn't participating in Hacktoberfest. Is it a glitch?

Hey, it says that this project isn't participating in Hacktoberfest. Is it a glitch?

oh got it. It got refreshed


raise meaningful error when no servers found for cluster check

@keymone  looks like this will conflict with #66 - do you want to roll this in together?


Makes fetch notes parallel and eliminates unnecessary retry

Since we should retry 404s automatically based on our strategy, we can eliminate the infinite loop here.
In the future once polling is replaced by notifications this becomes further irrelevant.
Fixes #1479

Looks good. I think we should also apply a timeout to the fetch_notes call. I've observed this hanging forever.

Looks good. I think we should also apply a timeout to the fetch_notes call. I've observed this hanging forever.

This should fix that because it removes the 404 retry from calls to fetch_output_outcome which is where it's retrying forever.

This should fix that because it removes the 404 retry from calls to fetch_output_outcome which is where it's retrying forever.

We generally should wrap every networking operation in some form of a timeout at some point. Any network connection can hang indefinitely in certain (uncommon, but still) circumstances.
But that's more of a technicality that should not block this improvement.

We generally should wrap every networking operation in some form of a timeout at some point. Any network connection can hang indefinitely in certain (uncommon, but still) circumstances.

Hopefully moving from polling to event subscriptions will avoid a lot of the issues we get related to retries.


switch to new domain

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.
e add any other relevant info below:

@conda-forge-admin, please rerender

was from organizational account


FATAL empty src template

confd v0.16
log:
2021-12-09T09:59:55+08:00 confd[618239]: INFO Backend set to etcd
2021-12-09T09:59:55+08:00 confd[618239]: INFO Starting confd
2021-12-09T09:59:55+08:00 confd[618239]: INFO Backend source(s) set to .....something.....
2021-12-09T09:59:55+08:00 confd[618239]: DEBUG Loading template resources from confdir /data/confd
2021-12-09T09:59:55+08:00 confd[618239]: DEBUG Found template: /data/confd/conf.d/confd.toml
2021-12-09T09:59:55+08:00 confd[618239]: DEBUG Loading template resource from /data/confd/conf.d/confd.toml
2021-12-09T09:59:55+08:00 confd[618239]: DEBUG Found template: /data/confd/conf.d/nginx.toml
2021-12-09T09:59:55+08:00 confd[618239]: DEBUG Loading template resource from /data/confd/conf.d/nginx.toml
2021-12-09T09:59:55+08:00 confd[618239]: FATAL empty src template
nginx.toml content
[template]
src = "c.tmpl"
...
...
summary
I saw this error being thrown on the  confd/resource/template/resource.go   106 ，Why this content cannot be decode?

I used the same struct to write a small demo, it is possible to read the value of src.


Fix some typos in doc strings

Found by scrolling through the files in PyCharm and looking at
highlighted words.
Pull Request Check List
This is just a reminder about the most common mistakes.  Please make sure that you tick all appropriate boxes.  But please read our contribution guide at least once, it will save you unnecessary review cycles!

[N/A] Added tests for changed code.
[N/A] New features have been added to our Hypothesis testing strategy.
[N/A] Updated documentation for changed code.
[N/A] Documentation in .rst files is written using semantic newlines.
[N/A] Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives.
[N/A] Changes (and possible deprecations) are documented in CHANGELOG.rst.

If you have any questions to any of the points above, just submit and ask!  This checklist is here to help you, not to deter you from contributing!

Thanks!


Fix podman-remote images

Looks like we went too far with the linters.
Signed-off-by: Daniel J Walsh<EMAIL_ADDRESS>

@edsantiago PTAL
I think this fixes the podman-remote images.

Fixes: https://github.com/containers/libpod/issues/6623

Fix confirmed:
$ ./bin/podman-remote --url $MYSOCK images
REPOSITORY                TAG     IMAGE ID      CREATED      SIZE
docker.io/library/alpine  latest  a24bb4013296  2 weeks ago  5.85 MB

Gating test failure looks exactly like the one @mheon encountered in his PR

lgtm

/lgtm
/hold

Test failures: one is a known flake ("streaming something") that has been causing huge problems the past week. Two others are very weird (signal handling), but cirrus-flake-xref knows about them so I'm going to restart them all and hope they go away.

CI is passing after many, many test restarts! (the "podman logs" problem being discussed in #6614).
/hold cancel

cirrus-flake-analyze:
test ubuntu-19 ubuntu-19
2020-06-16T11:30:48  system_test

47 podman kill - test signal handling in containers

2020-06-16T11:35:47  integration_test

Podman trust [It] podman image trust show
Podman logs [It] streaming output
[same]
[same]

2020-06-16T12:32:30  integration_test

Podman logs [It] streaming output
[same]
[same]
Podman trust [It] podman image trust show
Podman rmi [It] podman rmi image that is a parent of another image

test ubuntu-20 ubuntu-20
2020-06-16T11:33:11  system_test

47 podman kill - test signal handling in containers

2020-06-16T12:28:57  system_test

47 podman kill - test signal handling in containers

2020-06-16T13:13:38  system_test

47 podman kill - test signal handling in containers

2020-06-16T14:22:50  system_test

47 podman kill - test signal handling in containers


schedule on a specific date

Currently adding a job like :
schedule.every(5).minutes.do(my_job)

Is there any option to do the specific job on a specific date like:
schedule.on(year/month/date).at(time).do(my_job)

If it is already available can you please share me an example.

I second this.  I can't seem to find a way to do something like this:
0 16 5 * *
“At 16:00 on the 5th of every month.”
Or something similar.
Thanks

I can definitely understand the interest in a feature like this. What I'm worried about is that schedule isn't  a good fit for this kind of scheduling problem.
For example, let's say we want to schedule a job function to run on August 27, 2016 in some kind of web app that uses schedule. What's going to happen if someone deploys a new version of the app around the instant the job's supposed to run? How are we going to keep track of whether the job ran or not when the app was restarted? How are we going to make sure the job is only going to run once, even if the app is restarted on the day the job's supposed to run?
If we wanted to solve these problems we'd have to talk about a persistence mechanism. And I believe that we'd basically end up on a path to re-invent cron 😐
In its current form schedule is a job scheduler well-suited for periodic tasks, preferably ones with relatively small intervals between executions.

Thanks for the quick reply!!!
I thought that schedule only kicks jobs if it "crosses" the scheduled time, and therefore doesn't need any persistent data.
So in the above example, if a job is scheduled to run on Aug 27 at midnight, then, if schedule is stopped at 11:59 and started at 12:01, it wouldn't start the job because it never "crossed" the time the job was supposed to run.
And likewise, if schedule was restarted continuously on Aug 27, then it wouldn't kick it off either, since it wasn't running "across" the time.
In other words, it only kicks off jobs if schedule was running a bit before and a bit after the time of scheduled job.
I'm not the author, so I wouldn't know :)  I'm only guessing.
I'm working on a Proof of Concept where I'm converting all our cron jobs to use schedule, and here it is, in all its glory.  This code will indeed be restarted every time I do a deploy, so I'm hoping that the tasks that run at "specific days" or "month" intervals don't get executed ever time I restart schedule:
(of course, I replaced "job" with a thread call, and i have global dictionary which ensures jobs that take 5 minutes to run but are on a 1 minute cron do not run in parallel)
If specific day, or month intervals is not a good match for schedule, let me know I'll look for an alternative, but I love the simplicity of this package!

  # Load up the crons. Use http://crontab.guru/ to convert existing crons:
  schedule.every(1).minutes.do(job)
  schedule.every(1).minutes.do(job)
  schedule.every(1).minutes.do(job)
  schedule.every().day.at("11:00").do(job)
  schedule.every(1).minutes.do(job)
  schedule.every().hour.at(":59").do(job)

  for i in range(6):
      schedule.every().friday.at("{}:7".format(i * 4)).do(job)
      schedule.every().saturday.at("{}:7".format(i * 4)).do(job)
      schedule.every().sunday.at("{}:7".format(i * 4)).do(job)

  schedule.every(3).hour.at(":7").do(job)
  schedule.every(15).minutes.do(job)
  schedule.every().day.at("15:00").do(job)

  # I'll create a ticket on my backlog, since I can't figure out how to do this with 'schedule':
  #schedule.every().month.at("16:00").do(job)
  #0 16 5 * *

  for i in range(24):
      schedule.every().monday.at("{}:17".format(i)).do(job)
      schedule.every().tuesday.at("{}:17".format(i)).do(job)
      schedule.every().wednesday.at("{}:17".format(i)).do(job)

  schedule.every().day.at("13:33").do(job)
  schedule.every(4).hour.at(":40").do(job)
  schedule.every(5).minutes.do(job)
  schedule.every(15).minutes.do(job)
  schedule.every().hour.at(":33")(job)
  schedule.every().hour.at(":40").do(job)
  schedule.every().day.at("00:00").do(job)
  schedule.every().day.at("9:05").do(job)
  schedule.every().day.at("7:08").do(job)
  schedule.every().day.at("9:13").do(job)
  schedule.every().day.at("9:00").do(job)
  schedule.every().hour.at("0:47").do(job)
  schedule.every(2).hour.at(":20").do(job)
  schedule.every().day.at("7:25").do(job)
  schedule.every().saturday.at("8:35").do(job)
  schedule.every().day.at("4:00").do(job)
  schedule.every().day.at("5:00").do(job)
  schedule.every().day.at("9:00").do(job)
  schedule.every(15).minutes.do(job)
  schedule.every().day.at("13:15").do(job)

  # This one is set to run at odd times on the hour:
  schedule.every().hour.at(":16").do(job)
  schedule.every().hour.at(":46").do(job)

  schedule.every().day.at("8:30").do(job)
  schedule.every().day.at("14:00").do(job)
  schedule.every(15).minutes.do(job)
  schedule.every().day.at("6:30").do(job)
  schedule.every().hour.at(":11").do(job)
  schedule.every().day.at("12:00").do(job)
  schedule.every().day.at("12:05").do(job)
  schedule.every().sunday.at("3:00").do(job)
  schedule.every().friday.at("5:00").do(job)
  schedule.every().day.at("6:30").do(job)
  schedule.every().day.at("6:00").do(job)

it only kicks off jobs if schedule was running a bit before and a bit after the time of scheduled job

This is 100% correct.
If an 1-minute job execution doesn't happen, it's no big deal because the job will run 1 minute after the program starts again. But if a job is set to run each month, and the job is skipped, the job only runs again the next month. The impact of a missed monthly job is often much bigger. So I agree with @dbader that schedule is better suited for short periodic tasks. In this mindset, I don't think scheduling a job on a specific date suits this library.


April eco docs2

3x new KB articles: Flume, Hortonworks Hadoop, Call Me

Added some bonus material:  Bitnami GitLab Blueprint docs


[Volunteer] Add unit test for LocalMetadataController

Volunteer help
Hi community
Volunteer help to add unit test for org.apache.shenyu.web.controller.LocalMetadataController

Please assign to me.

done


Make install text a bit clearer

Improves the install docs text and make it clear, also adds a link to values.yaml.

Using yaml &version thing is not working with helm for some reason... closing it


Https with certificate

Hi,
It's more a question than a bug.
I'm using sailsjs for a project and i'd like to use a extension to sails named : sails-rest. The next version of this extension v0.1.0-waterline-v0.10 will use superagent. It look than superagent don't support https with certificate authentification. I had a discussion with the Zohararad who developed the sailsjs extension, but he asked me to check with you if you can give more information about that issue.
In my project I need to use certificate to authenticate on a API with sailsjs. Previously I developed a simple express project and used request with a certificate. It worked fine. So why this feature is not implemented in superagent, did you expect to integrate a certificate based authentication, if it is developed could you give an example on how to use it?
Regards,

Hello,
same question here.

Would this help? #681

@pornel Thanks, looks promising!
I'll give it a try.

@pornel exactly what I looked for.


[Feature] add selector: method to combine YAML selectors with command line selection

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature
Introduce a new selector method in order to explicitly select a selector on the command line, and be able to combine the result with the full power of CLI selection (union, intersection, exclude, etc.).
This would solve #5009, #10991, and #10596 (and probably others).
But the real power in this would be to turn YAML selectors into predefined selection subsets (rather than being the primary selection). For example, instead of: dbt ls -s something,tag:one something,tag:two something,tag:three, we could define a tag_one_two_three selector, and then simply do dbt ls -s something,selector:tag_one_two_three. This would provide tremendous reusability for cases that do not necessarily deserve new selectors to be created, and would offer the possibility to encapsulate repetitive or complex selection logic in predefined & reusable "selection components".
Composition over inheritance is a very valuable software design pattern/approach, and dbt users would gain from it being applied to selectors. Right now, the command line offers composition but no inheritance, whereas YAML selectors offer inheritance but "no composition". YAML composition is possible, but at the cost of preemptively creating a multitude of selectors for every possible combination, which very quickly become unmanageable. So you could imagine: selector_1_with_abc, selector_1_with_def, selector_1_with_abc_and_def, selector_2_with_abc, and on and on... and before you know it, YAML selectors are no longer DRY.
In fact, a selector method already exists (see also #1628, #4821, and #4827), but only for selector inheritance within YAML selectors. Unfortunately, this method is not available on the command line. 😞
Describe alternatives you've considered
There are none. Right now, we are forced to choose between YAML selectors or CLI selectors, but we cannot combine both.
Who will this benefit?
Dbt users with advanced selection needs. Especially those who want to tweak selectors for custom or ad-hoc use cases.
It would also benefit every DBT user by enabling YAML selectors to be used as subsets on the CLI.
Are you interested in contributing this feature?
Maybe. Not quite sure where to start.
Anything else?
No response

Another useful example: dbt ls -s something --exclude selector:large_models


override dict in propsbase to use camelCase

need to fix these https://github.com/reflex-dev/reflex/blob/63bf1b8817202ef70ba7e024a5e6321f2e220e55/reflex/components/sonner/toast.py#L174-L181
since they're going to be camelCase now


Feature Request: Allow capacity in cumulative constraint in CP solver to be time dependent

Hi,
at the moment the cumulative constraint in the CP solver is defined as follows
for all t:
  sum(demands[i]
    if (start(intervals[t]) <= t < end(intervals[t])) and
    (t is present)) <= capacity

As you can see, the capacity is "static", i.e. not dependent on the time $t$. My request would be to also allow a time dependent capacity. The definition would then be
for all t:
  sum(demands[i]
    if (start(intervals[t]) <= t < end(intervals[t])) and
    (t is present)) <= capacity[t]

with
capacity: The maximum capacity of the cumulative constraint. It must be a
  positive integer value or variable, or list of length t of positive integer values or variables.

Just create fixed intervals with fixed demands to shape the capacity.
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
Le mer. 4 sept. 2019 à 08:49, Jakob Schelbert<EMAIL_ADDRESS>a
écrit :

Hi,
at the moment the cumulative constraint in the CP solver is defined as
follows
for all t:
sum(demands[i]
if (start(intervals[t]) <= t < end(intervals[t])) and
(t is present)) <= capacity
As you can see, the capacity is "static", i.e. not dependent on the time
$t$. My request would be to also allow a time dependent capacity. The
definition would then be
for all t:
sum(demands[i]
if (start(intervals[t]) <= t < end(intervals[t])) and
(t is present)) <= capacity[t]
with
capacity: The maximum capacity of the cumulative constraint. It must be a
positive integer value or variable, or list of length t of positive integer values or variables.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1546?email_source=notifications&email_token=ACUPL3P2ZPOHPIPXVIXGHN3QH5LABA5CNFSM4ITOCXB2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HJFFI6Q,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3JRNQJDLO5XVCSDHDTQH5LABANCNFSM4ITOCXBQ
.

Hi Laurent,
Thank you for the hint. This will will work for fixed capacity.
It will, however, not work if you have a variable capacity for each time bucket. Any suggestions how to model this?

Fixed intervals with variable capacity.
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
Le mer. 4 sept. 2019 à 10:36, Jakob Schelbert<EMAIL_ADDRESS>a
écrit :

Hi Laurent,
Thank you for the hint. This will will work for fixed capacity.
It will, however, not work if you have a variable capacity for each time
bucket. Any suggestions how to model this?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1546?email_source=notifications&email_token=ACUPL3OVYD6OHU3Y2WGVPG3QH5XQ5A5CNFSM4ITOCXB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD52Z5IY#issuecomment-527802019,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3N7LRYQGXJI7MA66YLQH5XQ5ANCNFSM4ITOCXBQ
.

Thanks :)


Rename _canonical in levelized_file

The variable _canonical in the levelized_file class is dangerously close to canonical in the node_file subclass.

[x] Rename _canonical to _canonical_paths

Closed with #463 .


migrate and rollback commands use the env options, which parses the dsn

Use the parsed DSN for the migrate and rollback command #1406

Tests are failing, can you please fix those?

I updated the code to account for a null possibility, but the tests are still failing.  I think this may be a failing of the mock object (which I'm not very familiar with).  A function that should never return null is returning null due to the mocking.

I think the problem it that you needed to update the tests, it is safe to assume the manager is passed.

ping @watermark
Are you still interested in finishing this? Would be awesome.
Otherwise, we have to close.

Closing due to no activity.