crosstool_wrapper_driver_is_not_gcc failed: error executing command

Please go to Stack Overflow for help and support:
https://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.
It shouldn't be a TensorBoard issue. Those go here.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 1.11.0rc1
Python version: 3.6.5
Bazel version (if compiling from source): 0.17.1
GCC/Compiler version (if compiling from source): c++ (Ubuntu 7.3.0-16ubuntu3) 7.3.0
CUDA/cuDNN version: 9.2 / 7.2.1
GPU model and memory: GeForce 940MX
Exact command to reproduce: bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=cuda --verbose_failures //tensorflow/tools/pip_package:build_pip_package

You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
You can obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.
When the VERBS support is enabled the tensorflow build fails with the following error message:
~/tensorflow/tensorflow/contrib/verbs/BUILD:90:1: C++ compilation of rule '//tensorflow/contrib/verbs:rdma_rendezvous_mgr' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.
ERROR: ~/Documents/dev/git/tensorflow/tensorflow/contrib/verbs/BUILD:90:1: C++ compilation of rule '//tensorflow/contrib/verbs:rdma_rendezvous_mgr' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command
  (cd ~/.cache/bazel/_bazel_blablabla/cf67b2b2e967476eb2b1ee98e33ab5bd/execroot/org_tensorflow && \
  exec env - \
    CUDA_TOOLKIT_PATH=/usr/local/cuda \
    CUDNN_INSTALL_PATH=/usr/lib/x86_64-linux-gnu \
    GCC_HOST_COMPILER_PATH=/usr/bin/gcc \
    NCCL_INSTALL_PATH=/usr/local/nccl_2.2.13-1+cuda9.2_x86_64 \
    PATH=~/bin:/usr/local/sbin:/usr/local/lib:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/linuxbrew/.linuxbrew/opt/coreutils/libexec/gnubin:/usr/local/cuda/bin:/usr/local/share/apache/hadoop/sbin:/usr/local/share/apache/hadoop/bin:/usr/local/share/apache/spark/sbin:/usr/local/share/apache/spark/bin:/usr/games:/usr/local/games:~/bin:/snap/bin \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python3 \
    PYTHON_LIB_PATH=/usr/lib/python3/dist-packages \
    TF_CUDA_CLANG=0 \
    TF_CUDA_COMPUTE_CAPABILITIES=5.0 \
    TF_CUDA_VERSION=9.2 \
    TF_CUDNN_VERSION=7 \
    TF_NCCL_VERSION=2 \
    TF_NEED_CUDA=1 \
    TF_NEED_OPENCL_SYCL=0 \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/k8-opt/bin/tensorflow/contrib/verbs/_objs/rdma_rendezvous_mgr/rdma_rendezvous_mgr.pic.d '-frandom-seed=bazel-out/k8-opt/bin/tensorflow/contrib/verbs/_objs/rdma_rendezvous_mgr/rdma_rendezvous_mgr.pic.o' '-DGRPC_ARES=0' '-DPB_FIELD_16BIT=1' -D__CLANG_SUPPORT_DYN_ANNOTATION__ -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' -DTENSORFLOW_USE_JEMALLOC -DTF_USE_SNAPPY -DTENSORFLOW_USE_VERBS -DTENSORFLOW_USE_GDR -DCURL_STATICLIB -DPLATFORM_LINUX -DENABLE_CURL_CLIENT -DENABLE_NO_ENCRYPTION -iquote . -iquote bazel-out/k8-opt/genfiles -iquote bazel-out/k8-opt/bin -iquote external/protobuf_archive -iquote bazel-out/k8-opt/genfiles/external/protobuf_archive -iquote bazel-out/k8-opt/bin/external/protobuf_archive -iquote external/bazel_tools -iquote bazel-out/k8-opt/genfiles/external/bazel_tools -iquote bazel-out/k8-opt/bin/external/bazel_tools -iquote external/grpc -iquote bazel-out/k8-opt/genfiles/external/grpc -iquote bazel-out/k8-opt/bin/external/grpc -iquote external/zlib_archive -iquote bazel-out/k8-opt/genfiles/external/zlib_archive -iquote bazel-out/k8-opt/bin/external/zlib_archive -iquote external/boringssl -iquote bazel-out/k8-opt/genfiles/external/boringssl -iquote bazel-out/k8-opt/bin/external/boringssl -iquote external/com_google_absl -iquote bazel-out/k8-opt/genfiles/external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/nsync -iquote bazel-out/k8-opt/genfiles/external/nsync -iquote bazel-out/k8-opt/bin/external/nsync -iquote external/jemalloc -iquote bazel-out/k8-opt/genfiles/external/jemalloc -iquote bazel-out/k8-opt/bin/external/jemalloc -iquote external/eigen_archive -iquote bazel-out/k8-opt/genfiles/external/eigen_archive -iquote bazel-out/k8-opt/bin/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/k8-opt/genfiles/external/local_config_sycl -iquote bazel-out/k8-opt/bin/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/k8-opt/genfiles/external/gif_archive -iquote bazel-out/k8-opt/bin/external/gif_archive -iquote external/jpeg -iquote bazel-out/k8-opt/genfiles/external/jpeg -iquote bazel-out/k8-opt/bin/external/jpeg -iquote external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/genfiles/external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/bin/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/k8-opt/genfiles/external/farmhash_archive -iquote bazel-out/k8-opt/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/k8-opt/genfiles/external/fft2d -iquote bazel-out/k8-opt/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/k8-opt/genfiles/external/highwayhash -iquote bazel-out/k8-opt/bin/external/highwayhash -iquote external/local_config_cuda -iquote bazel-out/k8-opt/genfiles/external/local_config_cuda -iquote bazel-out/k8-opt/bin/external/local_config_cuda -iquote external/double_conversion -iquote bazel-out/k8-opt/genfiles/external/double_conversion -iquote bazel-out/k8-opt/bin/external/double_conversion -iquote external/curl -iquote bazel-out/k8-opt/genfiles/external/curl -iquote bazel-out/k8-opt/bin/external/curl -iquote external/jsoncpp_git -iquote bazel-out/k8-opt/genfiles/external/jsoncpp_git -iquote bazel-out/k8-opt/bin/external/jsoncpp_git -iquote external/aws -iquote bazel-out/k8-opt/genfiles/external/aws -iquote bazel-out/k8-opt/bin/external/aws -isystem external/protobuf_archive/src -isystem bazel-out/k8-opt/genfiles/external/protobuf_archive/src -isystem bazel-out/k8-opt/bin/external/protobuf_archive/src -isystem external/grpc/include -isystem bazel-out/k8-opt/genfiles/external/grpc/include -isystem bazel-out/k8-opt/bin/external/grpc/include -isystem external/zlib_archive -isystem bazel-out/k8-opt/genfiles/external/zlib_archive -isystem bazel-out/k8-opt/bin/external/zlib_archive -isystem external/grpc/third_party/address_sorting/include -isystem bazel-out/k8-opt/genfiles/external/grpc/third_party/address_sorting/include -isystem bazel-out/k8-opt/bin/external/grpc/third_party/address_sorting/include -isystem external/boringssl/src/include -isystem bazel-out/k8-opt/genfiles/external/boringssl/src/include -isystem bazel-out/k8-opt/bin/external/boringssl/src/include -isystem external/nsync/public -isystem bazel-out/k8-opt/genfiles/external/nsync/public -isystem bazel-out/k8-opt/bin/external/nsync/public -isystem external/jemalloc/include -isystem bazel-out/k8-opt/genfiles/external/jemalloc/include -isystem bazel-out/k8-opt/bin/external/jemalloc/include -isystem external/eigen_archive -isystem bazel-out/k8-opt/genfiles/external/eigen_archive -isystem bazel-out/k8-opt/bin/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/k8-opt/genfiles/external/gif_archive/lib -isystem bazel-out/k8-opt/bin/external/gif_archive/lib -isystem external/farmhash_archive/src -isystem bazel-out/k8-opt/genfiles/external/farmhash_archive/src -isystem bazel-out/k8-opt/bin/external/farmhash_archive/src -isystem external/com_google_absl -isystem bazel-out/k8-opt/genfiles/external/com_google_absl -isystem bazel-out/k8-opt/bin/external/com_google_absl -isystem external/local_config_cuda/cuda -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda -isystem external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda/cuda/include -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include -isystem external/local_config_cuda/cuda/cuda/include/crt -isystem bazel-out/k8-opt/genfiles/external/local_config_cuda/cuda/cuda/include/crt -isystem bazel-out/k8-opt/bin/external/local_config_cuda/cuda/cuda/include/crt -isystem external/double_conversion -isystem bazel-out/k8-opt/genfiles/external/double_conversion -isystem bazel-out/k8-opt/bin/external/double_conversion -isystem external/curl/include -isystem bazel-out/k8-opt/genfiles/external/curl/include -isystem bazel-out/k8-opt/bin/external/curl/include -isystem external/jsoncpp_git/include -isystem bazel-out/k8-opt/genfiles/external/jsoncpp_git/include -isystem bazel-out/k8-opt/bin/external/jsoncpp_git/include -isystem external/aws/aws-cpp-sdk-core/include -isystem bazel-out/k8-opt/genfiles/external/aws/aws-cpp-sdk-core/include -isystem bazel-out/k8-opt/bin/external/aws/aws-cpp-sdk-core/include -isystem external/aws/aws-cpp-sdk-kinesis/include -isystem bazel-out/k8-opt/genfiles/external/aws/aws-cpp-sdk-kinesis/include -isystem bazel-out/k8-opt/bin/external/aws/aws-cpp-sdk-kinesis/include -isystem external/aws/aws-cpp-sdk-s3/include -isystem bazel-out/k8-opt/genfiles/external/aws/aws-cpp-sdk-s3/include -isystem bazel-out/k8-opt/bin/external/aws/aws-cpp-sdk-s3/include '-std=c++11' -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIC -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections -mavx -mavx2 -mfma '-mfpmath=both' -msse4.2 -c tensorflow/contrib/verbs/rdma_rendezvous_mgr.cc -o bazel-out/k8-opt/bin/tensorflow/contrib/verbs/_objs/rdma_rendezvous_mgr/rdma_rendezvous_mgr.pic.o)
In file included from ./tensorflow/core/framework/common_shape_fns.h:22:0,
                 from ./tensorflow/core/framework/resource_mgr.h:24,
                 from ./tensorflow/core/common_runtime/device.h:43,
                 from ./tensorflow/core/common_runtime/device_mgr.h:24,
                 from ./tensorflow/core/distributed_runtime/worker_session.h:21,
                 from ./tensorflow/core/distributed_runtime/base_rendezvous_mgr.h:24,
                 from ./tensorflow/contrib/verbs/rdma_rendezvous_mgr.h:22,
                 from tensorflow/contrib/verbs/rdma_rendezvous_mgr.cc:18:
./tensorflow/core/util/tensor_format.h: In function 'tensorflow::TensorShape tensorflow::ShapeFromFormat(tensorflow::TensorFormat, tensorflow::int64, tensorflow::gtl::ArraySlice<long long int>, tensorflow::int64)':
./tensorflow/core/util/tensor_format.h:501:45: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (format == FORMAT_NHWC_VECT_W && dim == spatial.size() - 1) {
                                         ~~~~^~~~~~~~~~~~~~~~~~~~~
tensorflow/contrib/verbs/rdma_rendezvous_mgr.cc: In member function 'virtual void tensorflow::RdmaRemoteRendezvous::RecvFromRemoteAsync(const tensorflow::Rendezvous::ParsedKey&, const tensorflow::Rendezvous::Args&, tensorflow::Rendezvous::DoneCallback)':
tensorflow/contrib/verbs/rdma_rendezvous_mgr.cc:66:41: error: 'using StringPiece = class absl::string_view {aka class absl::string_view}' has no member named 'ToString'
   string key(std::move(parsed.FullKey().ToString()));
                                         ^~~~~~~~
In file included from ./tensorflow/core/platform/default/logging.h:24:0,
                 from ./tensorflow/core/platform/logging.h:25,
                 from ./tensorflow/core/lib/core/refcount.h:22,
                 from ./tensorflow/core/platform/tensor_coding.h:21,
                 from ./tensorflow/core/framework/resource_handle.h:19,
                 from ./tensorflow/core/framework/types.h:31,
                 from ./tensorflow/contrib/verbs/verbs_util.h:21,
                 from ./tensorflow/contrib/verbs/rdma.h:30,
                 from ./tensorflow/contrib/verbs/rdma_mgr.h:24,
                 from ./tensorflow/contrib/verbs/rdma_rendezvous_mgr.h:21,
                 from tensorflow/contrib/verbs/rdma_rendezvous_mgr.cc:18:
./tensorflow/core/util/tensor_format.h: In instantiation of 'T tensorflow::GetTensorDim(tensorflow::gtl::ArraySlice<T>, tensorflow::TensorFormat, char) [with T = long long int; tensorflow::gtl::ArraySlice<T> = absl::Span<const long long int>]':
./tensorflow/core/util/tensor_format.h:452:47:   required from here
./tensorflow/core/util/tensor_format.h:420:29: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   CHECK(index >= 0 && index < dimension_attributes.size())
./tensorflow/core/platform/macros.h:87:47: note: in definition of macro 'TF_PREDICT_FALSE'
 #define TF_PREDICT_FALSE(x) (__builtin_expect(x, 0))
                                               ^
./tensorflow/core/util/tensor_format.h:420:3: note: in expansion of macro 'CHECK'
   CHECK(index >= 0 && index < dimension_attributes.size())
   ^
./tensorflow/core/util/tensor_format.h: In instantiation of 'T tensorflow::GetFilterDim(tensorflow::gtl::ArraySlice<T>, tensorflow::FilterTensorFormat, char) [with T = long long int; tensorflow::gtl::ArraySlice<T> = absl::Span<const long long int>]':
./tensorflow/core/util/tensor_format.h:461:54:   required from here
./tensorflow/core/util/tensor_format.h:435:29: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   CHECK(index >= 0 && index < dimension_attribute.size())
./tensorflow/core/platform/macros.h:87:47: note: in definition of macro 'TF_PREDICT_FALSE'
 #define TF_PREDICT_FALSE(x) (__builtin_expect(x, 0))
                                               ^
./tensorflow/core/util/tensor_format.h:435:3: note: in expansion of macro 'CHECK'
   CHECK(index >= 0 && index < dimension_attribute.size())
   ^
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 7400.912s, Critical Path: 180.41s
INFO: 10108 processes: 10108 local.
FAILED: Build did NOT complete successfully

It is still reproducible in the r1.11rc2 branch with CUDA 10.0, CuDNN 7.3.0, and NCCL 2.3.4.
The cause of the error is still exactly the same.

The PR #22415 has been approved and is pending merge. The issue should be resolved once #22415 is merged internally.

The issue is still reproducible during the tensorflow 1.12 build.
It was NOT reproducible in tensorflow 1.12rc1 and 1,12rc2.
My setup:

Linux Ubuntu 18.04
TensorFlow installed from source
TensorFlow version: 1.12
Python version: 3.6
Bazel version (if compiling from source): 0.19
GCC/Compiler version (if compiling from source): 7.3.0

Same issue here with 1.13

Linux Ubuntu 18.04
TensorFlow 1.13
Python 3.6
Bazel 19.2
GCC 4.8

Turns out this happens because third_party/gpus/cuda_configure.bzl has a hardcoded path to libtools, here:
cuda_defines["%{linker_bin_path_flag}"] = 'flag: "-B/usr/bin"'
If you have a system with a broken libtools (like Redhat/CentOS 7) and have your properly patched libtools somewhere else, Bazel will ignore this since someone didn't think this through when decided to hardcode a binary.

i meet the same problem when i compile tf1.15.x + cuda11 + cudnn8 + ubuntu18.04，what should i do to solve this problem?

i meet the same problem when i compile tf1.15.x + cuda11 + cudnn8 + ubuntu18.04，what should i do to solve this problem?

I gave up and I'm using NVIDIA's container version :-(

Turns out this happens because third_party/gpus/cuda_configure.bzl has a hardcoded path to libtools, here:
cuda_defines["%{linker_bin_path_flag}"] = 'flag: "-B/usr/bin"'
If you have a system with a broken libtools (like Redhat/CentOS 7) and have your properly patched libtools somewhere else, Bazel will ignore this since someone didn't think this through when decided to hardcode a binary.

i meet the same problem when i compile tf1.15.x + cuda11 + cudnn8 + ubuntu18.04，what should i do to solve this problem?

+1 I meet the same problem,1.15+cuda11+cudnn8+ubuntu16.04. either gcc 5.4 nor 7


Any way to limit node containers memory?

Is there any possibility to limit node containers memory?
I can use -m with docker when launching hub, but what about nodes?
Thank you.

Related to https://github.com/aerokube/selenoid/issues/63.

@Exoticalgi currently there's no support for this. But let us discuss a bit how we could better implement this feature. We agree that it's important.

Maybe by setting memory limit for particular image in browsers.json and read it when launching (equivalent to docker -m parameter).

@Exoticalgi yes, we consider adding a new field to browsers.json as one of the main solutions. Will implement somehow during next days.

I will be happy to test this feature.

Fixed in https://github.com/aerokube/selenoid/commit/7901f15f0956bafdeaba194fc4a41e1043679c44.
@Exoticalgi finally we decided to implement this as a global flag -mem and for cpu -cpu applied to each container. E.g.:
$ selenoid -mem 128m

Memory is specified in Docker format: https://docs.docker.com/engine/admin/resource_constraints/
Same stuff with CPUs:
$ selenoid -cpu 1.5

Specified as float value, e.g. 0.2 means 1/5 of CPU. Docs will be updated soon. Please confirm whether this works for you.


Add new tx when a new one is found during client polling

So if somebody is logged in, they will just have to refresh the page / navigate to another page to see their tx show up.
Later this will be done via a socket connection (#88) but it may be a while before we will have this working. Perhaps we can still set up a socket connect to do this with bchd/xecd?... Whatever works.

Should be working fine now.


OpenJDK Update (January 2024 Patch releases)

Is your feature request related to a problem? Please describe
OpenJDK JDK 21.0.2 [1], 17.0.10, 11.0.22, and 8u401 have been released [3]
[1] https://mail.openjdk.org/pipermail/jdk-updates-dev/2024-January/029090.html
[2] https://mail.openjdk.org/pipermail/jdk-updates-dev/2024-January/029089.html
[3] https://blogs.oracle.com/java/post/jdk-2102-17010-11022-and-8u401-have-been-released
Describe the solution you'd like
Update JDKs across all maintenance branches.
Related component
Build
Describe alternatives you've considered
N/A
Additional context
Possible update blockers:

https://bugs.openjdk.org/browse/JDK-8323659

[Triage - attendees 1 2 3]
Thanks for filing, lets make sure to take these update to get CVE fixes from the JDK

Hi, are we on track for this to be released in 2.12 ?

Hi, are we on track for this to be released in 2.12 ?

@kiranprakash154 I am not sure yet, watching [1], few platforms are still not available.
[1] https://adoptium.net/temurin/releases/

I see a new version released yesterday https://github.com/adoptium/temurin21-binaries/releases

@bbarani I know, we need more platforms, please check https://github.com/adoptium/temurin/issues/17

Would we want to pick up 21.0.2+13 if it has https://bugs.openjdk.org/browse/JDK-8323659 ? (The ExecutorScalingQueue class in that code snippet is still in use in OpenSearch.)

We'll double check, we use LinkedTransferQueue

A quick update, the regression in JDK-21.0.2 (https://bugs.openjdk.org/browse/JDK-8323659) seems to be reported by Elasticsearch folks and directly applies (and impacts) to OpenSearch (we've largely the same codebase).

Option 3 is to try to encourage the JDK folks to release <IP_ADDRESS>.
For the first option, the false return here appears to be the offending code:
https://github.com/opensearch-project/OpenSearch/blob/17f20c6bfef3a8a88ce02749f23974044c32f55c/server/src/main/java/org/opensearch/common/util/concurrent/OpenSearchExecutors.java#L428-L450
The offer() method's javadoc states:

Inserts the specified element at the tail of this queue. As the queue is unbounded, this method will never return false.

Option 3 is to try to encourage the JDK folks to release <IP_ADDRESS>.

Thanks @dbwiddis , I think this is off the table right now - I've went through mailing list and apparently this bug alone does not warrant the patch release (at least, this is what I have understood).


Incorrect strategy limit in space center

Need to reduce the strategy limit displayed in the space center.

Duplicate of #66 I think, and please, no. 😟
Increase the actual limit to match the displayed limit. 👍

The Space Center text seems to be correct in Strategia 1.7.3 and KSP 1.7.2.
However, it seems like it's been hardcoded.  Edits to CustomBarnKitConfig.cfg to change the strategies per level are correctly reflected in the Administration Building text but not in the Space Center's.


Use readability on HTML

Add the readability mechanism to get the good text from the HTML dump (or for insertion into mongo).

Looking into this.

Resolved here.
Also see this issue.

Resolved in #13 and #14, this issue can be closed.


Bug(manual_compact):replica lose manual compact finished status after replica migrate

Bug Report
Please answer these questions before submitting your issue. Thanks!

What did you do?
-Set a suitable  time and let  a table finish manual compact
-View the progress of manual compaction,now is 100
-Stop a node
-View the progress of manual compaction again ,now is 79
-Go to the alive nodes and view their logs,some replicas begin manual compact
-View the progress of manual compaction again ,now is 91
-However some replica DO NOT manual compaction this day manual compaction progress stack in 91

So that , in our online envs,some pegasus users can not finish manual compact after replica migrate.

What did you expect to see?
Replicas which finished manual compact can hold the status after replica migrate.

What version of Pegasus are you using?
pegasus2.4

Is the issue still open? I'm interested to solve this issue. Could please tell some more details. Thanks in Advance

@ninsmiracle Each replica of a partition will do manual compaction independently, when one node shutdown, the other replicas on other nodes may not start or in progress manual compaction.
What tools you were using to check the progress, could you leave some details? Thanks!

I used admin-cli to check the manaul compaction progress. And this is the result I use manual-compaction query -a TABLE_NAME commond after manual compact begin or node stop:

And we can use pegasus http interface to check the replicas compact detail of target table.

Is the issue still open? I'm interested to solve this issue. Could please tell some more details. Thanks in Advance

This is my way to recurrent the issus:

Is the issue still open? I'm interested to solve this issue. Could please tell some more details. Thanks in Advance

This is my way to recurrent this bug:

Create a Pegasus table and input some data.
Set the parameters of this Pegasus app by using set_app_envs. To achieve our  target, we should set manual_compact.disabled to false and set manual_compact.periodic.trigger_time to a specific time, such as 10:00.(when I do this job,it's 9:58.So that I can observe the result soon)
Build the admin-cli tool and use it to connect to our table.
Use the manual-compaction query -a command to check the progress. We can see that it has reached 100.
Stop one node of our cluster.
The progress will not be 100 anymore until the next day's 10:00.


add multi-arch images using ko

This is an alternative take to #86.
Fixes #85.

Example multi-arch repo: https://github.com/ahmetb/grpc-health-probe/pkgs/container/grpc-health-probe/9380802?tag=v0.99.10
Build output: https://github.com/ahmetb/grpc-health-probe/runs/4005113125?check_suite_focus=true

advantages:

no Dockerfile
multi-arch images can be built/pushed in one command.
reproducible builds as they are timestamped to 0.

disadvantages:

limited to os/arch combonatios supported on the base
image (gcr.io/distroless/static:nonroot) and linux/386
is currently not published on distroless.
binaries are compiled twice
produced binaries in GH releases vs container images
are not identical (there's visible size diff, needs debugging)
can't use OCI labels such as org.opencontainers.image.source
but they are not critical as GitHub still associates them
to the repo correctly and they probably will be supported in
ko at some point.

PTAL @sxd @jzelinskie @mnencia

cc:@sxd I will go with this patch instead and cut a release.


deopped and some rank postions names are glitched out

Some of my ranks like a builder and the default rank names glitch out as shown below. the owner and admin go away when de opped. in the picture, I am an opped owner and my other owner is a non opped owner. his name his completely random for some reason

Not a web editor issue.
Read here:
https://github.com/lucko/LuckPerms/wiki/FAQ#why-are-prefixessuffixes-not-working
If it still isn't working then join our Discord


Use Case: ros2_control should support tool chaning on a robot

Tool changers are a usual component in an industrial robotic cell. ros2_control should be able to support a change of tools, i.e. hardware interfaces, of a robot.
This should be done without restarting the control node.
This could be done by dynamically loading and unloading the tool components (different sensors and actuators).
To support real-time constrains, one could load all the components on start and use init, start, stop, halt commands to control a tool.

done


refactor: breakdown twomap into multiple modules

The twomap.rs file was 1500+ lines long, which is not ideal for readability. It's now separated into 7 files, using the pub(super) qualifier to keep CMap2 fields private:

cmap2/mod.rs: module declarations, re-exports and common cmap2 items.
cmap2/basic_ops.rs: basic operations such as read/edit darts & beta functions, i-cell computations
cmap2/embed.rs: vertex-related code; this is where generic attribute accesses should be implemented
cmap2/link_and_sew.rs: sew, unsew, link, unlink implementations
cmap2/structure.rs: main structure & constructor definitions
cmap2/tests.rs
cmap2/utils.rs: all cmap2-related implementations gated  behind the utils feature

Also add a common.rs file with crate-level common definitions.
Scope

[x] Code: honeycomb-core

Type of change

[x] Refactor

Necessary follow-up

[x] Needs documentation

Codecov Report
Attention: Patch coverage is 76.43411% with 152 lines in your changes are missing coverage. Please review.

Project coverage is 85.71%. Comparing base (c63d4ca) to head (a5d8c47).

Files
Patch %
Lines

honeycomb-core/src/cmap2/utils.rs
0.00%
89 Missing :warning:

honeycomb-core/src/cmap2/link_and_sew.rs
78.14%
40 Missing :warning:

honeycomb-core/src/cmap2/basic_ops.rs
86.30%
20 Missing :warning:

honeycomb-core/src/cmap2/embed.rs
88.00%
3 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #65      +/-   ##
==========================================
- Coverage   85.78%   85.71%   -0.08%
==========================================
  Files          10       15       +5
  Lines        1935     1939       +4
==========================================
+ Hits         1660     1662       +2
- Misses        275      277       +2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add paymentMethod on account page

Describe the bug
saving a payment method on the account page will crash and save a corrupt payment method witch in turn will make the account model crash
To Reproduce
Steps to reproduce the behavior:

Go to Account

try to add a paymentMethod

this will crash on createAdyenRequestObject cause it has no order at that point (image 1)

this in turn will make the account model crash making the site unusable for the end user

it will crash in the SFRA on (image 2)

Expected behavior
the payment method to bee saved
Screenshots

Hi @Vanesro ,
Thanks for reaching out.
This issue was indeed known and we have built a fix for it, which you can find here.
This fix will be included in the next release.
Kind regards,
Bas Maassen
Adyen


1161 dev gas grid input agent

Updated storage and retrieval scripts of instantaneous gas flow data to use latest time series implementation. Also included respective unit tests and dockerised integration tests for SPARQL queries.

This PR has been left far too long, which is my own fault. I have now merged from main back into this branch, can @markushofmeister give that merge a quick glance to check that it doesn't ruin anything? If it doesn't then I will approve and merge this PR.

As @msff2 took my work over and forward, I suggest that he gives it a final review. I believe he should have the best understanding of what the agent should do

This PR has been left far too long, which is my own fault. I have now merged from main back into this branch, can @markushofmeister give that merge a quick glance to check that it doesn't ruin anything? If it doesn't then I will approve and merge this PR.

As @msff2 took my work over and forward, I suggest that he gives it a final review. I believe he should have the best understanding of what the agent should do

@msff2: Are you taking this forward by any chance?

This PR has been left far too long, which is my own fault. I have now merged from main back into this branch, can @markushofmeister give that merge a quick glance to check that it doesn't ruin anything? If it doesn't then I will approve and merge this PR.

As @msff2 took my work over and forward, I suggest that he gives it a final review. I believe he should have the best understanding of what the agent should do

@msff2: Are you taking this forward by any chance?

@msff2: What is our strategy with this branch at the moment?


Add optional parameters for more customizations

add matrix builder parameter for custom transitions
add close on tap flag instead of using isDraggable flag
add optional animated background
add example to show custom edits above
here is a video of the example
https://user-images.githubusercontent.com/38167511/109388678-412ff900-7911-11eb-95dd-ef7db70db751.mp4

Hi @youssefali424 !
It’s  awesome !
If I can resolve conflicts I will be adding this feature in the next release

the conflicts wasn't there at the time, I will try to merge master to the fork in my free time and resolve them if possible


Close #200: Automatic configuration download: UI improvements

@laurijamsa could you please check the translation into Finnish?


is that mono repository project layout？

Hi! Could you provide more details and context?

If the question is this a code repository that holds many projects? Then the answer is no. This repository is only for greenmask core util.


improve the overall styling

The hyperlinks are not clearly visible so in the white background so you can change the font color and shadow color to make it more pleasing for eyes.

I would like to work on this issue.

@aniruddha-5 yeah sure


Make the backends selectable (and maybe add more)

Would be fine to make user-selectable the storage backend and/or make it user-implementable?
I'd like to experiment with sled but I understand you might not want to add to the main sources.

Hi! It's definitely doable but challenging: outside of basic get/set/delete operations key-value stores differ quite a lot and it's challenging to write something efficient targeting different key-value stores. So, it's sadly not in my roadmap for now as I am still the only one working on Oxigraph.


Client module import is undefined

I am able to import the w3cwebsocket and server modules however client comes in as undefined. Logging the index module only returns { "version:" "1.0.34" }.

Not a bug, really. I'm afraid this is a bug tracker not a support forum.


Annotate action.js headings

As it stands, there are multiple headings entitled action.js. On the official docs page, it offers a anchor to each heading, but this doesn't work because it will just match the id of the first #### action.js heading. This change adds some context, which will—in turn—make each heading unique.

Thanks!


[virt-controller] fix out-of-bound slice index in evacuation controller

virt-controller panic can happen if we play with migration configuration in Kubevirt CR.  Steps for reproduction can be
found in the BZ link below.
What this PR does / why we need it:
In case the user drained a node with evictable VMs, migrations have started according to the configured maximum concurrent migrations per node. While the migrations are active the user decides lower the configured maximum per node. In this case the virt-controller can panic, although there are no free spots, there are no migration candidates either (because all VMs have active migrations). The execution loop continues with negative index.
Fixes #
https://bugzilla.redhat.com/show_bug.cgi?id=2171395
Special notes for the reviewer:
  runtime error: slice bounds out of range [:-9]

  Full Stack Trace
    kubevirt.io/kubevirt/pkg/virt-controller/watch/drain/evacuation.(*EvacuationController).sync(0xc000712300, 0xc000032000, {0xc0001c0000?, 0xa, 0x10}, {0xc0001c0080?, 0xa, 0x10})
    	/home/ibezukh/go/src/kubevirt.io/kubevirt/pkg/virt-controller/watch/drain/evacuation/evacuation.go:417 +0x997
    kubevirt.io/kubevirt/pkg/virt-controller/watch/drain/evacuation.(*EvacuationController).execute(0xc000712300, {0x18f07d9, 0x6})
    	/home/ibezukh/go/src/kubevirt.io/kubevirt/pkg/virt-controller/watch/drain/evacuation/evacuation.go:337 +0x176
    kubevirt.io/kubevirt/pkg/virt-controller/watch/drain/evacuation.(*EvacuationController).Execute(0xc000712300)
    	/home/ibezukh/go/src/kubevirt.io/kubevirt/pkg/virt-controller/watch/drain/evacuation/evacuation.go:298 +0x108
    kubevirt.io/kubevirt/pkg/virt-controller/watch/drain/evacuation_test.glob..func1.7.4()
    	/home/ibezukh/go/src/kubevirt.io/kubevirt/pkg/virt-controller/watch/drain/evacuation/evacuation_test.go:334 +0x4ad

Reproduced with the new unit test from this PR
Release note:
NONE

/hold
for now

@xpivarc FYI. Panic attached and corresponding test introduced.

@xpivarc @iholder101 Hi. I've addressed the comment, can you PTAL?

/lgtm
Thank you @enp0s3!

/unhold
/retest-required

Didn't even know of this new generic pointer function!
Cool!!
/lgtm

/hold
@enp0s3
You need to run make

/hold @enp0s3 You need to run make

For some reason make build-functests didn't reveal it.

/unhold

/retest-required

/cherrypick release-0.59

/cherrypick release-0.58


[WIP] VS for Mac extension to allow File->New Elmish app

This is work in progress, raising this PR to discuss the changes.
This adds a VS for Mac extension that allows the user to create a new Elmish.XamarinForms app from inside VS for Mac. It wraps the dotnet new template inside an extension.

Builds are failing on AppVeyor as the VS for Mac add-in needs to build on a Mac. Any way this can be done?

Heh, nice. I was just mentioning the other day that we needed this :+1:

Builds are failing on AppVeyor as the VS for Mac add-in needs to build on a Mac. Any way this can be done?

@jimbobbennett Travis is using an OSX agent, so you should try and make it work there. We can exclude it from the AppVeyor build.

This is fantastic work
You should be able to make the build of the package in build.fsx dependent on the FAKE value isUnix

Closing since nothing happened for more than a year and there is a lot of conflicts


[FEATURE REQUEST] HTTP2 support would be cool

Could maybe do it myself, but simply dont have time for it in the foreseeable future.

There's 3 of us here, I work full time (and sometimes overtime) as a programmer, study engineering at night, have guitar classes and still need to shave time off here and there so I can have a personal life and take care of NanoHTTPD. As for the other 2, they seem to be M.I.A. for god knows how long.
I think we can wait for HTTP2. It's not like it's everywhere already, or will be any time soon, and it should only be started after I manage to deliver v3.0.0

I'm assigning you to this issue.
If you want to push this forward, you'll first have to define what needs to be done (doesn't need to be very fine-grained) and how much effort would it take to make it fit into current 3.0.0 code (more specifically, what need to be changed and why).
Finding a decent java HTTP2 client would be nice as well.
You don't have to do this analysis in a single sitting, but sharing that information with us will allow you, me, and every other contributor to push this forward more easily, and more specifically me to give you insight on what is being done or even preparing 3.0.0 to accept HTTP2 more easily in the future.
I'll even start right now and tell you we can probably implement the protocol using request interceptors (a new interface I've developed for 3.0.0 that allows multiple protocols to run on the same server and on the same port)

I raise your studies with 2 small kids (soon 3) and running a startup, hobbies etc being only a distant thing I can dream of when I'm middle aged.
Disagree that this is not widely supported yet, all modern browsers have full support.
But oh well, might try to contribute sthing here, as I've been using NanoHTTPD quite heavily for years. Not promising anything tho.

Any news?

This is the kind of thing that's known on the software world as Vaporware. Anyone simply expecting for it to be done will have to be very patient. Anyone willing to get their hands dirty and start pushing this forward will be oh-so-very-welcome.

I understand this situation. Just asking if there any changes. Sorry if I'm rude.

ETA around 2027 if it's up to me.

We'll figure this out. Eventually.


Re-factor credential handling

Move core credential parsing and validating into provider plugin.

Provider class now has props method to expose client properties to users rather than calling Credentials class directly.


[New test] All the required module are specified in the script header

All scripts should be using a #Required if it is relying on external PS modules.
For each file

Collect the #Requires modules
Search the file for all the commands
list the modules they belong to
see if they are apart of the Get-PSResource module list
for each of the modules found, verify they are defines in the #Required list
If it is missing, then fail the test.

Probably needs AST to get all the function calls, and see if the function calls are available on the runner. If its present, get the module name and see if the file has the name of the module in the #Requires section on the top of the file.

Test does not make sense as we don't download and import modules while checking source code. And when evaluating the module, the source code is not considered. Removing this PR and closing the issue.


Support for multiple images per post

Fix for issue https://github.com/madwork/tumblr-sync/issues/1
Now with working progress bar.

This PR seems to be open for like a year. Any change it could be merged?

Working well! Thanks :+1:


Add Help link

Add a Help link. This link could initially just go to sotera.github.io/newman/features.
o	Provide better explanation on the custom algorithm used to determine important email addresses.
o	Explain how topics are generated.

Fixed.


Provide a "read-only" public instance

For starters it should probably have only the "query" interfaces made available, so it would not be able to trigger any action/update database.
Think about how could this be done?
TODOs

[ ] @yarikoptic -- do provide test "public URL" which would scope only to overview and api/v2/url-metadata
[x] @candleindark

[x] make it switchable to enable POST request to report a URL to consider
[x] run non-devel instance
[x] make celery turnable off
[x] replication of DB so we could do stuff on one instance and just expose information read-only to the world

this is done AFAIK -- we have http://registry.datalad.org/


Don't need to give the private containers a public IP

The routing works just fine without public IPs for the containers

Nice, thanks!


2.11.x | Fix range for MarkupSafe to stay below 2.1

MarkupSafe 2.1.0 has a breaking change

Remove soft_unicode, which was previously deprecated. Use soft_str instead. #261

Jinja 2.11.x requires MarkupSafe >= 0.23. This allows it to pick up the latest version of MarkupSafe with the breaking change. This PR fixes the range of MarkupSafe to stay below 2.1.0.

fixes #1587

Checklist:

[ ] Add tests that demonstrate the correct behavior of the change. Tests should fail without the change.
[ ] Add or update relevant docs, in the docs folder and in code.
[ ] Add an entry in CHANGES.rst summarizing the change and linking to the issue.
[ ] Add .. versionchanged:: entries in any relevant code docs.
[x] Run pre-commit hooks and fix any issues.
[x] Run pytest and tox, no tests failed.

Could you clarify the harm in merging this pull request? It seems like an easy one-line change that allows us to avoid breaking existing Flask installations, and it seems like a better use of time than copy-pasting across the internet.
We couldn't make an upgrade to Flask 2.0 on release, due to a few bugs that weren't patched until 2.0.2. Not pinning transitive dependencies is on us, but expecting everyone to be up to date with breaking changes within just a few months seems like an unnecessary ask.

It wouldn't help because it'd also require a new release of the older versions...

Pinning transitive dependencies is how you control when you become up to date with changes.

I fully understand those links. We should recognize that the headache is partially caused by Jinja2 not following their advice.
Let's take an example pinned against Flask 1.1.4. It requires Jinja2, and Jinja2 requires MarkupSafe.
Flask==1.1.4
- click [required: >=5.1,<8.0]
- itsdangerous [required: >=0.24,<2.0]
- Jinja2 [required: >=2.10.1,<3.0]
  - MarkupSafe [required: >=0.23]

Releasing a Jinja2 2.11.4 would result in the following dependency tree:
Flask==1.1.4
- click [required: >=5.1,<8.0]
- itsdangerous [required: >=0.24,<2.0]
- Jinja2 [required: >=2.10.1,<3.0]
  - MarkupSafe [required: >=0.23,<2.1]

So anyone not pinning transitive dependencies on flask would automatically pick up the change, saving a large number of developers some headaches while everyone gets their act together.


[SEMANTIC-182] [Bug] Error when using metrics.calculate with Amazon Athena adapter

Is this a new bug in dbt_metrics?

[X] I believe this is a new bug in dbt_metrics
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
When the metrics.calculate macro is used with a time grain in the dbt-athena adapter, a SYNTAX_ERROR is thrown and the model does not finish to run.
My metric:
version: 2

metrics:
  - name: num_images_received
    label: Number of Images received
    model: ref('stg_images_received')
    description: The number of unique images received in a time period.
    type: count_distinct
    sql: image_id
    timestamp: created_at
    time_grains: [day, week, month, quarter, year, all_time]

My model:
select *
from {{ metrics.calculate(
    metric('num_images_received'),
    grain='week',
) }}

Expected Behavior
The model compiles and runs correctly and stores the results in a table of the Athena data catalog, the table contains the columns date_week and num_images_received, just like when using other adapters such as Redshift.
Steps To Reproduce
You can reproduce the same behavior with the following example model:
1. Set your environment:
Use and configure the dbt-athena-community==1.3.5 provider with an AWS account and S3 Bucket following these instructions, then create a packages.yml file with:
packages:
  - package: dbt-labs/metrics
    version: 1.3.2

2. Create a model in models/metric_example.sql
Can be something simple such as:
select *
from
    (
        select
            1 as id,
            date('2023-01-08') as created_at
    )
union
(
    select
        2 as id,
        date('2023-01-09') as created_at
)
union
(
    select
        3 as id,
        date('2023-01-10') as created_at
)

3. Create a metric in models/metric_example.yml
version: 2

metrics:
 - name: num_examples
   label: Number of metric examples
   model: ref('metric_example')
   type: count
   sql: id
   timestamp: created_at
   time_grains: [day, week, month, quarter, year, all_time]

4. For the calculations you can create a model in models/metric_calculation_example.sql
-- depends_on: {{ ref('dbt_metrics_default_calendar') }}
select *
from {{ metrics.calculate(
    metric('num_examples'),
    grain='week',
) }}

Then you'll see that running dbt run -m metric_calculation_example will fail with an error message like:
Runtime Error in model metric_calculation_example (models/metric_calculation_example.sql)
line 82:13: Column 'parent_metric_cte.date_week' cannot be resolved

Relevant log output
I extracted this relevant part from the logs:

Column 'parent_metric_cte.date_week' cannot be resolved

Environment
- dbt-adapter & version: `dbt-athena-community==1.3.5`
- dbt_metrics version: `1.3.2`

Which database adapter are you using with dbt?
other (mention it in "Additional Context")
Additional Context
From compiling the model I was able to find that the error was in the CTE <metric>__final which does something like:
    select

        parent_metric_cte.date_week,
        coalesce(num_examples, 0) as num_examples

    from num_examples__spine_time as parent_metric_cte
    left outer join num_examples__aggregate
        using (
            date_week
        )

But I think Presto/Athena does not support using something like parent_metric_cte.date_week with the USING clause, rather it would expect it to use simply select date_week or do something like:
    select

        parent_metric_cte.date_week,
        coalesce(metric, 0) as metric

    from num_examples__spine_time as parent_metric_cte
    left outer join ON parent_metric_cte.date_week=metric__aggregate.date_week`

Reference: https://prestodb.io/docs/current/sql/select.html#using

Hi @Josersanvil! Appreciate such a well written issue - it really helps lay out what the issue. Unfortunately, Amazon Athena is not one of our supported adapters for dbt_metrics. If you're interested in resolving this issue, I recommend using an override of the join logic in that specific macro to make it Athena compliant.

Hi @callum-mcdata!
Oh I understand, makes sense. I must've missed that part when reading the documentation 🤦♂.
I had not think about overriding the macro! You are right, I think that would make it work. Thanks for your help!
Closing this issue now.

Hey @Josersanvil, I'm experiencing the same issue. Did you find which macro needed to be overriden?

@Jake-Gillberg Yes! I found it it's called default__gen_metric_cte (this file in the repo).
Here is how I did it. I created a file in macros/dbt_metric/gen_metric_cte.sql and then modified the columns that used select parent_metric_cte.date_{{grain}} to just select date_{{grain}} . Here is the full macro that you can just copy paste:
{%- macro gen_metric_cte(metric_name, grain, dimensions, secondary_calculations, start_date, end_date, relevant_periods, calendar_dimensions, treat_null_values_as_zero) -%}
    {{ return(adapter.dispatch('gen_metric_cte', 'metrics')(metric_name, grain, dimensions, secondary_calculations, start_date, end_date, relevant_periods, calendar_dimensions, treat_null_values_as_zero)) }}
{%- endmacro -%}

{%- macro default__gen_metric_cte(metric_name, grain, dimensions, secondary_calculations, start_date, end_date, relevant_periods, calendar_dimensions, treat_null_values_as_zero) %}

, {{ metric_name }}__final as (

    {%- if not treat_null_values_as_zero -%}
        {%- set metric_val = metric_name -%}
    {%- else -%}
        {%- set metric_val = "coalesce(" ~ metric_name ~ ", 0) as " ~ metric_name -%}
    {%- endif %}

    select
        {% if grain != 'all_time' %}
        date_{{ grain }},
            {%- if secondary_calculations | length > 0 -%}
                {% for period in relevant_periods %}
        parent_metric_cte.date_{{ period }},
                {%- endfor -%}
            {%- endif -%}
        {%- endif -%}

        {%- for calendar_dim in calendar_dimensions %}
        parent_metric_cte.{{ calendar_dim }},
        {%- endfor %}

        {%- for dim in dimensions %}
        parent_metric_cte.{{ dim }},
        {%- endfor %}
        {{ metric_val }}

    {%- if grain == 'all_time' %}

        ,metric_start_date
        ,metric_end_date

    from {{ metric_name }}__aggregate as parent_metric_cte

    {% else %}

    from {{ metric_name }}__spine_time as parent_metric_cte
    left outer join {{ metric_name }}__aggregate
        using (
            date_{{ grain }}
            {%- for calendar_dim in calendar_dimensions %}
            , {{ calendar_dim }}
            {%- endfor %}
            {%- for dim in dimensions %}
            , {{ dim }}
            {%- endfor %}
        )

        {% if not start_date or not end_date -%}
        where (
            {% if not start_date and not end_date -%}
            date_{{ grain }} >= (
                select
                    min(case when has_data then date_{{ grain }} end)
                from {{ metric_name }}__aggregate
            )
            and date_{{ grain }} <= (
                select
                    max(case when has_data then date_{{ grain }} end)
                from {{ metric_name }}__aggregate
            )
            {% elif not start_date and end_date -%}
            date_{{ grain }} >= (
                select
                    min(case when has_data then date_{{ grain }} end)
                from {{ metric_name }}__aggregate
            )
            {% elif start_date and not end_date -%}
            date_{{ grain }} <= (
                select
                    max(case when has_data then date_{{ grain }} end)
                from {{ metric_name }}__aggregate
            )
            {%- endif %}
        )
        {% endif %}
    {% endif -%}

)

{% endmacro %}

@Josersanvil so I don't know what the adapter name for the Athena connection is (gonna assume it's athena but you can also create an override for your adapter so that you don't need to change the base macro. That way you can keep the package updating and not override the local code.
You'd do this by creating a macro called athena__gen_metric_cte, with the logic specified above. This would the adapter.dispatch code to look for the athena version of the macro before it goes to the default. So as long as that macro exists in your project (location doesn't matter) it would be good to go.

@Josersanvil Thanks so much! My override looks a bit different than yours. I only modified a couple of the failing lines and kept it much closer to the current source (and used the adapter override stuff that Callum is talking about)
{%- macro athena__gen_metric_cte(metrics_dictionary, group_name, group_values, grain, dimensions, secondary_calculations, start_date, end_date, relevant_periods, calendar_dimensions) %}

{%- set combined_dimensions = calendar_dimensions | list + dimensions | list -%}
, {{group_name}}__final as (
    {# #}
    select
        {%- if grain %}
        parent_metric_cte.date_{{grain}},
            {%- if secondary_calculations | length > 0 -%}
                {% for period in relevant_periods %}
        parent_metric_cte.date_{{ period }},
                {%- endfor -%}
            {%- endif -%}
        {%- endif -%}

        {%- for calendar_dim in calendar_dimensions %}
        parent_metric_cte.{{ calendar_dim }},
        {%- endfor %}

        {%- for dim in dimensions %}
        parent_metric_cte.{{ dim }},
        {%- endfor %}

        {%- for metric_name in group_values.metric_names -%}
            {# TODO: coalesce based on the value. Need to bring this config #}
            {%- if not metrics_dictionary[metric_name].get("config").get("treat_null_values_as_zero", True) %}
        {{ metric_name }}
            {%- else %}
        coalesce({{ metric_name }}, 0) as {{ metric_name }}
            {%- endif %}
        {%- if not loop.last-%},{%endif%}
        {%- endfor %}

    {%- if secondary_calculations | length > 0 %}
    from {{group_name}}__spine_time as parent_metric_cte
    left outer join {{group_name}}__aggregate
        using (date_{{grain}} {%- if combined_dimensions | length > 0 -%}, {{ combined_dimensions | join(", ") }} {%-endif-%} )

    {% if not start_date or not end_date -%}
    where (
        {% if not start_date and not end_date -%}
        date_{{grain}} >= (
            select
                min(case when has_data then date_{{grain}} end)
            from {{group_name}}__aggregate
        )
        and date_{{grain}} <= (
            select
                max(case when has_data then date_{{grain}} end)
            from {{group_name}}__aggregate
        )
        {% elif not start_date and end_date -%}
        parent_metric_cte.date_{{grain}} >= (
            select
                min(case when has_data then date_{{grain}} end)
            from {{group_name}}__aggregate
        )
        {% elif start_date and not end_date -%}
        parent_metric_cte.date_{{grain}} <= (
            select
                max(case when has_data then date_{{grain}} end)
            from {{group_name}}__aggregate
        )
        {%- endif %}
        )
    {%- endif %}

    {%- else %}
    from {{group_name}}__aggregate as parent_metric_cte
    {%- endif %}
)

{% endmacro %}

This would also be a good candidate for a PR over at https://github.com/lalalilo/athena-utils/
I'll let you open the PR over there if you like, since you did the bulk of the work!

@callum-mcdata @Jake-Gillberg Thanks! Those are very good suggestions! I will try it out with the dispatched macro for Athena 🙂
Didn't know about the athena-utils/ package! It will definitely be useful for me. I will definitely open a PR for this there.

(FYI, if you add this to and are using athena-utils, you will need to add something like this to your dbt_project.yml - this had me tripped up a bit):
dispatch:
  - macro_namespace: metrics
    search_order: ['athena_utils', 'metrics']


Remove .rendered_content=None from database model

When running the 0257_fix_has_link_attribute.py migration on zulipchat.com, it crashed and needed to be restarted because the message with ID 153818321 had a .rendered_content of None.  We can and should perhaps adjust that migration to skip messages with .rendered_content being None just like it does '', but I think we should also work to eliminate the possibility in the database model for a message to have .rendered_content=None.  I believe that possibility in the data model dates to very early in Zulip's development, when the flow was to render a message after saving it to the database (I think at the time, we rendered when fetching messages, not when sending them).  We have for years now done rendering before saving messages to the database.  Given that, we should just change message.rendered_content to be not nullable.
I'm not sure what the first stage of the migration should do with any messages that have .rendered_content=None to fix them; the correct thing would be to rerender the message, but that's not something we have the infrastructure to easily do from within a database migration.  So I think the reasonable options are (1) to delete the message (2) to change its .rendered_content to ''.  Given that we currently also validate in check_message that messages don't have the empty string as rendered content (though that was introduced recently enough that there are plenty of Zulip servers with older messages with '' as rendered_content), I'm tempted to go with (1) and have the migrations be a bit more aggressive in also deleting any messages with the empty string as rendered content; then we would return to the invariant enforced by our code for processing of new messages that every message has non-empty content.
@mateuszmandera this is probably a good issue for you to take.

@zulipbot claim

Hi, I am new contributor to the zulip community. Please help me to get started with a good first issue. I tried claiming this issue as it seemed to be for beginners. Please help me. Thanks

This isn't a great issue for beginners, as it requires understanding details of how the database works.

@timabbott I see that hipchat, mattermost, gitter and slack import tools seem to set rendered_content = None - unless I'm missing some place where it gets rendered later - so that's probably something that needs to be addressed next?

The actual rendering happens in the import process; those are just placeholder values.


Monoid Solver for lists

A few weeks back, me, Exe and Guido tried to do a monoid solver for lists using FStar.Algebra.Monoid and it did not work as expected. Any recommendations?
Proof of concept:
module Test

open FStar.Tactics
open FStar.Algebra.Monoid

let right_unitality_lemma (m:Type) (u:m) (mult:m -> m -> m) =
  forall (x:m). x `mult` u == x

let left_unitality_lemma (m:Type) (u:m) (mult:m -> m -> m) =
  forall (x:m). u `mult` x == x

let associativity_lemma (m:Type) (mult:m -> m -> m) =
  forall (x y z:m). x `mult` y `mult` z == x `mult` (y `mult` z)

let list_append_monoid : monoid (list int) =
  let u : (list int) = [] in
  let mult (p q : (list int)) : list int  = p  @  q in

  let left_unitality_helper (p:(list int)) : Lemma ((u `mult` p) == p) = () in

  let right_unitality_helper (p:(list int)) : Lemma ((p `mult` u) == p) =
    List.Tot.Properties.append_l_nil p
  in

  let associativity_helper (p1 p2 p3 : (list int)) : Lemma (p1 `mult` p2 `mult` p3 == p1 `mult` (p2 `mult` p3)) =
    List.Tot.Properties.append_assoc p1 p2 p3
  in

  Classical.forall_intro right_unitality_helper ;
  assert (right_unitality_lemma (list int) u mult) ;
  Classical.forall_intro left_unitality_helper ;
  assert (left_unitality_lemma (list int) u mult) ;
  Classical.forall_intro_3 associativity_helper;
  assert (associativity_lemma (list int) mult) ;
  intro_monoid (list int) u mult

// fails
let test1 (a b c d : list int) =
  assert (a @ (b @ c) @ d == a  @  b  @  c  @  d) by
  (dump "h"; FStar.Tactics.CanonMonoid.canon_monoid list_append_monoid (* string_of_int *); dump "x")

// fails
let test2 (a b c d : list int) =
  assert (a @ (b @ c) @ d == a  @  b  @  c  @  d) by
  (
    dump "h";
    FStar.Tactics.CanonMonoid.canon_monoid list_append_monoid (* string_of_int *); dump "x"; trefl())

Thank you!

I am not sure why in the example we said mult is 'let mult (p q : (list int)) : list int  = p  @  q, but if we change that definition in let mult = (@)` the example works.

I am not sure why in the example we said mult is 'let mult (p q : (list int)) : list int  = p  @  q, but if we change that definition in let mult = (@)` the example works.


RewardForCollectingItem failing

This line:
        ItemStack itemStack = ByteBufUtils.readItemStack(buf);

in RewardForCollectingItemImplementation.onMessage sometimes fails.
The resulting itemStack is null, leading to a null pointer exception.
The null pointer exception can be fixed easily, but the main concern is why the readItemStack call fails. Rewards are going missing as a result.

This turned out to be due to corrupt ItemStacks, caused by the way we were encoding/decoding the ItemStack to a string for passing between the server and client.


Save File Inspector

Description: Simple custom inspector when selecting a save file. Just displaying some informations, button to open it, etc.
State: None.

Implemented Canvas and Node save inspector with 79a6567 and 6333c1c :)


Script to generate Singularity recipe from Dockerfile

Adding a bash script to generate a Singularity recipe from the Dockerfile.
Tested using:

RHEL 8
Python 3.9.1
Singularity Python (spython) 0.1.15
Singularity 3.8.0-1.el8 (via EPEL)

Unfortunately we don't have the capacity to maintain and support Singularity. We added a link in the README to the two AlphaFold issues that have links to third party repositories with Singularity definition files (https://github.com/deepmind/alphafold/issues/10 and https://github.com/deepmind/alphafold/issues/24).


Don't report result analysis separately on GitHub

Result analysis doesn't even fail at the moment (except for infrastructure errors). Once we complete https://github.com/Mbed-TLS/mbedtls/issues/2691, result analysis will have a failure case, and we expect it to be a stopper. There's no particular reason to report result analysis separately, so stop doing it. Treat it like any other job for reporting purposes. (It's separate from the other jobs for running purposes because it has to run after all the other jobs since it's analyzing their logs.)
(The reason result analysis reports separately is partly because I wasn't sure where it was going at the time and wanted it to be easily skippable, and partly because I don't know Groovy and the underlying libraries well and this ended up to be easy to write. But I don't think it's difficult to fix.)

From memory, the requirements for result analysis were:

It has access to the artifacts created by all test jobs. As a consequence, it must run after all test jobs. (Having access to the Jenkins logs of previous jobs might be nice but is currently not required and we don't have any concrete plans to use this information.)
It runs whether there have been test failures or not.
It has access to the source tree.
It doesn't do any builds, so it can run on any platform that has the required tools for the analysis (which currently just means non-antique Python).
It doesn't need to know the target branch for a pull request: it just analyses the content of the current branch. (It's plausible that we'll want to perform additional analyses for merge jobs one day, but currently we don't and have no concrete plans.)
It can set the build status to fail. (Which it currently never does unless there's a bug in the script, but that was the intent, we just have to get around to resolving https://github.com/Mbed-TLS/mbedtls/issues/2691.)

This is fixed by https://github.com/Mbed-TLS/mbedtls-test/pull/68. A follow-up will be needed to address https://github.com/Mbed-TLS/mbedtls-test/issues/120.


Replace profile sold statistic value number with subgraph value

Fixes #906  .
Changes proposed in this PR:

get subgraph sales value by user accountId
add value in profile provider
use new value in Profile Stats

This looks good! The issue with number of sales not cleared when no network selected is also solved.
One question, do we still need the getAccountNumberOfOrders function?


Localforage selects LocalStorage as default storage instead of IndexedDB on Firefox

Hi, thank you for this awesome library, it's extremely useful since I'm developing native code for the different platforms!
I'm testing it with different browsers and I have troubles using it with Firefox:

It seems Localforage sets LocalStorage driver as default instead of IndexedDB. When I open the developer tools, open the Strorage tab, the Local Storage filter shows my stored key-value pairs, the Indexed Database filter shows no results, it's empty.

I'm getting Quota Exceeded errors (I guess because Localstorage is much more limited than Indexed DB). I'm building an offline-response cache feature for our app so I'd need to store a lot of data locally.  What is the recommended way to do it with localforage?

Thx

Hmmm, that's pretty strange. Do you have any add-ons enabled that might mess with IndexedDB? Are you using private browsing mode?
localForage is running on https://localforage.github.io/localForage/; when I check the storage engine selected in the console I see asyncStorage, which is IndexedDB.

Having the same problem. LocalForage v1.7.3, imported into React component (import localForage from 'localforage';).

Firefox (Developer 78.0b6 (64-bit)):

Chromium-based Yandex browser (recentish, don't know exact chromium version):

Strangely enough Firefox call to localforage._driver on https://localforage.github.io/localForage/ reports "asyncStorage".

I've checked non incognito mode behavior and both Chrome and Firefox report asyncStorage.
I guess this is limitation on incognito mode on FF?

Encountered the same issue, it only seems to happen in Incognito mode on FF (tested on v73).
There's an open Mozilla bug for this, check out: https://bugzilla.mozilla.org/show_bug.cgi?id=1639542


hub.docker.com/r/homeassistant/home-assistant:latest

IMAGE SYNC

docker.io/homeassistant/home-assistant:latest


Sometimes the channel labelled "si10" (10 meter wind speed) has data from the hcct channel in it

The problem is in the intermediate Zarr:  Sometimes the channel labelled "si10" (10 meter wind speed) has data from the hcct channel in it!

my guess is that, sometimes, the variables are in the wrong order before appending to the Zarr (maybe dataset.to_array() puts the channels into a non-deterministic order?)

rough plan of action:
if appending to existing zarr then:

use reindex to reorder the variables dim to the order on disk
assert that all the other dims have the same coords
flip the y axis.

@jacobbieker this issue might also affect the EUMETSAT native-to-Zarr conversion.... but let me do a bit more digging into the NWP to Zarr conversion...

Okay! I believe Satip reorders the dims before writing to disk, but I'd have to check tomorrow.

OK, I've forced the order of the channels in the NWP zarr, and am writing again to /mnt/storage_ssd_8tb/data/ocf/solar_pv_nowcasting/nowcasting_dataset_pipeline/NWP/UK_Met_Office/UKV/zarr/UKV_intermediate_version_3.zarr
PR to follow...

In Satip, I use transpose instead of reindex here: https://github.com/openclimatefix/Satip/blob/9091a92fc1ef05d84821d2e41dfd42734797c8fc/satip/utils.py#L214 which looking at the docs for transpose and reindex look like they do similar things, but not the same. So that could be the issue with the satellite Zarrs


Check for heat api availability before creating the heat stack

Due to onadmin_cleanup_db_mq_vips seemingly restarting the db/rpc
services on the resource cleanup, the heat-api may be unavailable
for some time after the cleanup.
This introduces a little wait before creating the heat stack to
see if the heat api is up and running correctly

Alternative to: https://github.com/SUSE-Cloud/automation/pull/1839
Fixes(?): https://trello.com/c/Ut0rc4Ux/331-heat-stack-create-error-500-on-ha-upgrade-jobs

removed the FIXME as codeclimate does not like that. Also mkcloud run failing has nothing to do with this patch

so we are adding this workaround because the other workaround created undesired behaviour? I hope, this does not continue much further.
 Did it help?

@bmwiedemann my local env is not working due to some stonith issues with using libvirt so I cannot test it properly. I'll try to test & deploy it on mkcloud2


Delete Trigger Feed Before Trigger

Dissociate a feed from a trigger before a trigger is deleted

Related to https://github.com/openwhisk/openwhisk-package-alarms/issues/55

@jasonpet, @bjustin-ibm, FYI.

PG2 1495

@mdeuser, please review.

The deletion of the trigger before the feed is deliberate.

@dubeejw - is there resolution re: @rabbah's comment above?

Resolution: ttrigger is deleted whether the feed action succeeds or not.

PG4 182, approved. Need to make review changes still.

@mdeuser, PR updated per review comments.

@mdeuser, in regard to the concern above, the trigger will always be deleted even if the feed fails to delete.

@dubeejw - go changes look good.
How about https://github.com/openwhisk/openwhisk/pull/2187#issuecomment-297492641 ?

@mdeuser, see https://github.com/openwhisk/openwhisk/pull/2187#issuecomment-298384969.


[BACK-1443] [BACK-1466] Improve error handling, fix units in plural p…

…umpSettings
Improved error handling when downloading datasets that are too large
to be processed within the k8s timeout periods.
Fix incorrect mg/dL units in "plural" pumpSettings data.

Upgrade dependencies
Fixes BACK-1443
Fixes BACK-1466

@derrickburns I don't believe so. Looking at https://github.com/tidepool-org/export/pull/26/files#diff-0364f57fbff2fabbe941ed20c328ef1aR128-R135, is all in the same block, so either the end to the stream will fire, or the timeout will make sure that the catch block is executed.


Feature init js

Completed the initial JS -

Display front-end,
load autocorrect,
load current weather
load forecast weather - include the cleaning up of the previous forecast (if any)

Keep the branches in this project


Impossible to read thrift object from parquet file if it has List field that was removed from thrift schema.

If thrift class has a field with type List<some_enum> ParquetReader makes list's elements type as enum (type id = 16) but it has to make it Int32.
What happens is all fields that have field type as enum in thrift schema file in java class have field type as Int32. Same is true for List fields if list's elements are enum.
But when ParquetReader creates an object it uses type enum for list's elements instead of Int32.
Because of this fact we have an issue. We can not remove list field if it has enum elements. If we remove field like this from schema file but it will present in parquet file, when ParquetReader reads this field it tries to skip it because this field is not in the schema and it calls method TProtocolUtil.skip method with type = 15 for list and then it calls same method for each list element with type 16 for enum but TProtocolUtil.skip doesn't have this type in switch-case and it is not skipping list elements and because of this it throws exception later when it tries to skip List end.
Reporter: Andrei Stankevich
Note: This issue was originally created as PARQUET-1046. Please see the migration documentation for further details.

Rob Russo:
For anyone else hitting this issue, at the moment we implemented a workaround by adding an ENUM case to the switch for TProtocolUtil.skip and just made the logic the same as it is for I32. This should be safe since it should be stored as an I32 and nothing else should be internally using the ENUM type so it should only impact the parquet thrift implementation. This seemed to be an easier workaround than modifying the parquet-thrift library.


dataset 第一列是数字时，饼状图显示错误

[ ] I am using English in this issue. 在这个 Issue 中我使用了英文（强烈建议）。

General Questions

[x] Required: I have read the document and examples and tried to solve it by myself. （必填）我读过了文档和教程，并且曾试图自己解决问题。
[x] Required: I have searched for similar issues and that didn't help. （必填）我搜索过 issue 但是没有帮助。
[x] Required: I have tried with the latest version of ECharts and still have this problem. （必填）我试过最新版本的 ECharts，还是存在这个问题。

In this issue, I have provided information with: 在这个 issue 中我提供了以下信息：

[x] Required: issue type；（必填）issue 类型
[x] Required: one sentence description in issue details；（必填）一句话的问题描述
[x ] Required: demo；（必填）能反映问题的例子（如果你想提问或报 bug）
[x] Required: ECharts version；ECharts 版本

Issue Type

[ ] I have a question to ask about how to use ECharts to ...；我想提问如何使用 ECharts 实现某功能
[x] I have a bug to report；我想要报 bug
[ ] I have a feature to request, e.g.: I'd like a new feature that ...；我需要一个新功能
[ ] I have a feature to enhance, e.g.: The current feature should be improved in the way that ...；我希望改进某个功能
[ ] There's something wrong with the documents；文档有些问题
[ ] Others, or I'm not sure which issue types to choose here；其他，或我不知道应该选什么类型

Issue Details

dataset 有两列，第一列全是数字时，饼状图会将第一列既当成维度，也当成指标
Expected Behavior

dataset 有两列，即使第一列全是数字，饼状图将第一列作为维度，第二列作为指标
Current Behavior

dataset 有两列，第一列全是数字时，饼状图会将第一列既当成维度，也当成指标
Online Example

https://codepen.io/AliJser/pen/EOqYQj

var option = {
  legend: {},
  tooltip: {},
  dataset: {
    source: [
      ["时间（分钟）", "数量"],
      ["10", "51141"],
      ["40", "23130"],
      ["220", "4803"],
      ["500", "1282"],
      ["575", "931"],
      ["665", "674"],
      ["825", "440"],
      ["1375", "21"],
      ["1435", "18"],
      ["1145", "8"]
    ]
  },
  series: [{
    type: 'pie',
  }]
};

Topics

[ ] Legend
[ ] Tooltip
[ ] Event
[ ] Performance
[ ] SVG
[ ] Map
[ ] ECharts GL
[ ] Third-party libraries, e.g.: Vue.js, React

Anything Else We Need to Know

Environment

ECharts version；ECharts 版本: {DESCRIBE_HERE}
v4.1.0

[ ] It happens only on certain browsers or operating systems. 对于特定浏览器或操作系统才会出现的问题，请提供相应环境信息：{BROWSER_VERSION_OR_OS_INFORMATION_HERE}

this is a bug.you can set tooltip.formatter first

This is not a bug. Please use it in this way:
var option = {
  legend: {},
  tooltip: {},
  dataset: {
    source: [
      ["时间（分钟）", "数量"],
      ["10", "51141"],
      ["40", "23130"],
      ["220", "4803"],
      ["500", "1282"],
      ["575", "931"],
      ["665", "674"],
      ["825", "440"],
      ["1375", "21"],
      ["1435", "18"],
      ["1145", "8"]
    ]
  },
  series: [{
    type: 'pie',
    encode: {
        value: '数量'
    }
  }]
};

@Ovilia

there isn't a 'encode' property on pie series

不用dataset 直接用series的 data赋值 就不会有这个问题

Set sourceHeader to be false.

option = {
    legend: {},
    tooltip: {},
    dataset: {
        dimension: ['product', 'count'],
        source: [
            { name: '123', count: 2 },
            { name: '355', count: 2 },
            { name: 'prsd', count: 2 }
        ],
        sourceHeader: false
    },
    xAxis: {type: 'category'},
    yAxis: {},
    // Declare several bar series, each will be mapped
    // to a column of dataset.source by default.
    series: [
        {type: 'pie'},
    ]
};

option = {
    legend: {},
    tooltip: {},
    dataset: {
        dimension: ['product', 'count'],
        source: [
            { product: '123', count: 2 },
            { product: '355', count: 2 },
            { product: 'prsd', count: 2 }
        ],
        sourceHeader: false
    },
    xAxis: {type: 'category'},
    yAxis: {},
    // Declare several bar series, each will be mapped
    // to a column of dataset.source by default.
    series: [
        {type: 'pie'},
    ]
};


Section 1.2 mentions the DILCIS Board without any previous introduction to DILCIS

This goes in line with my comment #231 about the poor structuring of the introduction section. Key concepts are not being well introduced to the reader.
Maybe @DrJaime  can review this entire section to make it clear to the unexperienced reader.

Hi Miguel,
This certainly can be looked at but at present the CS IP text is in a state
of limbo. We can do changes but not make approved PR's until after we have
seen what the review comments are.
Best wishes,
Jaime
On Mon, 17 Dec 2018 at 11:51, Miguel Ferreira<EMAIL_ADDRESS>wrote:

This goes in line with my comment #231
https://github.com/DILCISBoard/E-ARK-CSIP/issues/231 about the poor
structuring of the introduction section. Key concepts are not being well
introduced to the reader.
Maybe @DrJaime https://github.com/DrJaime can review this entire
section to make it clear to the unexperienced reader.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/DILCISBoard/E-ARK-CSIP/issues/236, or mute the thread
https://github.com/notifications/unsubscribe-auth/Al-qggkwWV-gB5VUaU1L_xtq773bC9t1ks5u54UtgaJpZM4ZWM5r
.

Common introduction made and gets added in the transformations.


os-ddclient plugin does not support linode dynamic dns update

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[ X ] I have read the contributing guidelines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
[ X ] I have searched the existing issues, open and closed, and I'm convinced that mine is new.
[ X ] The title contains the plugin to which this issue belongs

Describe the bug
Linode updates are not supported ddclient, but are supported with legacy dyndns client
With the forced update to ddclient in the future, there is a lack of provider options compared to the DynDNS-OS plugin. Can before the forced switchover parity to all providers as previously available with the os-dyndns plugin be achieved?
I use Linode to update my dynamic DNS domains, are there at least instructions on how to add Linode as a custom provider to ddclient?
To Reproduce
Steps to reproduce the behavior:

Install os-ddclient
Try to convert os-dyns configuration to os-ddlcient config

Expected behavior
Have feature parity with the to be depreciated plugin
Environment
OPNsense 22.1.4_1-amd64

I second this, as I realized it after moving away from pfSense to OPNsense. Would be nice to have it supported.

Same here, using Linode as my domain DNS hosting. Ok for now but I don't find a propet way to convert my configuration to os-ddclient. I think Linode is a pretty serious company and should be there.

No Traction on this,  or at least instructions on using custom?  We're on the cusp of forced update

Maybe not the best solution, but worked for me while a native ddclient (or other plugin) solution arrives:
I created a no-ip host that I update from opnsense (any ddclient supported service should work) and then I created a CNAME record on the Linode Domain (DNS) Manager pointing to the no-ip host.
A script that employs curl (as shown in https://www.linode.com/docs/api/domains/#domain-record-update) probably would be a better way to implement a more straightforward solution.


Deprecate Event.get_by_ident() and Event.get_ident.

Rename URL params from event_ident to slug.
Fixes #874

Cleaning! I like this.


Update FAQ page

Split FAQ page up by category
Update each category with additional links and information

Pinging @reactjs/redux if anyone wants to take a look-through.  If there's no immediate complaints, I'll merge this in tomorrow.

Note that I kept all the existing short hashtags in the main page, and moved them to be adjacent to the TOC links to the new locations.  So, all the old hashtags will still point to a relevant spot in the page, just with an extra click's worth of indirection.

Structurally, this is definitely a big improvement.

Cool.  If no other comments in the next hour or so, I'll merge.
@timdorr , you mind re-publishing the docs once this is in?  I've got other stuff I'd like to tackle atm, rather than figuring out exactly how to sort through issues publishing docs under Windows right now.

Yep, I'll do that a little bit later tonight.

I like it, nice one!

No further complaints, aaaaaand merging!


Invoke firing before transition completes?

Bug or feature request?
Bug
Description:
On transitioning to a new state, invoke is called prior to the transition ending, leading to an incorrect final state if invoke meanwhile transitions.
(Bug) Expected result:
(Bug) Actual result:
(Bug) Potential fix:
(Feature) Potential implementation:
Link to reproduction or proof-of-concept:
https://codesandbox.io/s/k5opkj68vr

I'm sure this isn't a bug, just a gap in my understanding. How would I go about accomplishing the use-case in the provided example? I want to make sure the states occur in serial: initial -> validateOrder -> showErrors.

Confirmed fixed with 4.4.0


problem with gen_proposal_list.py

Hi.
I tried to use SSN.
I extracted frame images to "output"directory.
Then, I run this code.
But this error appears, and not completed.
hogehoge:/app/action-detection# python gen_proposal_list.py thumos14 output parse frames under folder output 0 videos parsed 200 videos parsed Traceback (most recent call last):   File "gen_proposal_list.py", line 29, in <module>     frame_dict = parse_directory(args.frame_path, key_func=key_func)   File "/app/action-detection/ops/io.py", line 80, in parse_directory     all_cnt = count_files(f, (rgb_prefix, flow_x_prefix, flow_y_prefix))   File "/app/action-detection/ops/io.py", line 73, in count_files     lst = os.listdir(directory) OSError: [Errno 40] Too many levels of symbolic links: 'output/video_test_0000966'
I tried to solve this error, but I couldn't.
(for example: I made a symbolic links in ../output)
Please someone tells me the good way to solve this problem.

Too many levels of symbolic links: 'output/video_test_0000966'
Are you making self-looped symbolic links ?

I cheked again and found the self-looped symbolic links.
Thank you!!
However, new problem appeared.
I ran this command.
python3 ssn_train.py thumos14 RGB -b 16 --lr_steps 20 40 --epochs 45
I got  new error
RuntimeError: cuda runtime error (2) : out of memory at
Then, I read #26.
I tried this code
python ssn_train.py thumos14 RGB -b 1 -i 16 --lr_steps 320 640 --epochs 720
The error didn't appear.
Do I need to change learning rate when I use above command???

It means your GPU doesn't have enough memory or you have too few GPUs installed. Its recommend to 4~8 GPUs with 12G memory. You can try a smaller batchsize if you don't have that I cannot guarantee the accuracy.

Why is thumos 14 _ tag _ val _ normalized _ proposal _ list.txt having so many videos, isn't it 413 in total verified and tested?

Why is thumos 14 _ tag _ val _ normalized _ proposal _ list.txt having so many videos, isn't it 413 in total verified and tested?

Do you kone why?@pengxiaoxiao


Change background look and add analyze lapse feature in home page

These are the changes I made :

Added Particles.js feature to make the background look dynamic
Blur effect to the background behind foreground
Add Analyze lapse feature in the home page to enhance user experience
Set the nav bar to -z-10

https://github.com/user-attachments/assets/322c1be5-9d1b-4184-9b21-77faab250adb

Why isn't the PR merged yet ?  Any issue with the code ?

Great Work!!


Possible error in InstallDependency logic

I'm not fixing that because I don't understand the intention behind that code and maybe I'm missing something important but possibly there is an error in the logic of this function:
https://github.com/Bedrock-OSS/regolith/blob/cd426fbbe8fa478b8581519a968098e3807d0dba/src/remote_filters.go#L96-L109
On line 100 there is an the condition used in if statement will always return true. The values in checked map are at that point either missing or true.

CASE 1: checked[filter.RunWith] exists -> ok == true AND c == true -> ok || !c == true
CASE 2: checked[filter.RunWith] is missing-> ok == false AND c == false  -> ok || !c == true
ok || !c is always true.

Similar code here:
https://github.com/Bedrock-OSS/regolith/blob/cd426fbbe8fa478b8581519a968098e3807d0dba/src/filters.go#L73-L86


The tool is not supported on Linux

I'm trying to write my own python script to interface with my BASICR3, i have managed to connect to the device and get the configurations(apiKey, deviceID, etc.), but when i try semding my POST requests i get no response at all, have you got any tips or recommendations for doing such a thing? PS. I am using Linux and as far as i can see the provided program is not supported on linux

From another customer:
Why is not possible to connect in diy mode with websocket? why you have excluded all linux / Mac users?

If you are using an APi key, I suspect you are are not in DIY mode, ie you do not have the jumper fitted. In this mode you need to encrypt the request.

Why do you say Linux and Max users are excluded, Python should run on those platforms and I’ve been using the API (though not the tool as yet) from Linux.

I am trying to use from linux too. No success. The provided python code has inconsistencies (at least, mixed usage of spaces and tabs and some incorrect indentations). Also you have to installa some python packages. Even after that, I still has a message I don't know how to address: 'Could not load the Qt platform plugin "xcb" in "" even though it was found'.
I am no proficient on python, so I don't know how to solve this.
The least I can say is that this code was not really writtten with portability in mind :(


The roots of a Lie algebra are a root system.

We already have the definition of root systems and the definition of roots of a Lie algebra as well as basic facts such as the fact that they span.
We should finish the proof that the roots are actually a root system.

The following pull requests are work toward this goal:

[x] https://github.com/leanprover-community/mathlib4/pull/10454
[x] https://github.com/leanprover-community/mathlib4/pull/10548
[x] https://github.com/leanprover-community/mathlib4/pull/10584
[x] https://github.com/leanprover-community/mathlib4/pull/10638
[x] https://github.com/leanprover-community/mathlib4/pull/10748
[x] https://github.com/leanprover-community/mathlib4/pull/10803
[ ] https://github.com/leanprover-community/mathlib4/pull/10879
[ ] https://github.com/leanprover-community/mathlib4/pull/10893

The following two sorry captures the remaining work until this issue can be closed:
import Mathlib.Algebra.Lie.Weights.Killing
import Mathlib.LinearAlgebra.RootSystem.Basic

noncomputable section

namespace LieAlgebra.IsKilling

open LieModule Module

variable {K L : Type*} [Field K] [CharZero K] [LieRing L] [LieAlgebra K L]
  [IsKilling K L] [FiniteDimensional K L]
  {H : LieSubalgebra K L} [H.IsCartanSubalgebra] [IsTriangularizable K H L]

def reflectRoot
    {α β : Weight K H L} (hα : α.IsNonZero) (hβ : β.IsNonZero) : Weight K H L where
  toFun := β - β (coroot α) • α
  weightSpace_ne_bot' := by
    /- Several proofs of this. Probably best (though not shortest) is
      Bourbaki, Ch. VIII §2.2 Lemma 1 (page 80 of English translation, 88 of PDF)

      Approximate plan:
        1. define exponential of nilpotent things (independent of Lie theory)
        2. prove exponential of element with nilpotent adjoint endomorphism gives `LieEquiv`
           (slightly more is true: exponential of any nilpotent `LieDerivation` gives `LieEquiv`)
        3. prove adjoint endomorphism of element of rootspace is nilpotent
        4. establish the identities in Bourbaki lemma 1 for `θ := exp e * exp (- f) * exp e`
           where `e` and `f` belong to an `sl₂` triple associated to `α` provided by
           `exists_isSl2Triple_of_weight_isNonZero`.
    -/
    sorry

lemma reflectRoot_ne_zero
    {α β : Weight K H L} (hα : α.IsNonZero) (hβ : β.IsNonZero) :
    (reflectRoot hα hβ).IsNonZero := by
  sorry

variable (H)

/-- The root system of a finite-dimensional Lie algebra with non-degenerate Killing form over a
field of characteristic zero, relative to a splitting Cartan subalgebra. -/
def rootSystem :
    RootSystem {α : Weight K H L // α.IsNonZero} K (Dual K H) H :=
  RootSystem.mk'
    IsReflexive.toPerfectPairingDual
    { toFun := (↑)
      inj' := by
        intro α β h; ext x; simpa using LinearMap.congr_fun h x  }
    { toFun := coroot ∘ (↑)
      inj' := by rintro ⟨α, hα⟩ ⟨β, hβ⟩ h; simpa using h }
    (fun α ↦ by simpa using root_apply_coroot α.property)
    (by
      rintro ⟨α, hα⟩ - ⟨⟨β, hβ⟩, rfl⟩
      simp only [Function.Embedding.coeFn_mk, IsReflexive.toPerfectPairingDual_toLin,
        Function.comp_apply, Set.mem_range, Subtype.exists, exists_prop]
      exact ⟨reflectRoot hα hβ, reflectRoot_ne_zero hα hβ, rfl⟩)
    (by convert span_weight_isNonZero_eq_top K L H; ext; simp)

end LieAlgebra.IsKilling

@erdOne has done this and more, see Zulip and this gist
We just need to get it into master!

Done!


Remove PHP-FPM and use old plain CGI

Hello,
I'm working on a Drupal 7 project and I'm having some issue with the Shield module. Seems like it doesn't work when using php-fpm.
Is there a way to provision the VM without FPM, but with the mod_php only?
Thank you in advance for your help and ... your VM rocks :)

I managed to solve it by remooving the ProxyMatch directive in the apache config var.
Hope this is the right way :)

@condor616 - Depending on the version of Apache installed, OS, etc., just removing that directive and reprovisioning may switching things to mod_php, but the default/expected configuration is only using FPM at this point (for either Apache or Nginx, or any future webserver that we'll incorporate).
A lot of modules in Drupal-land assume only Apache/mod PHP for all webservers in the world, but there are a huge number of Nginx, IIS, Caddy, and Apache/FPM servers, so this smells like a bug with the Shield module to me ;)
Anyways, marking as closed for now.


Fix formatting issues

Fix formatting issues:

bold text that's supposed to be bolded
fix bullet point list

Thanks!


feat: take starry out of experimental

Take starry out of experimental, turned into jaxoplanet.starry
I created a jaxoplanet.starry.core module and adapted the directories of the tests to the new structure

I would like to make the following tutorials before merging and releasing:

[ ] units in jaxoplanet (and do #166 )
[ ] multi-wavelength light curves (for example #164)

* [ ]  multi-wavelength light curves (for example [Fitting multi-wavelength lightcurves #164](https://github.com/exoplanet-dev/jaxoplanet/issues/164))

I'll do this. I have a notebook that I just need to add some comments to for it to become a tutorial.

Great! I think it would be nice in the tutorial to show (%timeit) how well the approach scales with the number of wavelengths as it seems to be the concern raised in #164.

I ended up copying the units tutorial from @dfm in branch units-tutorial. Dan, anything you thing would be good to add?
ps: I've attached units to starry Surfaces in starry-units, I'll PR after this one


Shadows of Adam: White screen issue

On my OLED deck, this game boots to a white screen normally with audio, but it renders with wined3d.
Software information
Shadows of Adam
System information

GPU: AMD Van Gogh
Driver: Mesa 23.1.3
Wine version: 9.0
DXVK version: Unsure what version is being used in proton

Apitrace file(s)

Put a link here

For instructions on how to use apitrace, see: https://github.com/doitsujin/dxvk/wiki/Using-Apitrace
Log files
Please attach Proton or Wine logs as a text file:

When using Proton, set the Steam launch options for your game to PROTON_LOG=1 %command% and attach the corresponding steam-xxxxx.log file in your home directory.
When using regular Wine, use wine game.exe > game.log 2>&1 and attach the resulting game.log file.

steam-506510.zip

This uses ANGLE, which uses DirectComposition to present in D3D11 mode, which in turn isn't implemented in wine.
You can force this to use D3D9 via --use-angle=d3d9, which should work. On wined3d it seems like this falls back to a different renderer for some reason, but only on Deck, not on desktop.

This uses ANGLE, which uses DirectComposition to present in D3D11 mode, which in turn isn't implemented in wine and would also require some sort of DXVK support.
You can force this to use D3D9 via --use-angle=d3d9, which should work. On wined3d it seems like this falls back to a different renderer for some reason, but only on Deck, not on desktop.

I tried the angle=d3d9 method and that makes it render, but it's way too laggy to be considered playable, moving around on the menu takes multiple seconds to register.

Try with WINEDLLOVERRIDES=dcomp=d instead.

That seemed to fix it, now the game renders fine now without turning into a slideshow.


Get block info

Is there any API available to get the block info?
Are you planning to add it?
I'm looking for this functions
Get block info by height with list of transactions
Get last block synced
Thank you!

nothing like that yet

Resolved with https://github.com/ArkEcosystem/ark-rpc/blob/json-rpc/lib/methods/blocks/latest.js in an upcoming release.


Issue: output_parsers to work with lists of custom objects?

Issue you'd like to raise.
So It's possible to parse "complex" objects with
response_schemas = [
    ResponseSchema(name="date", description="The date of the event"),
    ResponseSchema(name="place", description="The place where the event will happen"),
]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)

and
class Pair_of_numbers(BaseModel):
    L: int = Field(description="A number")
    R: int = Field(description="A number that ends in 2, 3 or 4")

    @validator("R")
    def question_ends_with_question_mark(cls, field):
        if int(str(field)[-1]) not in [2, 3, 4]:
            raise ValueError("No bro :(")
        return field

parser = PydanticOutputParser(pydantic_object=Pair_of_numbers)

But I can't seem to find information on how to parse a list of custom objects/dicts? There is the list parser, but this is just for simple strings. It's possible there is a way to achieve this wrapping a pydantic BaseModel in a python List, but so far I've had no luck 😫
Suggestion:
No response

It doesn't work, somehow  I got
ValueError: Value not declarable with JSON Schema, field: name='_events' type=LittlyEvent required=True

when trying to parse this:
class LoggerOutput(BaseModel):
    events: List[LittlyEvent] = Field(
        ..., description="The list of events logged for the baby"
    )

    class Config:
        arbitrary_types_allowed = True

parser = PydanticOutputParser(pydantic_object=LoggerOutput)


[Suggestion] Make sorting by a country

Functionality of your mod already is fine. I think it's a good time to fill it with new features! First that came to my mind:
Make a sorting feature, that lets sort servers by country with options:

From farthest to closest
From closest to farthest
Disable a specific country from showing up in list of servers
Sort by ping from lowest to highest
Sort by ping from highest to lowest

Another suggestion is to add compatibility with Server Browser to make sorting even more useful. If it's possible of course.

Since it takes a while to get this information about servers, and that mostly only servers visible on the screen get pinged, I don't think real-time sorting would be a great idea, especially for a mod that just adds flags. Just a question, do flags show up in the Server Browser mod too?

No, they are not showing up.
Sorry to bother you, thought it was a good idea that came to my mind.

No worries! Although making the mod work with the Server Browser might be something to think about.


Draft up local queue mechanism

As a follow up of #40 if necessary.
But instead of pushing to Clickhouse, we might want to store an internal queue (maybe use SQLite) and then dump the aggregated data into Clickhouse at every 1 second (or 5 seconds)
The flow would look like:

Webhook POST
Webhook wait for response from Clickhouse Sink
Clickhouse sink receives POST request
Clickhouse sink stores message into local SQLite queue
Clickhouse sink responds "OK" to Webhook response
in background, Clickhouse sink waits for 1-5 seconds to push INSERT INTO bulk SQL inserts to Clickhouse

That way the Webhook response won't get blocked waiting for Clickhouse and allows us to aggregate the SQL inserts into Clickhouse at a slower rate by sending bulk updates

We have Bun SQLite code you can re-use here:
https://github.com/pinax-network/substreams-sink-websockets/blob/main/src/sqlite.ts
With some tests:
https://github.com/pinax-network/substreams-sink-websockets/blob/main/src/sqlite.spec.ts

First draft in #61

Proposed flow in #61

Merged #61. Followup will be done in #63.


Apply debugging redux example

Fix

Need to show the beginners how to use Redux and debugging

Changes

Add load spinners until you get to respond to API calls
Apply Redux to previous example
Change directory name

How Verified
(Automated Tests)

N.A

(Manual Tests)

Test in local

Notes

Need to write documents more to explain

References

https://redux.js.org/
https://react-redux.js.org/

I'm looking forward to your documentation.


Evaluation of px or 5% Error onKITTI 2015.(table 4)

Hello, I would like to ask if Table 4 is the model uploaded to KITTI website for testing? If not, how do I calculate them, and does bg refer to the occluded area, and does fg refer to the non-occluded area?

Hi @Miaowei-HNU these results are from KITTI test data, calculated by KITTI website.
‘bg’ refers to background. ‘fg’ refers to foreground.

Thank you for your reply

Hi @mli0603 ,I feel that my fine-tuning result is close to yours, but the L1_raw is always very high. Is L1_raw necessary? It can be seen from the code that the difference between L1_raw and L1 is disp_pred with different resolutions.

Hi @Miaowei-HNU , L1-raw is the metric of the cross-attention raw disparity at a lower resolution, which ideally should be low similarly to L1. In KITTI 2015 however, we have identified that the occlusion mask is ill-posed (our follow up paper in ECCV). Thus, the large error you see is mostly in the occlusion region (you can also visualize the raw disparity to see what is going on).
The context adjustment layer learns to smooth out the occlusion errors in raw disparity map, thus leading to a much lower L1 error in the final estimation.
What does this mean? KITTI 2015 gives an unfair evaluation against our approach and STTR has to unlearn the "correct" estimation from transformer and learns the "incorrect" estimation from the context-adjustment layer.
I hope this helps.


Improve PrecisionResult

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (9ec08ef) to head (4e0da51).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #111   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           36        36
  Lines          868       840   -28
  Branches        99        95    -4
=========================================
- Hits           868       840   -28

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Protect from lingering inactive summarizers

Currently, summarizer clients can outlive their parent containers for just long enough to summarize one more time.  This means that when other clients are deciding whether or not to spawn a summarizer, they also verify that there are no other summarizer clients in the quorum.
It might be nice to protect against the error case where a bad/inactive summarizer client remains in the quorum indefinitely, preventing other clients from spawning summarizers.

@arinwt , is this still interesting task? I think code accounts for any old client, including summarizer, so this issue is fixed?

We are already working on this anyway when changing elected summarizer client #5222. I will close it when I make that change.