Fix Firebase example

Apply fixes as per comment by @mocon at https://github.com/remix-run/remix/pull/1811#issuecomment-1090544230
Also:

throw meaningul errors when environment variables are missing or can't be parsed.
check for existence of environment variable rather than cast to string

Closes: issue reported in comment https://github.com/remix-run/remix/pull/1811#issuecomment-1090544230

[ ] Docs N/A
[ ] Tests N/A

Testing Strategy:
cd examples/firebase-auth-firestore
yarn
yarn dev
# open http://localhost:3000
# assert "Error: Missing CLIENT_CONFIG environment variable" thrown as env variables are missing
# exit ^C
cp .env.example .env
yarn dev
# open http://localhost:3000
# assert "Error: Failed to parse private key" thrown as example env does not contain valid keys
# exit ^C
# manually edit .env and set invalid keys
yarn dev
# open http://localhost:3000
# assert "Error: Invalid CLIENT_CONFIG environment variable"
# exit ^C
# manually edit .env and set valid keys
yarn dev
# open http://localhost:3000
# assert login page loads

I plan to follow up with a couple of other PRs:

fallback to Firebase emulator on localhost when environment variables are not set
use REST API for auth instead of client side SDK as detailed at https://github.com/remix-run/remix/pull/1811#discussion_r884011463
integrate other Firebase services in to this example as suggested at https://github.com/remix-run/remix/pull/1811#issuecomment-1050238130

@remix-run-bot CLA signed


Buckey by Feeds / Communities rather than Mine / Global

Issue 4 https://github.com/near-daos/astro-ui/issues/91
Order:
My Feed
Global Feed
--
My DAOs
All Communities
--

Example:

We are planning to get rid of the sidebar in the nearest feature. The global feed makes minimal sense since the user cannot interact with 99% of the global proposals. Also, grouping should be done by permissions level - currently, UI clearly separates what is available for logged in and what is available for non-authorized users.


NSSortDescriptors works differently

I got different results with same NSSortDescriptors. Here is the code
        let sortDescriptor = NSSortDescriptor(key: "startDate", ascending: false)
        finder.sortDescriptors = [sortDescriptor]
        _fetchedResultsController = finder.fetchedResultsController(nil)

tableView result
Item A
Item B
Item C

If I query all objects I get different order.
       var users = User.all().sorted(by:"startDate", false).find()
       for user in users {
       	println(user)
       }

Result:
Item C
Item B
Item A

It is very confusing. Is it a bug or for loop gives item in reverse order of fetchedResultsController?

@tosbaha thanks . Closing this and keeping in mind for 2.0 design.


Access Denied when fetching local_config_cuda while building TF from source

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): source
TensorFlow version: 2.3.+
Python version:  3.7.4
Installed using virtualenv? pip? conda?: pip
Bazel version (if compiling from source): 3.1.0
GCC/Compiler version (if compiling from source): N/A
CUDA/cuDNN version: 10.1/7.6.5
GPU model and memory: Nvidia GeForce 940 MX 4Gb

Describe the problem
When i try to build the tensorflow from the source i got from the repo, the build fails when trying to build package for TF 2.x
Provide the exact sequence of commands / steps that you executed before running into the problem
All steps were followed as mentioned in the official documentation here
Step 0: Got all the mentioned pre-requisites and included them all in the path variables
Step 1: Downloaded and checkout the master branch of tensorflow repo using git
Step 2: cd tensorflow
python ./configure.py
The output of the configure file is as follows:
F:\tfrep\tensorflow>python configure.py
You have bazel 3.1.0 installed.
Please specify the location of python. [Default is F:\python\python.exe]:

Found possible Python library paths:
  F:\python\lib\site-packages
Please input the desired Python library path to use.  Default is [F:\python\lib\site-packages]

Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Found CUDA 10.1 in:
    C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/lib/x64
    C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/include
Found cuDNN 7 in:
    C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/lib/x64
    C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/include

Please specify a list of comma-separated CUDA compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Each capability can be specified as "x.y" or "compute_xy" to include both virtual and binary GPU code, or as "sm_xy" to only include the binary code.
Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 3.5,7.0]: 5.0

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is /arch:AVX]:

Would you like to override eigen strong inline for some C++ compilation to reduce the compilation time? [Y/n]:
Eigen strong inline overridden.

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl            # Build with MKL support.
        --config=monolithic     # Config for mostly static monolithic build.
        --config=ngraph         # Build with Intel nGraph support.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v2             # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=noaws          # Disable AWS S3 filesystem support.
        --config=nogcp          # Disable GCP support.
        --config=nohdfs         # Disable HDFS support.
        --config=nonccl         # Disable NVIDIA NCCL support.

Even the cuDNN version and the path as shown in the output is not valid since i have cuDNN version 7.6.5 installed in C:/cuda
Step 4: bazel build //tensorflow/tools/pip_package:build_pip_package
This is where the problem originates:
Output:-
F:\tfrep\tensorflow>bazel build //tensorflow/tools/pip_package:build_pip_package
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=211
INFO: Reading rc options for 'build' from f:\tfrep\tensorflow\.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Options provided by the client:
  'build' options: --python_path=F:/python/python.exe
INFO: Reading rc options for 'build' from f:\tfrep\tensorflow\.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --java_toolchain=//third_party/toolchains/java:tf_java_toolchain --host_java_toolchain=//third_party/toolchains/java:tf_java_toolchain --define=tensorflow_enable_mlir_generated_gpu_kernels=0 --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --noincompatible_prohibit_aapt1 --enable_platform_specific_config --config=short_logs --config=v2
INFO: Reading rc options for 'build' from f:\tfrep\tensorflow\.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=F:/python/python.exe --action_env PYTHON_LIB_PATH=F:/python/lib/site-packages --python_path=F:/python/python.exe --config=xla --action_env CUDA_TOOLKIT_PATH=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=5.0 --config=cuda --define=override_eigen_strong_inline=true --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:short_logs in file f:\tfrep\tensorflow\.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file f:\tfrep\tensorflow\.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:xla in file f:\tfrep\tensorflow\.bazelrc: --define=with_xla_support=true
INFO: Found applicable config definition build:cuda in file f:\tfrep\tensorflow\.bazelrc: --config=using_cuda --define=using_cuda_nvcc=true
INFO: Found applicable config definition build:using_cuda in file f:\tfrep\tensorflow\.bazelrc: --define=using_cuda=true --action_env TF_NEED_CUDA=1 --crosstool_top=@local_config_cuda//crosstool:toolchain --define=tensorflow_enable_mlir_generated_gpu_kernels=1
INFO: Found applicable config definition build:windows in file f:\tfrep\tensorflow\.bazelrc: --copt=/w --copt=/D_USE_MATH_DEFINES --host_copt=/D_USE_MATH_DEFINES --cxxopt=/std:c++14 --host_cxxopt=/std:c++14 --config=monolithic --copt=-DWIN32_LEAN_AND_MEAN --host_copt=-DWIN32_LEAN_AND_MEAN --copt=-DNOGDI --host_copt=-DNOGDI --linkopt=/DEBUG --host_linkopt=/DEBUG --linkopt=/OPT:REF --host_linkopt=/OPT:REF --linkopt=/OPT:ICF --host_linkopt=/OPT:ICF --experimental_strict_action_env=true --verbose_failures --distinct_host_configuration=false
INFO: Found applicable config definition build:monolithic in file f:\tfrep\tensorflow\.bazelrc: --define framework_shared_object=false
INFO: Repository local_config_cuda instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule cuda_configure defined at:
  F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl:1407:18: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_config_cuda':
   Traceback (most recent call last):
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1377
                _create_local_cuda_repository(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1221, in _create_local_cuda_repository
                to_list_of_strings(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1222, in to_list_of_strings
                _cuda_include_path(<2 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 364, in _cuda_include_path
                inc_entries.append(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 364, in inc_entries.append
                realpath(repository_ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/remote_config/common.bzl", line 268, in realpath
                execute(repository_ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("C:\msys64\usr\bin" -c "realpath \"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/include\""): Access is denied.
 (error: 5)
ERROR: Skipping '//tensorflow/tools/pip_package:build_pip_package': no such package '@local_config_cuda//cuda': Traceback (most recent call last):
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1377
                _create_local_cuda_repository(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1221, in _create_local_cuda_repository
                to_list_of_strings(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1222, in to_list_of_strings
                _cuda_include_path(<2 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 364, in _cuda_include_path
                inc_entries.append(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 364, in inc_entries.append
                realpath(repository_ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/remote_config/common.bzl", line 268, in realpath
                execute(repository_ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("C:\msys64\usr\bin" -c "realpath \"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/include\""): Access is denied.
 (error: 5)
WARNING: Target pattern parsing failed.
ERROR: no such package '@local_config_cuda//cuda': Traceback (most recent call last):
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1377
                _create_local_cuda_repository(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1221, in _create_local_cuda_repository
                to_list_of_strings(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 1222, in to_list_of_strings
                _cuda_include_path(<2 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 364, in _cuda_include_path
                inc_entries.append(<1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/gpus/cuda_configure.bzl", line 364, in inc_entries.append
                realpath(repository_ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/remote_config/common.bzl", line 268, in realpath
                execute(repository_ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/remote_config/common.bzl", line 208, in execute
                fail(<1 more arguments>)
Repository command failed
java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("C:\msys64\usr\bin" -c "realpath \"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/include\""): Access is denied.
 (error: 5)
INFO: Elapsed time: 1.523s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: tensorflow/tools/pip_package

Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
Even if using no flag during configure.py for CUDA, the Access enied error originates again, although with a different error.
F:\tfrep\tensorflow>bazel build //tensorflow/tools/pip_package:build_pip_package
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=211
INFO: Reading rc options for 'build' from f:\tfrep\tensorflow\.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Options provided by the client:
  'build' options: --python_path=F:/python/python.exe
INFO: Reading rc options for 'build' from f:\tfrep\tensorflow\.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --java_toolchain=//third_party/toolchains/java:tf_java_toolchain --host_java_toolchain=//third_party/toolchains/java:tf_java_toolchain --define=tensorflow_enable_mlir_generated_gpu_kernels=0 --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --noincompatible_prohibit_aapt1 --enable_platform_specific_config --config=short_logs --config=v2
INFO: Reading rc options for 'build' from f:\tfrep\tensorflow\.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=F:/python/python.exe --action_env PYTHON_LIB_PATH=F:/python/lib/site-packages --python_path=F:/python/python.exe --config=xla --define=override_eigen_strong_inline=true --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:short_logs in file f:\tfrep\tensorflow\.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file f:\tfrep\tensorflow\.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:xla in file f:\tfrep\tensorflow\.bazelrc: --define=with_xla_support=true
INFO: Found applicable config definition build:windows in file f:\tfrep\tensorflow\.bazelrc: --copt=/w --copt=/D_USE_MATH_DEFINES --host_copt=/D_USE_MATH_DEFINES --cxxopt=/std:c++14 --host_cxxopt=/std:c++14 --config=monolithic --copt=-DWIN32_LEAN_AND_MEAN --host_copt=-DWIN32_LEAN_AND_MEAN --copt=-DNOGDI --host_copt=-DNOGDI --linkopt=/DEBUG --host_linkopt=/DEBUG --linkopt=/OPT:REF --host_linkopt=/OPT:REF --linkopt=/OPT:ICF --host_linkopt=/OPT:ICF --experimental_strict_action_env=true --verbose_failures --distinct_host_configuration=false
INFO: Found applicable config definition build:monolithic in file f:\tfrep\tensorflow\.bazelrc: --define framework_shared_object=false
INFO: Repository flatbuffers instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule third_party_http_archive defined at:
  F:/tfrep/tensorflow/third_party/repo.bzl:216:28: in <toplevel>
INFO: Repository 'flatbuffers' used the following cache hits instead of downloading the corresponding file.
 * Hash '62f2223fb9181d1d6338451375628975775f7522185266cd5296571ac152bc45' for https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/flatbuffers/archive/v1.12.0.tar.gz
If the definition of 'flatbuffers' was updated, verify that the hashes were also updated.
ERROR: An error occurred during the fetch of repository 'flatbuffers':
   Traceback (most recent call last):
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 193
                _apply_delete(ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 74, in _apply_delete
                _execute_and_check_ret_code(ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 52, in _execute_and_check_ret_code
                fail(<1 more arguments>)
Non-zero return code(256) when executing 'C:\msys64\usr\bin -l -c "rm" "-rf" "C:/users/nikhil/appdata/roaming/spb_data/_bazel_nikhil/5i2b27md/external/flatbuffers/build_defs.bzl"':
Stdout:
Stderr: java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("C:\msys64\usr\bin" -l -c "\"rm\" \"-rf\" \"C:/users/nikhil/appdata/roaming/spb_data/_bazel_nikhil/5i2b27md/external/flatbuffers/build_defs.bzl\""): Access is denied.
 (error: 5)
ERROR: F:/tfrep/tensorflow/tensorflow/tools/pip_package/BUILD:284:1: //tensorflow/tools/pip_package:build_pip_package depends on //tensorflow/lite/python:tflite_convert in repository @ which failed to fetch. no such package '@flatbuffers//': Traceback (most recent call last):
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 193
                _apply_delete(ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 74, in _apply_delete
                _execute_and_check_ret_code(ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 52, in _execute_and_check_ret_code
                fail(<1 more arguments>)
Non-zero return code(256) when executing 'C:\msys64\usr\bin -l -c "rm" "-rf" "C:/users/nikhil/appdata/roaming/spb_data/_bazel_nikhil/5i2b27md/external/flatbuffers/build_defs.bzl"':
Stdout:
Stderr: java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("C:\msys64\usr\bin" -l -c "\"rm\" \"-rf\" \"C:/users/nikhil/appdata/roaming/spb_data/_bazel_nikhil/5i2b27md/external/flatbuffers/build_defs.bzl\""): Access is denied.
 (error: 5)
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: no such package '@flatbuffers//': Traceback (most recent call last):
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 193
                _apply_delete(ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 74, in _apply_delete
                _execute_and_check_ret_code(ctx, <1 more arguments>)
        File "F:/tfrep/tensorflow/third_party/repo.bzl", line 52, in _execute_and_check_ret_code
                fail(<1 more arguments>)
Non-zero return code(256) when executing 'C:\msys64\usr\bin -l -c "rm" "-rf" "C:/users/nikhil/appdata/roaming/spb_data/_bazel_nikhil/5i2b27md/external/flatbuffers/build_defs.bzl"':
Stdout:
Stderr: java.io.IOException: ERROR: src/main/native/windows/process.cc(199): CreateProcessW("C:\msys64\usr\bin" -l -c "\"rm\" \"-rf\" \"C:/users/nikhil/appdata/roaming/spb_data/_bazel_nikhil/5i2b27md/external/flatbuffers/build_defs.bzl\""): Access is denied.
 (error: 5)
INFO: Elapsed time: 2.584s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (49 packages loaded, 14 targets configured)
    currently loading: tensorflow/lite/python

I have been working on TFDS for quite some time now and would like to work on TF as well. Therefore, I need to build the source ASAP. Thanks in advance!

Hey @Saduf2019, Thanks for the reply. After correcting the MSVC path variable and re-installing the Windows SDK, I was able to run the build, which took exceptionally long time (~6-7 hours on i7 7th gen 8GB laptop). In the end the bazel build got stuck at
[12638/12791] Linking tensorflow/python/_pywrap_tensorflow_internal.so 50000s
The seconds go on increasing but nothing happens.

@Saduf2019 One more thing that I wanted to ask was that how much time does the build supposed to take on a single cpu? and Am i supposed to build it every time new commits are done to the repo?

@Saduf2019 After i shut it down and re build it using the bazel command as mentioned in the instructions [here](bazel build //tensorflow/tools/pip_package:build_pip_package), it now got stuck at
C:\Users\nikhil\tensorflow>bazel build //tensorflow/tools/pip_package:build_pip_package
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=120
INFO: Reading rc options for 'build' from c:\users\nikhil\tensorflow\.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Options provided by the client:
  'build' options: --python_path=F:/python/python.exe
INFO: Reading rc options for 'build' from c:\users\nikhil\tensorflow\.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --java_toolchain=//third_party/toolchains/java:tf_java_toolchain --host_java_toolchain=//third_party/toolchains/java:tf_java_toolchain --define=tensorflow_enable_mlir_generated_gpu_kernels=0 --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --noincompatible_prohibit_aapt1 --enable_platform_specific_config --config=short_logs --config=v2
INFO: Reading rc options for 'build' from c:\users\nikhil\tensorflow\.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=F:/python/python.exe --action_env PYTHON_LIB_PATH=F:/python/lib/site-packages --python_path=F:/python/python.exe --config=xla --action_env CUDA_TOOLKIT_PATH=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=5.0 --config=cuda --define=override_eigen_strong_inline=true --action_env TF_CONFIGURE_IOS=0
INFO: Found applicable config definition build:short_logs in file c:\users\nikhil\tensorflow\.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file c:\users\nikhil\tensorflow\.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:xla in file c:\users\nikhil\tensorflow\.bazelrc: --define=with_xla_support=true
INFO: Found applicable config definition build:cuda in file c:\users\nikhil\tensorflow\.bazelrc: --config=using_cuda --define=using_cuda_nvcc=true
INFO: Found applicable config definition build:using_cuda in file c:\users\nikhil\tensorflow\.bazelrc: --define=using_cuda=true --action_env TF_NEED_CUDA=1 --crosstool_top=@local_config_cuda//crosstool:toolchain --define=tensorflow_enable_mlir_generated_gpu_kernels=1
INFO: Found applicable config definition build:windows in file c:\users\nikhil\tensorflow\.bazelrc: --copt=/w --copt=/D_USE_MATH_DEFINES --host_copt=/D_USE_MATH_DEFINES --cxxopt=/std:c++14 --host_cxxopt=/std:c++14 --config=monolithic --copt=-DWIN32_LEAN_AND_MEAN --host_copt=-DWIN32_LEAN_AND_MEAN --copt=-DNOGDI --host_copt=-DNOGDI --linkopt=/DEBUG --host_linkopt=/DEBUG --linkopt=/OPT:REF --host_linkopt=/OPT:REF --linkopt=/OPT:ICF --host_linkopt=/OPT:ICF --experimental_strict_action_env=true --verbose_failures --distinct_host_configuration=false
INFO: Found applicable config definition build:monolithic in file c:\users\nikhil\tensorflow\.bazelrc: --define framework_shared_object=false
INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[2 / 203] checking cached actions


Improve AccumulatorArgs after transition to all udafs

          The end state in my mind now.

pub struct AccumulatorArgs<'a> {
    /// Keep, this is return type, the name might be quite confusing.
    pub data_type: &'a DataType,

    /// We might only need one of `schema` or `dfschema`. It is likely we keep `dfschema`, since we can get `schema` from it.
    pub schema: &'a Schema,

    pub dfschema: &'a DFSchema,

    /// Keep
    pub ignore_nulls: bool,

    /// Convert to physical sort exprs instead
    pub sort_exprs: &'a [Expr],

    /// Keep
    pub is_reversed: bool,

    /// We might able to get the name from expressions
    pub name: &'a str,

    /// Keep
    pub is_distinct: bool,

    /// Get the type from schema and expressions
    pub input_type: &'a DataType,

    /// Convert to physical expressions
    pub input_exprs: &'a [Expr],
}

Originally posted by @jayzhan211 in https://github.com/apache/datafusion/issues/11666#issuecomment-2256183484

take

FYI @jayzhan211

There is #11739 that works on name by @lewiszlw , @xinlifoobar I think you can work on input_types


Angular JS fiddle link not working

Something not working as expected
In the JS fiddle example for angularJs export is not working. This due to library load issue. On the browser console the following error is shown:
404 Not found: http://alasql.org/console/xlsx.core.min.js
404 Not found: http://alasql.org/console/alasql.min.js

Ohh, thanks for letting us know. Do you have the link to the JS fiddle?

The examples on https://github.com/agershun/alasql/wiki/Angular.js have now been updated - Thank you for letting us know.


use new $*LANG.define_slang interface

This will update Slang::Tuxic to work with either the old or the new slang definition interface.

I confirm this works as a charm. Thank you. Please apply (plus a version increment)


could not compile dependency :ecto, "mix compile" failed

Elixir version
elixir 1.14.4-otp-24
Database and Version
16.3
Ecto Versions
ecto "~> 3.7.0" and ecto_sql "~> 3.7.1"
Database Adapter and Versions (postgrex, myxql, etc)
postgrex "~> 0.16.4" and myxql "~> 0.4.5"
Current behavior
`def project do
[
app: :automzero,
version: app_version(),
elixir: "~> 1.5",
elixirc_paths: elixirc_paths(Mix.env()),
compilers: [:phoenix] ++ Mix.compilers(),
start_permanent: Mix.env() == :prod,
aliases: aliases(),
deps: deps(),
releases: [
automzero: [
applications: [automzero: :permanent],
include_erts: true,
include_src: false,
steps: [:assemble, :tar]
]
]
]
end
defp deps do
[
# override because scrivener_html ~> 1.8 requires phoenix < 1.5.0
{:phoenix, "~> 1.6.0", override: true},
{:phoenix_pubsub, "~> 2.0"},
{:phoenix_ecto, "~> 4.4.3"},
{:ecto, "~> 3.7.0", override: true},
{:ecto_sql, "~> 3.7.1", override: true},
{:postgrex, ">= 0.0.0"},
{:phoenix_html, "~> 3.0", override: true},
{:phoenix_live_view, "~> 0.16.4"},
{:phoenix_live_reload,
git: "https://github.com/phoenixframework/phoenix_live_reload.git",
branch: "master",
only: :dev},
{:gettext, "~> 0.11"},
{:jason, "~> 1.0"},
{:plug_cowboy, "~> 2.3.0"},
{:plug_crypto, "~> 1.0", override: true},
{:scrivener_ecto, "~> 2.0"},
{:poison, "~> 3.1"},
{:phauxth, "~> 2.0"},
{:bcrypt_elixir, "~> 1.1"},
{:timex, "~> 3.7"},
{:thesis, "~> 0.3.4"},
{:hashids, "~> 2.0"},
{:distillery, "~> 2.0"},
{:canada, "~> 1.0.1"},
{:ecto_autoslug_field, "~> 2.0.1"},
{:httpoison, "~> 1.8"},
{:liquid, github: "steveops/liquid-elixir"},
{:pdf_generator, ">=0.6.0"},
{:scrivener_html, "~> 1.8"},
{:mox, "~> 0.5", only: :test},
{:honeydew, "~> 1.5.0"},
{:arc, "~> 0.11.0"},
{:flow, "~> 0.14"},
{:ex_machina, "~> 2.3", only: :test},
{:barlix, "~> 0.4.0"},
{:eqrcode, "~> 0.1.8"},
{:broadway, "~> 0.4.0"},
{:myxql, "~> 0.4.0"},
{:phoenix_live_dashboard, "~> 0.5"},
{:telemetry_metrics, "~> 0.6"},
{:telemetry_poller, "~> 0.5"},
{:number, "~> 1.0.1"},
{:recase, "~> 0.2"},
{:cachex, "~> 3.6"},
{:bureaucrat, "~> 0.2.7"},
{:corsica, "~> 1.0"},
{:money, "~> 1.4"},
{:bamboo, "~> 1.0", override: true},
{:bamboo_smtp, "~> 3.1.3"},
{:css_parser, ">= 0.1.0"},
{:phoenix_markdown, "~> 1.0"},
{:csv, "~> 2.4"},
{:xlsxir, "~> 1.6.4"},
{:erlsom, "~> 1.5"},
{:abacus, "~> 2.0.0"},
{:ex_crypto, "~> 0.10.0"},
{:sentry, "~> 8.0"},
{:oauth2, "~> 2.0"},
{:nimble_csv, "~> 1.2.0"}
]
end
`
mix deps.get
mix iex -S mix phx.server
Error i'm getting:

== Compilation error in file lib/ecto/query.ex ==
** (Kernel.TypespecError) lib/ecto/query.ex:428: type dynamic/0 is a built-in type and it cannot be redefined
(elixir 1.15.7) lib/kernel/typespec.ex:961: Kernel.Typespec.compile_error/2
(stdlib <IP_ADDRESS>) lists.erl:1599: :lists.foldl_1/3
(elixir 1.15.7) lib/kernel/typespec.ex:226: Kernel.Typespec.translate_typespecs_for_module/2
could not compile dependency :ecto, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile ecto --force", update it with "mix deps.update ecto" or clean it with "mix deps.clean ecto"

Expected behavior
Project to launch server but running into :ecto error.

Recent Erlang versions introduced a dynamic() type that conflicted with Ecto types. You need to update your Ecto.

but wouldn't it mean that earlier versions of Erlang would still compile. I'm using asdf to manage my project so as to carter for such conflicts.

Which Erlang version do you see when you run “iex”?

Erlang/OTP 24 [erts-<IP_ADDRESS>] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

It seems there is something wrong with your Erlang installation. Probably two different Erlangs were installed in the same directory.
you can see in your error message that the stdlib app in Erlang is 5.2, but the latest stdlib version for Erlang 24 is 3.x: https://github.com/erlang/otp/blob/maint-24/lib/stdlib/vsn.mk
which is why you see an error. Somewhere you have s more recent Erlang version.

when i run erl on root directory what is returned is

Erlang/OTP 26 [erts-<IP_ADDRESS>] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

which uses stdlib 5.2. But wheb i run it on project dir, it states it's using otp 24, i guess the asdf versions are being honoured. I'll look into it.


Prepare 2.16.0 release

Description

Resolves: FP-
Checklist

[ ] Update CHANGELOG.md

This might be a bit counterintuitive, but when 2.15 is released, main gets prepared for 2.16. So the title of the PR is correct, but the CHANGELOG entry isn't, because these changes were released in 2.15. I'll fix it in the next PR :+1:

This might be a bit counterintuitive, but when 2.15 is released, main gets prepared for 2.16. So the title of the PR is correct, but the CHANGELOG entry isn't, because these changes were released in 2.15. I'll fix it in the next PR 👍

This was released as part of v2.16: https://github.com/fiberplane/fp/releases/tag/v2.16.0


switch to UploadFileV2()

Hi!
Slack's files.upload API retires in March 2025.
https://api.slack.com/changelog/2024-04-a-better-way-to-upload-files-is-here-to-stay
switch from UploadFile() to UploadFileV2() method.

@fujiwara Thanks for your contribution!


Email validator error on valid email.

Hey guys.
I've been testing our app against possible email submissions (and some case insensitive checks but that's beyond the scope of this issue) and I noticed that some valid emails were being rejected by the email validator. The email set I used for testing come from the examples here: https://en.wikipedia.org/wiki/Email_address
Here are the emails that fail:
"much.more<EMAIL_ADDRESS>admin@mailserver1
"very.unusual<EMAIL_ADDRESS>"very.(),:;<>[]\".VERY.\"very@\\ <EMAIL_ADDRESS>"()<>[]:,;@\\\"!#$%&\'*+-/=?^_`{}|<EMAIL_ADDRESS>"<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

admin@mailserver1
"very.unusual<EMAIL_ADDRESS>
these are okay to reject because the first is a local address, you do not want your webserver try to resolve an internal name and try to deliver mails there.
second is delivering via a different server @ in the mailbox part is hardly allowed by any MTA.
allowing these is a security issue.
about the "..." syntax, while they may be valid, a normal user would not even understand what "" means in his email address so I am quite sure there is no mail provider that allows email addresses like that.

<EMAIL_ADDRESS><EMAIL_ADDRESS>
these are the only two I would consider to be important. They should work.

these are okay to reject because...

@cebe the point of the validator is to verify if the string meets the generic requirements to an abstract email address. It has nothing to do with host resolution, mail delivery or MTAs. What if we wand to develop an intranet web application and we want to validate local mailboxes? What is we use our custom MTA?

to develop an intranet web application and we want to validate local mailboxes

this would show a usecase for adding an extra flag to EmailValidator in which local emails will also be accepted

It's a popular thing to discuss :)

The point of the validator is to verify if the string meets the generic requirements to an abstract email address defined by RFC

The problem is that the most public email services does not allow to create fully RFC-compatible email address.
In case you validate email addresses according to RFC and have working website with at least 10k users, could you share statistics on how many users use emails that are rejected by EmailValidator?

I've added tests to confirm that üñîçøðé addresses work fine.
Right not I can't see any problems with local email addresses, especially when checkDNS property is set to false. In case I am wrong - please, open a separate issue with detailed description.
Thank you.


Removed redundant duplicate line of code

Presume it was a mistake, doesn't do any harm but is redundant code

The version field in meta stores the current database version to the underlying database. The Bolt data format isn't going to change anymore but it's useful for anyone who may have an old Bolt database when Bolt was still in alpha.

Maybe i'm missing something ? Hope i'm not wasting your time
I meant that line 256 and 258 in db.go are duplicates, line 258 is just re-setting the meta value with the same value again ? see https://github.com/boltdb/bolt/blob/master/db.go#L256-258
Thanks
Pat

Ah! Sorry about that. I misunderstood and read the code change too quickly. I didn't realize the lines were literally duplicated. I thought you meant setting the version wasn't necessary since it's already declared. lgtm. :+1: thanks!


Consider using cfg-if crate

Hello,
While reading through the code I noticed all the #[cfg(...)] statements, and I was wondering if you knew cfg-if existed? https://github.com/alexcrichton/cfg-if
This could help clean up the code and make sure that only 1 fn ever gets compiled in the code.
I'd be willing to implement this.

I was using cfg-if. I ripped it out very recently. Look at the recent commits.

👍


Support Exempt Balances

Fixes: #53

If Heimdall is stuck, please click the following link: https://heimdall-public.cbhq.net/v1/pull_request/stuck?pr_number=57&repository=coinbase/rosetta-specifications


Closing browser when Settings Sync starts leaves app in broken state

Turn on sync
Sign in
Close browser
Go back to app
:bug: no way to re-trigger sign in

Close browser

Which App are you referring to Code Insiders or Web?

@sandy081 I can repro this in Insiders, I imagine the same thing applies to web if you just close the tab that was opened to do auth.
I think the easiest fix is to not have a disabled "Turning on Preferences Sync..." menu item. There's a 5 minute timeout on MS auth after which it will reject, but that's a long time to wait for the UI to be enabled again.

Hmm good that you are able to repro, but I wonder how Turning on Preferences Sync... state is persisted across refreshes because this state is in memory and shall be cleared on refresh.
Can you provide me exact steps to repro on insiders?

Use "Turn on Preferences Sync" from the accounts menu
Choose a sign in option that's not for an existing account so that the browser opens
Close the browser window that opens (not vs code)

In VS Code, account menu is now stuck in disabled state and it's not possible to finish auth

Ah ok I understood now. Its closing the browser window that asks for login.

I think the easiest fix is to not have a disabled "Turning on Preferences Sync..." menu item.

I do not know if it is the right fix because it will allow user to turn on sync while sync is being turned on from the given window. Is it possible for the auth extension to return after closing the browser?
@misolori What do you suggest from UX point of view to show or not to show Turning on Preferences Sync... menu item?

Ok, it seems @misolori has a suggestion here - https://github.com/microsoft/vscode/issues/100411
I would go with toast notification for turning on sync progress and remove this menu item. Hence closing this as duplicate.

/duplicate

Actually I do not want to remove this Turning on Preferences Sync.. menu item under gear or accounts drop down. Instead I will add this item only after sign in successful. It means in this case (when you close browser window without signing in), you will see Turn on Preferences Sync item and you can turn on sync.


findOneAndUpdate & findOneAndDelete

I just wanted to check that the findOneAndUpdate & findOneAndDelete behaviour was intentional. I expected it to be similar to findOne where if no document satisfies the query then null is returned but it looks like a Mongo findAndModifyWriteOpResultObject is returned instead. Should I be doing something like this in my callbacks?
if (doc.ok && doc.lastErrorObject) {
    // doc not found
} else {
    // doc found
}

Many Thanks

You are right, it should probably return null


Proposal: Rocket Networking

Here is a proposal for Rocket networking model. It is not a spec -- it's too loosely worded but it should provide enough to generate a discussion. Feel free to comment on the Google Doc or here:
https://docs.google.com/a/coreos.com/document/d/1tK9HWYmG4_PJA3cS4FA_x9YUSG7STG3ijNFXndOdq8k

I created v2 document based on the feedback in the original proposal. Please see and comment on the new document: https://docs.google.com/document/d/1PUeV68q9muEmkHmRuW10HQ6cHgd4819_67pIxDRVNlM

Is there any known timeline when this proposal would likely be merged? My main interest is around experimenting integration with OVS and Open vSwitch Virtual Networking (OVN) project.

/cc @eyakubovich

@shettyg Current version of Rocket implements the proposal for the most part. I don't think it's locked in stone yet but there's enough there to start experimenting with writing plugins. Feel free to comment on the proposal as you experiment or ask questions about what's not clear.

The discussion has moved to https://github.com/appc/cni/ but most of the proposal has been implemented.


Multi view reconstruction

Dear author,
When i run the demo for multiview reconstruction, i get results for each view - 5 different meshes.
Is there a way that i can obtain the one final mesh for each subject?

Or more specifically is the last view's reconstruction the final result?

Dear author, When i run the demo for multiview reconstruction, i get results for each view - 5 different meshes. Is there a way that i can obtain the one final mesh for each subject?

I have the same question, thank you.

The model generates coeffs and displacement map for each view. You can average all the results to get the final displacement map and coeffs. Then by combining the final displacement map, coeffs and the shared deformation map, you can obtain a final mesh for each subject.

The model generates coeffs and displacement map for each view. You can average all the results to get the final displacement map and coeffs. Then by combining the final displacement map, coeffs and the shared deformation map, you can obtain a final mesh for each subject.

Hello, I am quite new to doing coding and I am not quite sure on how to averaging the results and combining the displacement map. May I ask how to do so? Because now what I can see is that the results generated from single-view and multi-view is the same. I am using photos of a same person from three angles, one from the front and two other sides. And I still get three seperate mesh when I use single-view mode or multi-view mode. Thank you so much for the help.

The model generates coeffs and displacement map for each view. You can average all the results to get the final displacement map and coeffs. Then by combining the final displacement map, coeffs and the shared deformation map, you can obtain a final mesh for each subject.

Hello, I am quite new to doing coding and I am not quite sure on how to averaging the results and combining the displacement map. May I ask how to do so? Because now what I can see is that the results generated from single-view and multi-view is the same. I am using photos of a same person from three angles, one from the front and two other sides. And I still get three seperate mesh when I use single-view mode or multi-view mode. Thank you so much for the help.

Did you could solve it ?

The model generates coeffs and displacement map for each view. You can average all the results to get the final displacement map and coeffs. Then by combining the final displacement map, coeffs and the shared deformation map, you can obtain a final mesh for each subject.

Hello, I am quite new to doing coding and I am not quite sure on how to averaging the results and combining the displacement map. May I ask how to do so? Because now what I can see is that the results generated from single-view and multi-view is the same. I am using photos of a same person from three angles, one from the front and two other sides. And I still get three seperate mesh when I use single-view mode or multi-view mode. Thank you so much for the help.

Did you could solve it ?

Unfortunately not yet


Add/subtract obsolete ? Or config for issue suppresion

Hi Lars,
Thanks for the tool, it looks really interesting. :)
Why does the tool claim that add/subtract operators are obsolete. It is not mentioned in abap help. Besides add is a great alternative to increment, it's more readable and safer than a=a+1 form.
Anyway, if kept like this, would it be possible to implement some kind of linter config to be able to supress certain errors?

see http://help.sap.com/abapdocu_740/en/abencalc_expresssion_guidl.htm
Its part of the big plan to have everything configurable, havent had much time lately to work on this project
have you also seen my other project https://github.com/larshp/abapOpenChecks it runs inside the system and can be cofigured

Thanks for the link! It is a pity they wouldn't introduce things like "+=" that would really help.
OpenChecks are great. I'm considering using them for my projects actually. Thanks ! As for abaplint I found the idea of CI/codecov badges awesome that's why I'm asking.

:smile:
the coverage is uploaded using https://github.com/larshp/abapCov

Yes, I know, I've seen :) BTW Would you mind if I make a 731 version (in my rep) ?

P.S. Porting codecov to 731 failed :(  cl_aucv_test_runner_coverage was introduces in 7.40.

no, feel free. But also feel free to submit the backport as a pull request

closing this issue
for the backport of abapCov I've created https://github.com/larshp/abapCov/issues/6 as a reminder, I'll probably not be looking into it anytime soon


[Trainer] Changing the dataset dynamically during training

Hello,
I am currently training a model using DPO, and I'm adapting the dataset dynamically during training. My current approach looks like this:
trainer = DPOTrainer(
    model,
    None,
    args=training_args,
    train_dataset=dataset,
    tokenizer=tokenizer,
    peft_config=peft_config,
    beta=args.beta,
    max_prompt_length=1024,
    max_length=1536,
)
for i in range(repetitions):
    train_result = trainer.train()
    # Adapt the dataset based on the training result
    dataset = get_adapted_dataset(train_result)
    with PartialState().local_main_process_first():
        # Tokenize the updated dataset
        print("Updating the training dataset")
        trainer.train_dataset = dataset.map(trainer.tokenize_row, num_proc=None)

Is this the correct way to adapt the dataset during training, or is there a more appropriate approach for this scenario?

Using an iterable dataset might be more suited. If the way you update the dataset depends on the results, you'll probably need to set a callback as well


stroke/strokeWidth doesn't work for SvgUri or SvgXml on iOS?

NB Never post screenshots of code. Post copy paste-able code if you include any.
Ignoring this will likely lead to direct closing of the issue.
Please respect the time and energy of open source maintainers.
Bug
I'm trying to style an svg with no in-line styles and I can't get it to render on iOS using the latest version of react-native-svg. Manually adding styles to the xml works fine. It is broken for both SvgXml and SvgUri. I have included links to expo snacks that show the bug.
Unexpected behavior
broken version in snack (SvgXml seems to ignore stroke and strokeWidth): https://snack.expo.io/W7ygrQOne --> broken in iOS, works in android
working version in snack (putting stroke and stroke-width inline : https://snack.expo.io/qptVLgipr --> running this in the iOS env

Interesting, seems to work in plain react-native with the latest version at least.

If you need it to work in expo, one workaround might be to use https://react-svgr.com/playground/ and move the spreading of props to a G element instead of the Svg root like this:
import * as React from 'react';
import Svg, { G, Path } from 'react-native-svg';

function SvgComponent(props) {
  return (
    <Svg viewBox="0 0 511 511">
      <G {...props}>
        <Path d="M131.5 96c-11.537 0-21.955 8.129-29.336 22.891C95.61 132 92 149.263 92 167.5s3.61 35.5 10.164 48.609C109.545 230.871 119.964 239 131.5 239s21.955-8.129 29.336-22.891C167.39 203 171 185.737 171 167.5s-3.61-35.5-10.164-48.609C153.455 104.129 143.037 96 131.5 96zm15.92 113.401C142.78 218.679 136.978 224 131.5 224s-11.28-5.321-15.919-14.599C110.048 198.334 107 183.453 107 167.5s3.047-30.834 8.581-41.901C120.22 116.321 126.022 111 131.5 111s11.28 5.321 15.919 14.599C152.953 136.666 156 151.547 156 167.5s-3.047 30.834-8.58 41.901z" />
        <Path d="M474.852 158.011c-1.263-40.427-10.58-78.216-26.555-107.262C430.298 18.023 405.865 0 379.5 0h-248c-26.365 0-50.798 18.023-68.797 50.749C45.484 82.057 36 123.52 36 167.5s9.483 85.443 26.703 116.751C80.702 316.977 105.135 335 131.5 335a57.57 57.57 0 005.867-.312 7.51 7.51 0 002.133.312h48a7.5 7.5 0 000-15h-16c10.686-8.524 20.436-20.547 28.797-35.749 4.423-8.041 8.331-16.756 11.703-26.007V503.5a7.501 7.501 0 0011.569 6.3l20.704-13.373 20.716 13.374a7.498 7.498 0 008.134 0l20.729-13.376 20.729 13.376a7.49 7.49 0 004.066 1.198c1.416 0 2.832-.4 4.07-1.2l20.699-13.372 20.726 13.374a7.5 7.5 0 008.133 0l20.732-13.377 20.738 13.377a7.5 7.5 0 008.126.003l20.783-13.385 20.783 13.385a7.5 7.5 0 0011.561-6.305v-344a7.377 7.377 0 00-.146-1.488zM187.154 277.023C171.911 304.737 152.146 320 131.5 320s-40.411-15.263-55.654-42.977C59.824 247.891 51 208.995 51 167.5s8.824-80.391 24.846-109.523C91.09 30.263 110.854 15 131.5 15s40.411 15.263 55.654 42.977C203.176 87.109 212 126.005 212 167.5s-8.824 80.391-24.846 109.523zm259.563 204.171a7.5 7.5 0 00-8.122 0l-20.78 13.383-20.742-13.38a7.5 7.5 0 00-8.131 0l-20.732 13.376-20.729-13.376a7.497 7.497 0 00-8.136.002l-20.699 13.373-20.727-13.375a7.498 7.498 0 00-8.133 0l-20.728 13.375-20.718-13.375a7.499 7.499 0 00-8.137.001L227 489.728V271h8.5a7.5 7.5 0 000-15H227v-96.5c0-.521-.054-1.03-.155-1.521-1.267-40.416-10.577-78.192-26.548-107.231C191.936 35.547 182.186 23.524 171.5 15h208c20.646 0 40.411 15.263 55.654 42.977C451.176 87.109 460 126.005 460 167.5V256h-.5a7.5 7.5 0 000 15h.5v218.749l-13.283-8.555z" />
        <Path d="M283.5 256h-16a7.5 7.5 0 000 15h16a7.5 7.5 0 000-15zM331.5 256h-16a7.5 7.5 0 000 15h16a7.5 7.5 0 000-15zM379.5 256h-16a7.5 7.5 0 000 15h16a7.5 7.5 0 000-15zM427.5 256h-16a7.5 7.5 0 000 15h16a7.5 7.5 0 000-15z" />
      </G>
    </Svg>
  );
}

export default function App() {
  return (
    <SvgComponent width="100%" height="100%" strokeWidth={5} stroke="black" />
  );
}

re: wei

Interesting, seems to work in plain react-native with the latest version at least.

Thanks for the reply @msand
re: translating to spreading props, I was hoping to store svgs in the cloud / in blobs, so I'm still hoping there's a way I can make it work from svg strings.
re: not working in expo - is that something to debug in react-native-svg or something I should file a bug against expo for?

I guess expo knows it's incompatible with v12 :-/
Some of your project's dependencies are not compatible with currently installed expo package version:

react-native-screens - expected version range: ~2.2.0 - actual version installed: ^2.4.0
react-native-svg - expected version range: 11.0.1 - actual version installed: ^12.0.3
@react-native-community/masked-view - expected version range: 0.1.6 - actual version installed: ^0.1.7
Your project may not work correctly until you install the correct versions of the packages.
To install the correct versions of these packages, please run: expo install [package-name ...]

This issue looks like a duplicate of https://github.com/software-mansion/react-native-svg/issues/1543. I tested the code you sent with the fix provided in that issue and it seems to resolve the problem. I will close this issue then. Feel free to comment here if something is wrong and we can always reopen it then.

Issue For Me Is I Use Figma And When I Download SVG
its  stroke Width is Like this  stroke-width={5}
Solution
Simply Change it To  strokeWidth={5}
It Worked For me


packer: command not found

i'm working on windows 11
got error  packer: command not found !

you have to install the dependencies as described in the readme.


Dry run doesn't always return correct height

The dry run logic doesn't always return the correct height if the content exceeds the height of a page. The problem has to do with page boundaries.
When we want to pre-calculate the height of a document fragment, we move to a scratch document, clone the page settings, and write into the document. We then measure the distance from where the cursor started to where the cursor ended, taking into account page traversals.
However, when we do this, we always start the cursor at the top of the page in the scratch document, regardless of where the cursor is in the main document. When we go back to write the content in the main document, we often end up with a different result because of where the content falls in relation to the page boundaries.
The fix seems simple. Just start at the same position in the scratch document that we do in the main document. The problem is, we sometimes want to know if the content will fit on a single page. In that case, we need to start at the top of the document so we get a clean measurement. This is necessary for the "keep together" logic.
If we're doing "keep together" logic, then we first need to start at the top of the document and write until we hit the end of the page. We need to intercept this event and stop writing. We know we can't fit in a single page. We then need to back up, move the cursor to the location it is in the main document and start again, this time proceeding until we finish writing. We return the height and a flag that says we can't fit on a single page.
If we aren't doing "keep together" logic, then there is no reason to do the first step. We can just move the cursor to the same location it is in the main document, write all the content, then measure.
If we rewrite the logic as described, we'll always get accurate measurements. We'll also have the ability to use dry run for either "keep together" logic or to simply pre-calculate the height of content. This will enable features like #578.
In short, here's the logic for keep together:

switch to draft document
move cursor to top of new page
configure document to stop when advancing to next page (perhaps override on_page_create and throw exception)
if content fits on one page

return height and a flag that content fits on one page

if content exceeds one page

remove the hook to stop when advancing to the next page
move cursor to y offset from main document
start run again, but proceed until all content is written
calculate height across pages
return height and flag that content does not fit on one page

Here's the logic when not using keep together:

switch to draft document
move cursor to y offset from main document
write all content
calculate height across pages
return height (perhaps include flag that indicates whether content ran over page, but remember we didn't start from the top)

NOTE: We might be able to avoid a dry run in obvious cases; a kind of engineering estimate.

Here's another point to consider. When Asciidoctor PDF was first released, it couldn't draw a background behind a block that split across pages. So we applied "keep together" logic to most panel-like blocks (listing, sidebar, example, etc). That's no longer necessary. We could return to splitting blocks across pages and only using "keep together" when requested.

Hey, I promised to look at this a while ago, but then life happened and I got pulled off to a bunch of other tasks (I'm sure you can relate :)). So I've read through what you've written above, and it all makes sense to me - certainly it matches up to what I was thinking previously. Also very much +1 to your last comment.
There is a slight wrinkle in the ointment which could cause an issue, possibly, though it could be deliberately not handled in the first iteration of this.
Let's say we fix #578, and make "keep_together" configurable for all blocks. Then someone does a "keep together" open block, and within that, another keep_together block (doesn't matter what).
The behaviour I'd like to see in that case is that first the whole lot tries to fit onto a single page, and if it can't, you then try to see if the sub-element can fit onto a single page (and so on and so forth - hopefully that makes sense?).
That maybe is something we can punt on for a while, but whenever someone tries to fix this, it'd be nice to at least have the notion of doing this in mind. It might just fall out of how it's coded.

I'm in the process of working on this. Here's the solution (assuming keep together logic):

In dry run document, start at current y and convert the block; capture the start and end location of the block into a range object
If the range is split across pages, retry from the top of page to recompute range
If the range still gets split, go back and use the original range (no point in starting on new page when it will get split anyway)
In main document, advance to new page if block can fit on a single page without being split
Draw boundaries based on range (instead of block height)

This issue has been replaced by #2003 which summarizes the new logic being implemented to compute the extent of a block's boundaries.


Add Pthread backend and check for empty list of backends

Not checking for the device types we support gave me some pretty hard to understand errors.

I wanted to try another parallel backend that might be simpler to debug than OpenMP and it seems to work out of the box. Any reason not to support it?

Any reason not to support it?

No strong reason. Pthread is uncommon, though, and is rarely enabled, I believe.

If you want to support, enable on one CI build at least.

There was some error with fetching the gpg-key (for CMake I believe) and I fixed a typo I introduced at some point in the .jenkins file for the CMake options.

Let's just go with the simplified version in #190.


fix: 修复 line 包围盒问题

👀 PR includes

✨ Feature

[ ] New feature

🎨 Enhance

[ ] Code style optimization
[ ] Refactoring
[ ] Change the UI
[ ] Improve the performance
[ ] Type optimization

🐛 Bugfix

[x] Solve the issue and close #1561

🔧 Chore

[ ] Test case
[ ] Docs / demos update
[ ] CI / workflow
[ ] Release version
[ ] Other ()

📝 Description
grid line 在绘制时，包围盒计算有点问题，会代入lineWidth
比如传入的 x1=0, x2=10, lineWidth=20
最后line得出来的包围盒 minX=-10, maxX=20，会将lineWidth/2纳入计算中
最后就会导致更新过程中，GridGroup的包围盒不断被放大，并且只会出现在 lineWidth >=2时（猜测是 lineWith=1 时，lineWidth/2会向下取整为0了，当好负负得正）
因此在传入时，将这部分坐标减去，并结合lineCap将这部分绘制出来，达到内容区域绘制不变，包围盒计算正确的目的
🖼 Screenshot

Before
After

🔗 Related issue link

🔍 Self-Check before the merge

[ ] Add or update relevant docs.
[ ] Add or update relevant demos.
[ ] Add or update test case.
[ ] Add or update relevant TypeScript definitions.

:tada: This PR is included in version @antv/s2-v1.24.0-alpha.2 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @antv/s2-v1.24.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Updated the Alpine baseimage to 3.19

A lot of packages like ffmpeg or chromium are now outdated for the deprecated Alpine 3.16.

Why not update to 3.20 instead ?

Because in https://github.com/jlesage/docker-jdownloader-2/issues/212 skycaptain only tested with 3.19. I didn't test 3.20, if it's usable it's of course better to use the newest image.

Image is now based on Alpine 3.20.


Difficulty creating signer

Hello! I'm trying to sign transactions on the Polygon Mumbai Testnet. This is the error i'm getting

Steps I've taken:

Uploaded wallet private key to Google Cloud KMS
Configured Goth process named GCPCredentials and verified I get a valid response on Goth.fetch(GCPCredentials)
Added the following config to the bottom of runtime.exs, replacing the necessary values from step 1
config :signet, :signer, [
{MySigner, {:cloud_kms, GCPCredentials, "projects/{project}/locations/{location}/keyRings/{keyring}/cryptoKeys/{keyid}" "1"}}]

It seems i'm missing something simple getting the genserver process started or someting. Please let me know of any other info I need to provide.

Can you add {:google_api_cloud_kms, "~> 0.38.1"}, to your mix.exs. The CloudKMS module doesn't load unless you include this optional lib. We should not this in the docs.

Ah, thank you @hayesgm. That eliminated the error I had! I should've thought of searching for a package, the error makes sense now.
Do you have an example or some guidance on how to properly add a wallet private key from something like Metamask to GCloud KMS? I've created a Project, Keyring, and a key with:

HSM protection level
Asymmetric sign purpose
Elliptic Curve secp245k1 key SHA256 Digest default algorithm

At this point, it looks like I need to make an upload job, format the key for import, then wrap the key using the provided wrapping key.
I really like how the library is designed and have enjoyed using it so far. When I get things done, I'll be happy to write a PR adding some setup instructions to the docs.
Thanks in advance!

Ok @hayesgm, I think I was able to get the key properly into GCP KMS... Happy to post my steps I've recorded once I validate them. Now, i'm running into a new error. It looks like its trying to pad something that doesn't need to be padded? The address it is signing with is correct which makes me think I have gotten the key uploaded properly. Any ideas?

@hayesgm Just wanted to bump this to see if you have any ideas as to why this is happening

@hayesgm Just wanted to bump this to see if you have any ideas as to why this is happening

See my PR, this issue happened when v is greater than 0xff;

@hayesgm This PR solved it for me when I forked the repo and integrated it
https://github.com/hayesgm/signet/pull/18


Implement drop-in replacement of Express.js with 6-x performance difference?

Hi users & stargazers!
I'd love to hear your ideas and agreements.
Also, for this feature (drop-in replacement), i am need two or three developer with free time and/or support for development hours (contact me for more information at dalisoft@mail.ru).
I am think this idea like nanoexpress Pro which is differs by performance, features and support.
If you like that, please leave "Emoji 👍".
Below you can see at which cost what you can get
Simple

Free
4x faster than Express (not in all-cases)
2x faster than Express (for Async routes)
Serialization
Latest Dependencies
Apache Licensed

Pro

10$/month (100$/yearly)
6x faster than Express (in most-cases)
5x faster than Express (for Async routes)
Lighter size
More polyfilled methods
Express.Route polyfill
Express.Static polyfill
Less bugs and errors
Much Stable Performance / RPS
Validation/Serialization out-of-the-box
Auto Swagger/OpenAPI documentation
Latest dependencies
Commercial license (license will be valid until you stop paying) for closed-source project per person, ONE person can use this if purchased, other can't if other not purchased.

Note: If you not agree, i will continue support this project.
Thank you.
I hope you understand me.
Update 1: You can try PRO in your open-source application
Update 2: Changing plans
Update 3: Update
Update 4: Change Commercial license term and remove support hour (who applied already until 19 August 2019 18:50:00 GMT+5, support hours will be applied for your purchased period of license)
Update 5: Typo fix

Ow, i see, you're don't want premium versions.
I'll close this issue and keep support current version

This library still in Alpha. I think it's too early to talking about paid versions.

@aprilcoskun You may be right, but i don't see any reason to don't make this way. I keep this project in Alpha, because there may be some bugs (tested, but not fully).
I want create 4 type of product which (2 of them already done at 50-60% locally).
Thanks for tip!

@aprilcoskun Can you try PRO and leave feedback?

This issue will be closed as i'll refered here


Schedule quarterly tasks

At the moment, I am doing something akin to:
      $schedule->command('foo_report:generate')
                 ->cron('0 0 1 */3 *');

to generate a quarterly task within the ConsoleKernel.
I think this would look syntactically nicer as:
      $schedule->command('foo_report:generate')
                 ->quarterly();

thoughts?

I don't see this as more than:
    //src/Illuminate/Console/Scheduling/Event.php
    /**
     * Schedule the event to run quarterly.
     *
     * @return $this
     */
    public function quarterly()
    {
        return $this->cron('0 0 1 */3 *');
    }

Thanks for the PR.


fix(clerk-js): Revalidate when user deletes organization

Description
We also need to revalidate when user creates an organization, and when user is leaving an organization

Checklist

[ ] npm test runs as expected.
[ ] npm run build runs as expected.
[ ] (If applicable) JSDoc comments have been added or updated for any package exports
[ ] (If applicable) Documentation has been updated

Type of change

[ ] 🐛 Bug fix
[ ] 🌟 New feature
[ ] 🔨 Breaking change
[ ] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

Packages affected

[ ] @clerk/backend
[ ] @clerk/chrome-extension
[ ] @clerk/clerk-js
[ ] @clerk/clerk-expo
[ ] @clerk/fastify
[ ] gatsby-plugin-clerk
[ ] @clerk/localizations
[ ] @clerk/nextjs
[ ] @clerk/clerk-react
[ ] @clerk/remix
[ ] @clerk/clerk-sdk-node
[ ] @clerk/shared
[ ] @clerk/themes
[ ] @clerk/types
[ ] build/tooling/chore

!preview

Hey @octoper, your preview is currently deploying and should be available shortly.

Status
Preview
Updated (UTC)

:cook: Deploying...
Inspect
Jan 03, 2024 07:38 PM


Simpler GPU Dask Comm benchmark

This uses only Dask comm's machinery rather than the rest of Dask.  Hopefully it's simpler to use.
import asyncio
import argparse
from time import perf_counter as clock

import numpy as np
import ucp
import cupy
from tornado.ioloop import IOLoop
import distributed.comm
from distributed.protocol import Serialized, serialize
from distributed.utils import parse_bytes

import base

def parse_args(args=None):
    parser = argparse.ArgumentParser(parents=[base.protocol])
    parser.add_argument("-b", "--n-bytes", type=parse_bytes,
                        default="100 Mb",
                        help="Message size, default '100 Mb'.")
    parser.add_argument("-n", "--n-iterations", type=int,
                        default=10,
                        help="Message size, default '100 Mb'.")
    parser.add_argument('--server', help='are we the server or client',
            action='store_true')

    return parser.parse_args(args)

async def server_handle_comm(comm):
    while True:
        msg = await comm.read()
        if msg['op'] == 'close':
            return
        assert msg['op'] == 'ping'
        msg['op'] = 'pong'
        await comm.write(msg)

async def listen(address):
    listener = distributed.comm.listen(address, server_handle_comm, deserialize=False)
    listener.start()
    while True:
        await asyncio.sleep(1)

async def client(address, nbytes, n_iterations):
    comm = await distributed.comm.connect(address, deserialize=False)
    start = clock()
    data = cupy.random.randint(0, 255, size=nbytes, dtype=np.uint8)
    item = Serialized(*serialize(data))

    for i in range(n_iterations):
        await comm.write({'op': 'ping', 'item': item})
        msg = await comm.read()
        assert msg['op'] == 'pong'
        assert isinstance(msg['item'], Serialized)
        print('.', end='', flush=True)

    print()
    end = clock()
    await comm.write({'op': 'close'})
    await comm.close()

    dt = end - start
    rate = len(data) * n_iterations/ dt
    print("duration: %s => rate: %d MB/s"
          % (dt, rate / 1e6))

async def main(args=None):
    args = parse_args(args)
    if args.protocol == 'tcp':
        addr = 'tcp://<IP_ADDRESS>:8989'
    else:
        addr = 'ucx://' + ucp.get_address() + ':8989'
    if args.server:
        await listen(addr)
    else:
        await client(addr, args.n_bytes, args.n_iterations)

if __name__ == '__main__':
    asyncio.get_event_loop().run_until_complete(main())

Run with
(along with any other environment variables necessary)
CUDA_VISIBLE_DEVICES=0 python ucx-comm-gpu.py --server --protocol ucx
CUDA_VISIBLE_DEVICES=1 python ucx-comm-gpu.py --protocol ucx --n-bytes "100 MiB"  -n 20

I get around 1-3 GB/s, depending on how long I run this for.  (I suspect that the first communication may take a bit longer?)
I also occasionally get a failure like the following:
[1557070494.459012] [dgx15:12230:0]         ucp_mm.c:111  UCX  ERROR failed to register address (nil) length 104857600 on md[1]=ib/mlx5_0: Invalid parameter
[1557070494.459032] [dgx15:12230:0]    ucp_request.c:264  UCX  ERROR failed to register user buffer datatype 0x8 address (nil) len 104857600: Invalid parameter
[dgx15:12230:0:12230] Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))
==== backtrace ====
    0  /home/nfs/mrocklin/miniconda/envs/ucx-gpu/lib/libucs.so.0(+0x24c0c) [0x7fc15d9bdc0c]
    1  /home/nfs/mrocklin/miniconda/envs/ucx-gpu/lib/libucs.so.0(+0x24e84) [0x7fc15d9bde84]
    2  /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7fc16d04e390]
    3  /lib/x86_64-linux-gnu/libc.so.6(+0x14e156) [0x7fc16cdc1156]
    4  /home/nfs/mrocklin/miniconda/envs/ucx-gpu/lib/libucp.so.0(ucp_rndv_data_handler+0xf1) [0x7fc15e05af11]
    5  /home/nfs/mrocklin/miniconda/envs/ucx-gpu/lib/ucx/libuct_ib.so.0(uct_rc_mlx5_iface_progress+0x120) [0x7fc11efed560]
    6  /home/nfs/mrocklin/miniconda/envs/ucx-gpu/lib/libucp.so.0(ucp_worker_progress+0x6a) [0x7fc15e04040a]
    7  /home/nfs/mrocklin/ucx-py/ucp/_libs/ucp_py.cpython-37m-x86_64-linux-gnu.so(+0xad43c) [0x7fc15e34543c]
    8  /home/nfs/mrocklin/ucx-py/ucp/_libs/ucp_py.cpython-37m-x86_64-linux-gnu.so(+0x73e39) [0x7fc15e30be39]
    9  python(_PyMethodDef_RawFastCallKeywords+0x141) [0x5592193815c1]
   10  python(_PyCFunction_FastCallKeywords+0x21) [0x559219381801]
   11  python(+0x20abc3) [0x559219412bc3]
   12  python(_PyMethodDef_RawFastCallDict+0x199) [0x5592193448b9]
   13  python(_PyCFunction_FastCallDict+0x21) [0x559219344ae1]
   14  python(_PyEval_EvalFrameDefault+0x5d21) [0x5592193de151]
   15  python(_PyFunction_FastCallKeywords+0xfb) [0x55921938079b]
   16  python(_PyEval_EvalFrameDefault+0x6a0) [0x5592193d8ad0]
   17  python(_PyFunction_FastCallKeywords+0xfb) [0x55921938079b]

OK, I'm able to get 8GB/s one way using Dask with the following setup
dask-scheduler --host ucx://<IP_ADDRESS>  # replace with your ib0 address
CUDA_VISIBLE_DEVICES=0 dask-worker ucx://<IP_ADDRESS>:8786 --no-nanny --dashboard-address <IP_ADDRESS>:9000
CUDA_VISIBLE_DEVICES=1 python ucx-client.py

# ucx-client.py
from dask.distributed import Client
import asyncio
import cupy
from time import time
from distributed.utils import format_bytes

niter = 10
nbytes = 200_000_000

async def f():
    x = cupy.random.randint(0, 255, size=nbytes, dtype='uint8')
    async with Client('ucx://<IP_ADDRESS>:8786', asynchronous=True,
            direct_to_workers=True) as client:
        await (await client.scatter(x, hash=False, direct=True))
        start = time()
        for i in range(niter):
            future = await client.scatter(x, hash=False, direct=True)
            resp = await client.gather(future, direct=True)
        stop = time()
        print(format_bytes(2 * x.nbytes * niter / (stop - start)), '/s')

if __name__ == '__main__':
    asyncio.get_event_loop().run_until_complete(f())

However, if I change away from devices 0 and 1 performance degrades quickly.  Sending 200MB takes anywhere from 20ms to 1s depending on the source and destination device.  I understand that there is some structure to the network of a DGX, but the magnitude of difference here makes me think that there is something else going on.

I can replicate this without Dask using the recv-into-client benchmark
(ucx-gpu) mrocklin@dgx15:~/ucx-py$ UCX_MEMTYPE_CACHE=n UCX_RNDV_SCHEME=put_zcopy UCX_TLS=rc,cuda_copy UCX_NET_DEVICES=mlx5_0:1 CUDA_VISIBLE_DEVICES=0 python benchmarks/recv-into-client.py -s <IP_ADDRESS> -p 13337 -o cupy
Roundtrip benchmark
-------------------
n_iter   | 10
n_bytes  | 10.00 MB
recv     | recv_into
object   | cupy
inc      | False

===================
6.90 GB / s
===================
(ucx-gpu) mrocklin@dgx15:~/ucx-py$
(ucx-gpu) mrocklin@dgx15:~/ucx-py$ UCX_MEMTYPE_CACHE=n UCX_RNDV_SCHEME=put_zcopy UCX_TLS=rc,cuda_copy UCX_NET_DEVICES=mlx5_0:1 CUDA_VISIBLE_DEVICES=7 python benchmarks/recv-into-client.py -s <IP_ADDRESS> -p 13337 -o cupy
Roundtrip benchmark
-------------------
n_iter   | 10
n_bytes  | 10.00 MB
recv     | recv_into
object   | cupy
inc      | False

===================
359.28 MB / s
===================

Do we know if this is this likely to be a hardware issue or a software issue?

Hrm, the assymetric performance that prefers devices 0 and 1 is possibly consistent with this image: https://docs.nvidia.com/dgx/bp-dgx/index.html#networking

Perhaps with our addressing we're somehow forcing everyone to go through ib0?

@mrocklin Thanks for sharing that figure - I noticed that the performance of the recv-into-client.py benchmark is dramatically improved for the second case (with CUDA_VISIBLE_DEVICES=7 ) if you include both UCX_NET_DEVICES in the client-side command: UCX_NET_DEVICES=mlx5_0:1,mlx5_3:1

Ah, that's interesting.  There are a variety of environment variables that we're depending on.  It sounds like we're at the point where we might need to know what they mean :)

I should note that I'm running this benchmark in a somewhat custom setup.
I'm running from with @TomAugspurger 's repository https://github.com/TomAugspurger/dask-perf
I'm also using a version of @quasiben's branch https://github.com/dask/distributed/pull/2591 but with a few recently merged PRs (make sure to pull or reset as of 20 seconds ago)

Just in case it is useful to anyone else... The UCX_NET_DEVICES variable does have a significant effect on the ucx-client.py benchmark:
Starting Scheduler on ib1:
UCX_NET_DEVICES=mlx5_1:1 dask-scheduler --host ucx://<IP_ADDRESS>

Starting Worker on ib1:
CUDA_VISIBLE_DEVICES=2 UCX_NET_DEVICES=mlx5_1:1 dask-worker ucx://<IP_ADDRESS>:8786 --no-nanny

Running ucx-client.py on ib1:
$ CUDA_VISIBLE_DEVICES=3 UCX_NET_DEVICES=mlx5_1:1 python ucx-client.py
7.57 GB /s

Running ucx-client.py on ib2:
$ CUDA_VISIBLE_DEVICES=4 UCX_NET_DEVICES=mlx5_1:1,mlx5_2:1 python ucx-client.py
7.78 GB /s

Running ucx-client.py on ib3:
$ CUDA_VISIBLE_DEVICES=6 UCX_NET_DEVICES=mlx5_1:1,mlx5_3:1 python ucx-client.py
7.73 GB /s

When I don't include UCX_NET_DEVICES in the command, I get ~330MB/s when I run ucx-client.py with CUDA_VISIBLE_DEVICES > 3.

How should we deploy workers on a DGX?  We will have eight dask-worker
processes, each attached to a particular GPU.  What environment variables
should each have?  How should they be started?  (in particular, should they
bind to a different network interface)
On Mon, May 6, 2019 at 2:25 PM (Rick) Richard J Zamora <
<EMAIL_ADDRESS>wrote:

Just in case it is useful to anyone else... The UCX_NET_DEVICES variable
does have a significant effect on the ucx-client.py benchmark:
Starting Scheduler on ib1:
UCX_NET_DEVICES=mlx5_1:1 dask-scheduler --host ucx://<IP_ADDRESS>
Starting Worker on ib1:
CUDA_VISIBLE_DEVICES=2 UCX_NET_DEVICES=mlx5_1:1 dask-worker ucx://<IP_ADDRESS>:8786 --no-nanny
Running ucx-client.py on ib1:
$ CUDA_VISIBLE_DEVICES=3 UCX_NET_DEVICES=mlx5_1:1 python ucx-client.py
7.57 GB /s
Running ucx-client.py on ib2:
$ CUDA_VISIBLE_DEVICES=4 UCX_NET_DEVICES=mlx5_1:1,mlx5_2:1 python ucx-client.py
7.78 GB /s
Running ucx-client.py on ib3:
$ CUDA_VISIBLE_DEVICES=6 UCX_NET_DEVICES=mlx5_1:1,mlx5_3:1 python ucx-client.py
7.73 GB /s
When I don't include UCX_NET_DEVICES in the command, I get ~330MB/s when
I run ucx-client.py with CUDA_VISIBLE_DEVICES > 3.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Akshay-Venkatesh/ucx-py/issues/103#issuecomment-489743925,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACKZTAR3NZWNJYHY57B7MDPUCA3JANCNFSM4HK3QQBQ
.

Sorry @mrocklin - I haven't tested with multiple workers yet.  Unfortunately, the approach I used to get consistently good performance might not make sense at scale... Just for reference, I found that this procedure usually results in ~7GB/s for the ucx-client.py test:

Scheduler: Set UCX_NET_DEVICES to the nearest HSA when starting the scheduler (e.g. mlx5_1:1 on ib1)
Worker: Set UCX_NET_DEVICES to the nearest HSA of the worker and the nearest HSA of the scheduler (e.g. UCX_NET_DEVICES=mlx5_1:1,mlx5_2:1 when scheduler is on ib1 and the worker is on ib2)
Client (e.g. ucx-client.py): Use the same setting as the worker, but add the nearest HSA of the client (if not already in the list)

In practice every worker will have to talk to every other worker.  Does
this change things?
On Mon, May 6, 2019 at 6:27 PM (Rick) Richard J Zamora <
<EMAIL_ADDRESS>wrote:

Sorry @mrocklin https://github.com/mrocklin - I haven't tested with
multiple workers yet. Unfortunately, the approach I used to get
consistently good performance might not make sense at scale... Just for
reference, I found that this procedure usually results in ~7GB/s for the
ucx-client.py test:

Scheduler: Set UCX_NET_DEVICES to the nearest HSA when starting
the scheduler (e.g. mlx5_1:1 on ib1)
Worker: Set UCX_NET_DEVICES to the nearest HSA of the worker and
the nearest HSA of the scheduler (e.g.
UCX_NET_DEVICES=mlx5_1:1,mlx5_2:1 when scheduler is on ib1 and the
worker is on ib2)
Client (e.g. ucx-client.py): Use the same setting as the worker,
but add the nearest HSA of the client (if not already in the list)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Akshay-Venkatesh/ucx-py/issues/103#issuecomment-489823352,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACKZTHWIP3IE5HYM3Y5BOLPUC5G5ANCNFSM4HK3QQBQ
.

I suspect that the behavior will just become the same as UCX_NET_DEVICES=all (which is default, and slow for some reason).

This is probably something to ask @Akshay-Venkatesh tomorrow

@mrocklin @rjzamora
This comment https://github.com/Akshay-Venkatesh/ucx-py/issues/103#issuecomment-489469676 captures the crux of the issue. Infiniband (IB) devices that are closest to the GPUs should be used for communication else there are known bandwidth bottlenecks (as you're seeing). UCX_NET_DEVICES controls which devices are used and in ucx-1.5 or this branch there's logic that I added to automatically pick the right IB device based on the GPU detected - if UCX_NET_DEVICES=all. If this automatic picking isn't working then it's a bug. Let's dissect this more today.

that I added to automatically pick the right IB device based on the GPU detected

This is accessible with the ucp.get_address() function?

This isn't accessible with ucp.get_address(). The GPU context may be created at a much a later point in the program after all. Also, it only makes sense to get this information once the GPU is selected. Before which point, all IB devices are more or less treated the same way. Even then UCP doesn't expose a way to get device information because it tries to hide such details and leave it to the implementation to make the best choice.
I should point out that this bandwidth bottleneck is GPU-memory specific. Host memory undergoes some minor degradation but in the same order as GPU-memory.

I guess my question is "How should I set things up, on, for example, a DGX".  I'm curious about what environment variables I should everywhere, what network interfaces processes should listen, and so on.
Recall that Dask's situation is that it will have a separate process attached to each GPU.  That process will open up a listener on some network interface.  Each process will want to talk to every other process.

I think we can close this now. @madsbk  has added a benchmark test here: https://github.com/rapidsai/dask-cuda/pull/192
and
https://github.com/rapidsai/dask-cuda/pull/179


[Pipelines] Pipeline and intype=url

This pipeline does not work:
[{
		"method": "POST",
		"endpoint": "http://api.freme-project.eu/current/toolbox/nif-converter/",
		"parameters": {
			"intype": "url",
			"input" : "https://www.theguardian.com/world/2009/oct/27/berlin-wall-short-history"
		},
		"headers": {
			"content-type": "text/html",
			"accept": "text/turtle"
		}
	},
	{
		"method": "POST",
		"endpoint": "http://api.freme-project.eu/current/e-entity/freme-ner/documents/",
		"parameters": {
			"language": "en",
			"dataset" : "dbpedia"
		},
		"headers": {
			"content-type": "text/turtle",
			"accept": "text/html"
		}
	}

]

It causes an Internal Server Error.

The reason is, that roundtripping in the pipeline service requires any content to convert to turtle, but the body of the first request is empty.

Lets leave this bug open, currently we have more important things to do.

You can get the urlfilter functionality (also in combination with roundtripping) when you store a pipeline which handles just turtle and call it via pipelining/chain/{id} with the request parameters intype=url and input={YOUR_URL}. Furthermore, you can set the accept header to text/html to get roundtripping funcitonaility. The same holds for #89.
Try adding this pipeline:
[
{
  "method": "POST",
  "endpoint": "https://api-dev.freme-project.eu/current/e-entity/freme-ner/documents",
  "parameters": {
     "dataset": "dbpedia",
    "language": "en"
  }
},
{
  "method": "POST",
  "endpoint": "https://api-dev.freme-project.eu/current/e-link/documents/",
  "parameters": {
    "templateid": "1"
  }
}
]

and call it with:
curl -X POST -H "Accept: text/html" -H "Content-Type: text/html" -d '' "http://api-dev.freme-project.eu/current/pipelining/chain/10?intype=url&input=http://beautifulberlin.blogspot.de/2009/07/short-history-of-berlin.html"


[Feature] About model upgrading

What's the feature?
I want to migrate the project made on mmdetection2.0 to mmdetection3. x. What is the conversion method of the model that can make it run on mmdetection3. x, and whether there will be any loss of accuracy？
Any other context?
No response

The model checkpoint does not need to be upgraded because the compatibility is fully kept. You only need to update the interface of your code.


[PROPOSAL] How I migrated from 2.x to 3.x

I see lot of people complaining about the new version, "everything changed", so I share my story of migration to the new version.
Here's my old 2.x code:
I have a ReportController with more than 20 different reports. So my report 1 example:
Route:
/admin/reports/{report1}/{xls}

Controller:
public function export1(Company $company, $type)
{
    $result = $company->whereHas('years', function ($q) {
        $q->where('year_id', currentYearId());
        $q->where('year_company.status', 1);
    })
    ->with(['company_datas', 'stands', 'co_companies' => function ($q) {
        $q->with('co_company_datas');
    }])
    ->where('canceled', 0)
    ->get();

    $data = [];

    foreach ($result as $r) {

        $pending = ($r->first_step == 1) ? 'No' : 'Yes';

        if (strlen(trim($r->company_datas[0]->flyer)) > 0) {
            $flyer = $r->company_datas[0]->flyer;
        } else {
            $flyer = '';
        }

        if (isset($r->stands[0]) && strlen($r->stands[0]->code) > 0) {
            $code = $r->stands[0]->code;
        } else {
            $code = '';
        }

        $data[] = [$r->name, $flyer, 'N', $code, $r->company_datas[0]->online_invitation_name, $pending];

        if (isset($r->co_companies) && ! $r->co_companies->isEmpty()) {
            foreach ($r->co_companies as $cco) {
                if (strlen(trim($cco->co_company_datas[0]->flyer)) > 0) {
                    $flyer = $cco->co_company_datas[0]->flyer;
                } else {
                    $flyer = '';
                }

                $data[] = [$cco->name, $flyer, 'Y', $code, '', ''];
            }
        }
    }

    $captions = ['Company name', 'Flyer name', 'Co Company', 'Stand', 'Online invitation', 'Pending'];

    $this->export($data, 'flyer_data', $captions, $type);
}

Then I have a private function which generates the output
private function export($data, $filename, $captions, $type)
{
    if (! in_array($type, ['xls', 'csv'])) {
        $type = 'csv';
    }

    $fn = $filename.'-'.date('Y-m-d_H-i-s');

    Excel::create($fn, function ($excel) use ($data, $captions) {

        $excel->sheet('SHEET NAME', function ($sheet) use ($data, $captions) {

            $sheet->fromArray($data, null, 'A1', false, false);
            $sheet->prependRow(1, $captions);
            $sheet->freezeFirstRow();

        });

    })->export($type);
}

For the 3.x version I moved some code to a new Export class. I'll comment it in the source code
(app/Exports/Report1Export.php)

namespace App\Exports;

use App\Models\Company;
use Maatwebsite\Excel\Concerns\FromCollection;
use Maatwebsite\Excel\Concerns\WithEvents;
use Maatwebsite\Excel\Concerns\WithHeadings;
use Maatwebsite\Excel\Events\AfterSheet;

class Report1Export implements FromCollection, WithHeadings, WithEvents {

    // set the headings
    public function headings(): array
    {
        return [
            'Company name', 'Flyer name', 'Co Company', 'Stand', 'Online invitation', 'Pending'
        ];
    }

    // freeze the first row with headings
    public function registerEvents(): array
    {
        return [
            AfterSheet::class => function(AfterSheet $event) {
                $event->sheet->freezePane('A2', 'A2');
            },
        ];
    }

    public function collection()
    {
        // the above code is the same as in 2.x was ..
        $result = Company::whereHas('years', function ($q) {
            $q->where('year_id', currentYearId());
            $q->where('year_company.status', 1);
        })
        ->with(['company_datas', 'stands', 'co_companies' => function ($q) {
            $q->with('co_company_datas');
        }])
        ->where('canceled', 0)
        ->get();

        $data = [];

        foreach ($result as $r) {

            $pending = ($r->first_step == 1) ? 'No' : 'Yes';

            if (strlen(trim($r->company_datas[0]->flyer)) > 0) {
                $flyer = $r->company_datas[0]->flyer;
            } else {
                $flyer = '';
            }

            if (isset($r->stands[0]) && strlen($r->stands[0]->code) > 0) {
                $code = $r->stands[0]->code;
            } else {
                $code = '';
            }

            $data[] = [$r->name, $flyer, 'N', $code, $r->company_datas[0]->online_invitation_name, $pending];

            if (isset($r->co_companies) && ! $r->co_companies->isEmpty()) {
                foreach ($r->co_companies as $cco) {
                    if (strlen(trim($cco->co_company_datas[0]->flyer)) > 0) {
                        $flyer = $cco->co_company_datas[0]->flyer;
                    } else {
                        $flyer = '';
                    }

                    $data[] = [$cco->name, $flyer, 'Y', $code, '', ''];
                }
            }
        }

        // ..but I return a collection from the built array data
        return collect($data);
    }
}

Added the freezePane macro to a service provider boot method (eg.: AppServiceProvider, or better make a specific one like ExportServiceProvider):
Sheet::macro('freezePane', function (Sheet $sheet, $pane) {
    $sheet->getDelegate()->getActiveSheet()->freezePane($pane);  // <-- https://stackoverflow.com/questions/49678273/setting-active-cell-for-excel-generated-by-phpspreadsheet
});

My ReportController method is simpler now:
public function export1($type)
{
    // now I return the result
    return $this->export('App\Exports\Report1Export', 'flyer_data', $type);
}

And the export() method looks linke this:
use App\Exports\Report1Export;

...

private function export($class, $filename, $type)
{
    if (! in_array($type, ['xls', 'csv'])) {
        $type = 'csv';
    }

    $fn = $filename.'-'.date('Y-m-d_H-i-s');

    return Excel::download(new $class, $fn.'.'.$type);
}

And that's it. I have fully accomplished the same result as it was in the 2.x version.
I hope it helped to someone..

Thanks @subdesign for providing your experience. I might add a simplified version of your example to the docs soon. I'd plan to write a blog post with a migration example in the future.

This is really helpful. I used 2.x regularly in the past.
How can you run native PhpSpreadsheet methods via Laravel Excel? In 2.x I was able to construct the sheet row by row, allowing me to keep count, etc. but I see no quick way of doing this now, unless I use PhpSpreadsheet directly. See my old 2.x code below.
I cycle through all invoice items (keeping count of the items) and add them as rows. At the end I add totals as follows:
// Cycle through all invoice items here, incrementing $counter with each row...
// ...

// Grand total
$counter++;
$sheet->row($counter, [
	'', '', '', '', '', '', '', '', '', $grand_totals['vat'], $grand_totals['excl'], $grand_totals['incl']
]);
$sheet->setColumnFormat(['J' . $counter => '[$£]#,##0.00_-']);
$sheet->setColumnFormat(['K' . $counter => '[$£]#,##0.00_-']);
$sheet->setColumnFormat(['L' . $counter => '[$£]#,##0.00_-']);
$sheet->cells('J' . $counter, function($cells) {
	$cells->setBorder(['top' => ['style' => 'double']]);
	$cells->setFontWeight('bold');
});
$sheet->cells('K' . $counter, function($cells) {
	$cells->setBorder(['top' => ['style' => 'double']]);
	$cells->setFontWeight('bold');
});
$sheet->cells('L' . $counter, function($cells) {
	$cells->setBorder(['top' => ['style' => 'double']]);
	$cells->setFontWeight('bold');
});

Figured it out, simply set a property in the export class. Cycle through the results in the collection() method (and increment the property). Finally, apply relevant styles in the registerEvents() method.
Very different from 2.x, but much better structure. Great job.

How to merge cells and change workersheet name?

@DarrenChowles  can You please provide some code of your export class?
I'm interested in making cell by cell table because of complex data that I want to show.

@DarrenChowles thank You for your code.
I will publish my solution after I will finish, because I'm using multi dimensional array with data shows in rows instead columns.
My array contains data for multiple shops and each shop is represented as separated table with headers , so it wont be so simple.
It would be nice to use array() to return array with data, and styling / formatting for each cell :)
But I also need some calculations made for each cell during inserting them to excel.

My way to create cell by cell table is to create at least two classes: standardExport, viewGenerator.
First class has to implements WithEvents, the only Event I need for now is afterSheet. From that I call another static function (now I see I don't realy need it) generate with my $sheet passed. This function call another class witch will generate the sheet.
    namespace App\Exports;

    use App\Products;
    use Maatwebsite\Excel\Concerns\WithEvents;
    use Maatwebsite\Excel\Concerns\Exportable;
    use Maatwebsite\Excel\Events\AfterSheet;
    use Maatwebsite\Excel\Concerns\RegistersEventListeners;

    class StatisticsRingsExport implements  WithEvents
    {
        use Exportable, RegistersEventListeners;
        static $stats;

	    public function __construct($statistics = [])
            {
		$this->statistics = $statistics;
                self::$stats = $statistics;
	    }

        public static function generate($sheet)
        {
            $generator = new RingsSheetGenerator($sheet, self::$stats);
        }

        public static function afterSheet(AfterSheet $event)
        {
            $sheet = $event->sheet;
            self::generate($sheet);
        }

    }

Second class RingsSheetGenerator looks like:
    namespace App\Exports;

    use \PhpOffice\PhpSpreadsheet\Cell;

    class RingsSheetGenerator
    {
        var $row = 1; // this will hold current row position
        var $col = 1; // this will hold current column position
        var $sold_rows = []; // some helpers
        var $stock_rows = [];
        var $gray_rows = [];
        var $cols_chars = [];
        var $headers_rows_styles = [ // I have skipped the rest of arrays with styles
            'font' => [
                'bold' => true,
            ]
        ];

        public function __construct(&$sheet, $statistics) // Sheet ought to be passed as alias
        {
            $this->statistics = $statistics;
            $this->generateColsChars();
            $this->generate($sheet, $statistics); // is my function that calls other functions that generates sheet
        }

        public function generateColsChars() // this function is very extremly helpfull for generating cells rows
        {
            $cols_chars_1 = range('A', 'Z');
            $cols_chars_2 = range('A', 'Z');
            $this->cols_chars = $cols_chars_1;
            array_unshift($this->cols_chars, '');

            foreach ($cols_chars_1 as $char) {
                foreach($cols_chars_2 as $c)
                    $this->cols_chars[] = $char.$c;
            }
        }

        public function cell($col, $row) // this will return ie: M22 - very usefull for formulas
        {
            return $this->cols_chars[$col].$row;
        }

    }

I am getting this issue on excel file import on web server. But this is working fine on local server.
Maatwebsite \ Excel \ Exceptions \ NoTypeDetectedException
No ReaderType or WriterType could be detected. Make sure you either pass a valid extension to the filename or pass an explicit type.
Please help to resolve this issue.

I am getting this issue on excel file import on web server. But this is working fine on local server.
Maatwebsite \ Excel \ Exceptions \ NoTypeDetectedException
No ReaderType or WriterType could be detected. Make sure you either pass a valid extension to the filename or pass an explicit type.
Please help to resolve this issue.

Please open a new issue for this and fill in the issue template. Thanks!

Thanks Bro!!...I stucked for 1 week you solved my problem within minutes...Thanks a lot!!

Thanks @subdesign && @DarrenChowles that really helped!

I see lot of people complaining about the new version, "everything changed", so I share my story of migration to the new version.

You're spot on. For some reason we now have to do a significantly higher amount of work to achieve the same output. This is not an upgrade.
What I did on 2.1:
Excel::create($filename, function($excel) use($data) {

    $excel->sheet("Export_Data", function($sheet) use($data) {

        $sheet
            ->setColumnFormat([
                "L"     => "d/m/y",
                "M"     => "d/m/y",
                "N"     => "h:mm",
                "O"     => "h:mm",
                "AN"    => "d/m/y",
                "AO"    => "d/m/y",
                "AP"    => "d/m/y",
                "AQ"    => "d/m/y",
            ])
            ->fromArray($data, null, "A1", true)
            ->freezeFirstRow()
        ;

    });

})->store($extension, storage_path("app/downloads"));

Suddenly this package becomes excessive compared to using PHPSpreadsheet directly, for example:
$spreadsheet = new Spreadsheet();

$worksheet = $spreadsheet->getActiveSheet();
$worksheet
    ->setTitle("Export_Data")
    ->fromArray(array_keys(current($data)), null, "A1", true)
    ->fromArray($data, null, "A2", true)
    ->freezePane("A2")
;

foreach ([
    "L"     => "d/m/y",
    "M"     => "d/m/y",
    "N"     => "h:mm",
    "O"     => "h:mm",
    "AN"    => "d/m/y",
    "AO"    => "d/m/y",
    "AP"    => "d/m/y",
    "AQ"    => "d/m/y",
] as $c => $format) {
    $worksheet
        ->getStyle(sprintf("%s2:%s%d", $c, $c, $worksheet->getHighestRow($c)))
        ->getNumberFormat()
        ->setFormatCode($format)
    ;
}

(SpreadsheetIOFactory::createWriter($spreadsheet, ucfirst($extension)))
    ->save(sprintf("%s/%s.%s", storage_path("app/downloads"), $filename, $extension))
;

Opinions differ, for all of my use cases the amount of code needed to create my exports greatly reduced.
However this package is still 100% optional, if you don't like it, you can use any other package that does fit your needs.


Refactor client connect checks

Reuse code to create client, check messages sent/recv
Print exceptions and translate to true/false
Close clients after use

@lulf I see

@kornys The currently failing tests doesnt seem to be related to this change.


Port game content over from old repo

See the porting kanban for progress

Some work still to do but all v1 issues resolved