feat(elixir): implement elixir ABAC policy checks

Policies contain attribute rules
Attribute rules match against ABAC.Request
ABAC.Request is associated to policy by ActionId (action name + resource name) ABAC.Request contains action resource and subject attributes
ABAC.Authorization implements message flow authorization using ABAC:
Subject attributes are taken from the credential attributes storage Resource attributes are taken from the worker state (e.g. "address") Action attributes are taken from the Message (e.g. "to", "from")
Workers can have :attributes configured on start to go to resource attributes Message local metadata goes to action attributes (except :identity and :identity_id)

Current Behavior

Proposed Changes

Checks

[x] All commits in this Pull Request are signed and Verified by Github.
[x] All commits in this Pull Request follow the Ockam commit message convention.
[x] I accept the Ockam Community Code of Conduct.
[x] I have accepted the Ockam Contributor License Agreement by adding my Git/Github details in a row at the end of the CONTRIBUTORS.csv file in a separate pull request to the build-trust/ockam-contributors repository.


PHP 7.1 - Undefined symbols for architecture x86_64: "_tidyLibraryVersion"

I'm trying to install PHP 7.1 on Mojave 10.14.6
Have the latest Xcode installed 11.3.1
Was able to install 7.2, 7.3 and 7.4 just fine
Keep getting this error below:
Installing<EMAIL_ADDRESS><EMAIL_ADDRESS>is not installed, installing it now via Brew... 🍻
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> Updated Formulae
bit
broot
contentful-cli
dvc
fluxctl
fuseki
grafana
now-cli
yadm

==> Installing<EMAIL_ADDRESS>from exolnet/deprecated
==> Downloading https://www.php.net/distributions/php-7.1.33.tar.xz
Already downloaded: /Users/omitted/Library/Caches/Homebrew/downloads/947253863d40b36e7d22cab588e833b5b73e03f7f17fb3c5cce4531750d09e99--php-7.1.33.tar.xz
==> Patching
patching file acinclude.m4
Hunk #1 succeeded at 444 (offset 3 lines).
Hunk #2 succeeded at 459 (offset 3 lines).
Hunk #3 succeeded at 494 (offset 3 lines).
Hunk #4 succeeded at 506 (offset 3 lines).
Hunk #5 succeeded at 2541 (offset 122 lines).
==> ./buildconf --force
==> ./configure<EMAIL_ADDRESS>--localstatedir=/usr/local/var --sysconfdir=/usr/local/etc/php/7.1 --with-config-file-path=/usr/local/etc/php/7.1 --with-config-file-scan-dir=/usr/local/etc/php/7.1/conf.d --with-pear=/usr/local/Cellar/php<EMAIL_ADDRESS>--enable-bcmath --enable-calendar --enable-dba --enable-dtrace --enable-exif --enable-ftp --enable-fpm --enable-intl --enable-mbregex --enable-mbstring --enable-mysqlnd --enable-opcache-file --enable-pcntl --enable-phpdbg --enable-phpdbg-webhelper --enable-shmop --enable-soap --enable-sockets --enable-sysvmsg --enable-sysvsem --enable-sysvshm --enable-wddx --enable-zip --with-apxs2=/usr/local/opt/httpd/bin/apxs --with-bz2=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr --with-curl=/usr/local/opt/curl-openssl --with-fpm-user=_www --with-fpm-group=_www --with-freetype-dir=/usr/local/opt/freetype --with-gd --with-gettext=/usr/local/opt/gettext --with-gmp=/usr/local/opt/gmp --with-iconv=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr --with-icu-dir=/usr/local/opt/icu4c --with-jpeg-dir=/usr/local/opt/jpeg --with-kerberos=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr --with-layout=GNU --with-ldap=/usr/local/opt/openldap --with-ldap-sasl=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr --with-libedit=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr --with-libxml-dir=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr --with-libzip --with-mcrypt=/usr/local/opt/mcrypt --with-mhash=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr --with-mysql-sock=/tmp/mysql.sock --with-mysqli=mysqlnd --with-ndbm=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr<EMAIL_ADDRESS>--with-pdo-dblib=/usr/local/opt/freetds --with-pdo-mysql=mysqlnd --with-pdo-odbc=unixODBC,/usr/local/opt/unixodbc --with-pdo-pgsql=/usr/local/opt/libpq --with-pdo-sqlite=/usr/local/opt/sqlite --with-pgsql=/usr/local/opt/libpq --with-pic --with-png-dir=/usr/local/opt/libpng --with-pspell=/usr/local/opt/aspell --with-sqlite3=/usr/local/opt/sqlite --with-tidy=/usr/local/opt/tidy-html5 --with-unixODBC=/usr/local/opt/unixodbc --with-webp-dir=/usr/local/opt/webp --with-xmlrpc --with-xsl=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr --with-zlib=/Applications/_DEV/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr
==> make
Last 15 lines from<EMAIL_ADDRESS>ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [sapi/cli/php] Error 1
Undefined symbols for architecture x86_64:
  "_tidyLibraryVersion", referenced from:
      _zm_info_tidy in tidy.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [libs/libphp7.bundle] Error 1
Undefined symbols for architecture x86_64:
  "_tidyLibraryVersion", referenced from:
      _zm_info_tidy in tidy.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [sapi/phpdbg/phpdbg] Error 1

If reporting this issue please do so at (not Homebrew/brew or Homebrew/core):
  https://github.com/exolnet/homebrew-deprecated/issues

In Brew.php line 140:

  Brew was unable to install<EMAIL_ADDRESS>

My tidy-html5 is already at the latest version 5.6.0
If anyone could help me resolve this error that would be greatly appreciated
Thank you

if it's still not fixed you can try brew reinstall --build-from-source<EMAIL_ADDRESS>

Hi!
Thank you for your interest in this repository. Unfortunately, this repository will now be archived with no further actions. We are sorry for the inconvenience.
Why are we closing this repository? This repository was only meant as a temporary measure, not a permanent one. Its only purpose was to ease the transition, considering that formulae from the homebrew-core tap are removed almost the day they became unsupported by the vendor. We needed a few more months to allow us to upgrade the code base of the various projects we have. But it was always with the intention of doing those upgrades, not by relying on a repository to keep old php versions artificially alive. We do not condone the use of deprecated software that could lead to serious security vulnerabilities.
Why are we not redirecting to another repository? Redirecting to another repository could be interpreted as an endorsement of said repository. If we were to do such a thing, we would not do it without vetting it first. And we do not wish to put the time and energy required in a vetting process of a third party repository. As the reason why a vetting process would be required, consider this. Before installing a software library on all our developer computers from an untrusted source, we would need to make sure that this software library is free from any malicious code (Trojan, ransomware, etc.), both in the repository itself and in the packaged binaries (the homebrew bottles, if any).
Thank you for your understanding.

Hi!
Thank you for your interest in this repository. Unfortunately, this repository will now be archived with no further actions. We are sorry for the inconvenience.
Why are we closing this repository? This repository was only meant as a temporary measure, not a permanent one. Its only purpose was to ease the transition, considering that formulae from the homebrew-core tap are removed almost the day they became unsupported by the vendor. We needed a few more months to allow us to upgrade the code base of the various projects we have. But it was always with the intention of doing those upgrades, not by relying on a repository to keep old php versions artificially alive. We do not condone the use of deprecated software that could lead to serious security vulnerabilities.
Why are we not redirecting to another repository? Redirecting to another repository could be interpreted as an endorsement of said repository. If we were to do such a thing, we would not do it without vetting it first. And we do not wish to put the time and energy required in a vetting process of a third party repository. As the reason why a vetting process would be required, consider this. Before installing a software library on all our developer computers from an untrusted source, we would need to make sure that this software library is free from any malicious code (Trojan, ransomware, etc.), both in the repository itself and in the packaged binaries (the homebrew bottles, if any).
Thank you for your understanding.


fetch-kde-qt.sh: use mirrorlist files to get sha256 sums

Motivation for this change
make this script much faster
checksums are stored in *.mirrorlist files, for example qtwebview-everywhere-src-6.2.0.tar.xz.mirrorlist
i use grep + sed to parse the sha256 sums
tested with success, to update qt-6/6.2/srcs.nix
related
part of #141883
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

KDE doesn't provide the mirrorlist files anymore in a way that wget can find them. Please make separate scripts for Qt and KDE so we can continue to download KDE sources.

KDE doesn't provide the mirrorlist files anymore in a way that wget can find them

example package?

example package?

cd nixpkgs/pkgs

grep -r -F -l fetch-kde-qt.sh | grep -v development/libraries/qt-
desktops/plasma-5/default.nix
desktops/plasma-5/srcs.nix
development/libraries/kde-frameworks/default.nix
development/libraries/kde-frameworks/srcs.nix
applications/kde/default.nix
applications/kde/srcs.nix
applications/plasma-mobile/default.nix
applications/plasma-mobile/srcs.nix

good news : )

getting the sha256 is much simpler: append .sha256 to the url
both qt.io and kde.org provide this interface

examples:

https://download.kde.org/stable/frameworks/5.85/attica-5.85.0.tar.xz.sha256
https://download.qt.io/official_releases/qt/6.2/6.2.1/submodules/qtwebview-everywhere-src-6.2.1.zip.sha256

challenge:
these *.sha256 files do not appear in the directory listing
so for example this fails
wget -nH -r -c --no-parent https://download.kde.org/stable/frameworks/5.85/ -A *.tar.xz.sha256

test script to update all sources. takes around 6 minutes

#!/usr/bin/env bash

# run this in nixpkgs/

pkglist=()
while read srcs
do
  grep -F 'fetch-kde-qt.sh' $srcs >/dev/null || continue
  pkg=$(dirname $srcs)
  echo "$pkg"
  #echo "$pkg/fetch.sh"
  #echo "$pkg" | grep ^pkgs/development/libraries/qt-5/ >/dev/null && continue # known to work
  #echo "$pkg" | grep ^pkgs/development/libraries/qt-6/ >/dev/null && continue # known to work
  pkglist+=("$pkg")
  ./maintainers/scripts/fetch-kde-qt.sh $pkg/
done < <(find pkgs/ -name srcs.nix)

x=""
for pkg in "${pkglist[@]}"
do
  x+=" $pkg/srcs.nix"
done
echo "compare all:"
echo "git diff $x"

move to #148121


Add ejv2.cc

This PR is:

[x] Adding a new domain
[ ] Updating existing domain size
[ ] Changing domain name
[ ] Removing existing domain from list
[ ] Website code changes (512kb.club site)
[ ] Other not listed

[x] I used the uncompressed size of the site
[x] I have included a link to the Cloudflare report
[x] This site is not an ultra minimal site
[x] The following information is filled identical to the data file

I confirm that I have read the FAQ section, particularly the two red items around minimal pages and inappropriate content and I attest that this site is neither of these things.

[x] Check to confirm

- domain: ejv2.cc
  url: https://ejv2.cc/
  size: 221.54
  last_checked: 2024-09-16

Cloudflare Report: https://radar.cloudflare.com/scan/e1f4c2d0-9851-4493-9e8d-fea40c1ad14f/summary

Thanks!

Chers!


the dropdown not showing....

i cant select any country, state and city... the dropdown is not showing

try csc_picker

okay thanks


error when qemu testing riscv64_zvl128b

HI! I am trying to cross compile openblas( target:riscv,host:x86_64).I'm doing what the guthub action do for testing riscv64_zvl128b.
My code is on 8483a71 and my riscvtoolchain is riscv64-glibc-ubuntu-20.04-llvm-nightly-2024.02.02-nightly.
my build scrips for openblaslib:
#!/bin/bash

export SRC_PATH=`pwd`
export RISCV_TOOLCHAIN=${SRC_PATH}/opt/riscv
export TOOLCHAIN_VERSION=13.2.0
export PATH=${RISCV_TOOLCHAIN}/bin:$PATH
make TARGET=RISCV64_ZVL128B CFLAGS="-DTARGET=RISCV64_ZVL128B" \
CC='clang --rtlib=compiler-rt -target riscv64-unknown-linux-gnu --sysroot ${RISCV_TOOLCHAIN}/sysroot --gcc-toolchain=${RISCV_TOOLCHAIN}/lib/gcc/riscv64-unknown-linux-gnu/${TOOLCHAIN_VERSION}' \
AR='riscv64-unknown-linux-gnu-ar' AS='riscv64-unknown-linux-gnu-gcc' LD='riscv64-unknown-linux-gnu-gcc' \
RANLIB='riscv64-unknown-linux-gnu-ranlib' \
FC='riscv64-unknown-linux-gnu-gfortran'  TARGET=RISCV64_ZVL128B BINARY=64 ARCH=riscv64 \
HOSTCC=gcc HOSTFC=gfortran -j32

it seems no errors:
OpenBLAS build complete. (BLAS CBLAS LAPACK LAPACKE)

  OS               ... Linux
  Architecture     ... riscv64
  BINARY           ... 64bit
  C compiler       ... CLANG  (cmd & version : clang version 17.0.2 (https://github.com/llvm/llvm-project.git b2417f51dbbd7435eb3aaf203de24de6754da50e))
  Fortran compiler ... GFORTRAN  (cmd & version : GNU Fortran () 13.2.0)
  Library Name     ... libopenblas_riscv64_zvl128bp-r0.3.28.dev.a (Multi-threading; Max num-threads is 32)

To install the library, you can run "make PREFIX=/path/to/your/installation install".

Note that any flags passed to make during build should also be passed to make install
to circumvent any install errors.

my build scripts for OpenBLAS tests:
#!/bin/bash

export src_path=`pwd`
export RISCV_TOOLCHAIN=${src_path}/opt/riscv
export riscv_gnu_toolchain_version=13.2.0
export PATH=${RISCV_TOOLCHAIN}/bin:$PATH
# build OpenBLAS tests
make TARGET=RISCV64_ZVL128B CFLAGS="-DTARGET=RISCV64_ZVL128B" \
CC='riscv64-unknown-linux-gnu-gcc' \
AR='riscv64-unknown-linux-gnu-ar' AS='riscv64-unknown-linux-gnu-gcc' LD='riscv64-unknown-linux-gnu-gcc' \
RANLIB='riscv64-unknown-linux-gnu-ranlib' \
FC='riscv64-unknown-linux-gnu-gfortran' TARGET=RISCV64_ZVL128B BINARY=64 ARCH=riscv64 \
HOSTCC=gcc HOSTFC=gfortran -j tests

There are no errors too;
Then i run the scrips for running OpenBLAS tests:
#!/bin/bash

export SRC_PATH=`pwd`
export RISCV_TOOLCHAIN=${SRC_PATH}/opt/riscv
export TOOLCHAIN_VERSION=13.2.0
export PATH=${RISCV_TOOLCHAIN}/bin:$PATH

# run OpenBLAS tests
export QEMU_CPU=rv64,g=true,c=true,v=true,vext_spec=v1.0,vlen=128,elen=64
rm -rf ./test_out
mkdir -p ./test_out
run_test() { local DIR=$1; local CMD=$2; local DATA=$3; local OUTPUT="./test_out/$DIR.$CMD"; \
echo "`pwd`/$DIR/$CMD $DIR/$DATA" >> $OUTPUT; \
if [[ -z $DATA ]]; then qemu-riscv64 ./$DIR/$CMD |& tee $OUTPUT ; \
else qemu-riscv64 ./$DIR/$CMD < ./$DIR/$DATA |& tee $OUTPUT ; fi ; \
RV=$? ; if [[ $RV != 0 ]]; then echo "*** FAIL: nonzero exit code $RV" >> $OUTPUT ; fi \
}
run_test test cblat1 &
run_test test cblat2 cblat2.dat &
run_test test cblat3 cblat3.dat &
run_test test dblat1 &
run_test test dblat2 dblat2.dat &
run_test test dblat3 dblat3.dat &
run_test test sblat1 &
run_test test sblat2 sblat2.dat &
run_test test sblat3 sblat3.dat &
run_test test zblat1 &
run_test test zblat2 zblat2.dat &
run_test test zblat3 zblat3.dat &
run_test ctest xccblat1 &
run_test ctest xccblat2 cin2 &
run_test ctest xccblat3 cin3 &
run_test ctest xdcblat1 &
run_test ctest xdcblat2 din2 &
run_test ctest xdcblat3 din3 &
run_test ctest xscblat1 &
run_test ctest xscblat2 sin2 &
run_test ctest xscblat3 sin3 &
run_test ctest xzcblat1 &
run_test ctest xzcblat2 zin2 &
run_test ctest xzcblat3 zin3 &
wait
grep -lZ FAIL ./test_out/* > temp_log_file
while IFS= read -r -d $'\0' LOG; do cat $LOG ; FAILURES=1 ; done < temp_log_file
if [[ ! -z $FAILURES ]]; then echo "==========" ; echo "== FAIL ==" ; echo "==========" ; echo ; exit 1 ; fi

There some errors:
./test/sblat2: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/cblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/zblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/cblat2: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
./test/cblat3: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/zblat2: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xccblat2: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xdcblat3: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xccblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xzcblat2: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xdcblat2: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xscblat3: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
run.sh: line 14:  : command not found
./test/dblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/sblat3: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/dblat2: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xzcblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xzcblat3: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xscblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/sblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xscblat2: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xccblat3: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/zblat3: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./test/dblat3: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
./ctest/xdcblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
./ctest/xccblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
*** FAIL: nonzero exit code 127
./ctest/xdcblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
*** FAIL: nonzero exit code 127
./ctest/xscblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
*** FAIL: nonzero exit code 127
./ctest/xzcblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
*** FAIL: nonzero exit code 127
./test/cblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
*** FAIL: nonzero exit code 127
./test/dblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
*** FAIL: nonzero exit code 127
./test/sblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
*** FAIL: nonzero exit code 127
./test/zblat1: error while loading shared libraries: libgfortran.so.5: cannot open shared object file: No such file or directory
*** FAIL: nonzero exit code 127
==========
== FAIL ==
==========

First i thought it can't find the libgfortran.so.5 which path is /opt/riscv/sysroot/lib/libgfortran.so.5,so I add/opt/riscv/sysroot/libinto LD_LIBRARY_PATH ,but there are still some errors:
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
run.sh: line 14:  : command not found
*** FAIL: nonzero exit code 127
*** FAIL: nonzero exit code 127
*** FAIL: nonzero exit code 127
*** FAIL: nonzero exit code 127
*** FAIL: nonzero exit code 127
*** FAIL: nonzero exit code 127
*** FAIL: nonzero exit code 127
*** FAIL: nonzero exit code 127
==========
== FAIL ==
==========

Then I try to execute the command run_test test cblat1 &manually:
qemu-riscv64 test/cblat1
Segmentation fault (core dumped)

The " : command not found" suggests that you have some invisible,unprintable character in your run script. The most likely one would be ^M (DOS/Windows line end marker) if you ever edited that script on a Windows system. Try running your run.sh file through dos2unix or similar.
Not sure why your manual execution of qemu-riscv64 test/cblat1 failed, perhaps you forgot to specify the cpu options that the script provides in the QEMU_CPU environment variable ?

Thanks for reply! I checked the run.sh, there is indeed an extra space at the end of line 14. After I fix this, the script runs without errors but no output results for tests. Then I try to execute the command run_test test cblat1 &manually again, adding -cpu rv64,g=true,c=true,v=true,vext_spec=v1.0,vlen=128,elen=64,still got a Segmentation fault (core dumped)
Here are the backtrace of qemu:
Thread 1 "qemu-riscv64" received signal SIGSEGV, Segmentation fault.
0x00007fffe82a6c5a in code_gen_buffer ()
(gdb) bt
#0  0x00007fffe82a6c5a in code_gen_buffer ()
#1  0x000055555565850c in cpu_tb_exec (tb_exit=<synthetic pointer>,
    itb=<optimized out>, cpu=0x5555558542c0)
    at ../qemu/accel/tcg/cpu-exec.c:458
#2  cpu_loop_exec_tb (tb_exit=<synthetic pointer>,
    last_tb=<synthetic pointer>, pc=<optimized out>,
    tb=<optimized out>, cpu=0x5555558542c0)
    at ../qemu/accel/tcg/cpu-exec.c:920
#3  cpu_exec_loop (cpu=cpu@entry=0x5555558542c0, sc=<optimized out>)
    at ../qemu/accel/tcg/cpu-exec.c:1041
#4  0x0000555555658b89 in cpu_exec_setjmp (
    cpu=cpu@entry=0x5555558542c0, sc=<optimized out>)
    at ../qemu/accel/tcg/cpu-exec.c:1058
#5  0x0000555555658bea in cpu_exec (cpu=cpu@entry=0x5555558542c0)
    at ../qemu/accel/tcg/cpu-exec.c:1084
#6  0x00005555555a62c8 in cpu_loop (env=0x555555856a80)
    at ../qemu/linux-user/riscv/cpu_loop.c:37
#7  0x000055555559c6ca in main (argc=<optimized out>,
    argv=<optimized out>, envp=<optimized out>)
    at ../qemu/linux-user/main.c:1014

I also tried qemu remote gdb, can't even enter the main function of the program.
I notice that the [guthub action](https://github.com/OpenMathLib/OpenBLAS/actions/runs/11334111250/job/31519620687) no errors about libgfortran.so.5 like me, Maybe it has something to do with this?

Maybe it is something related to the hardware you are running qemu on - there were earlier cases where qemu would crash on vector instructions e.g. https://gitlab.com/qemu-project/qemu/-/issues/1022
As the CI job runs correctly, I do not think this is an error in OpenBLAS, and I lack the time to provide further help with your local problem, sorry.

Thanks again for help! I tried to run the program on real riscv environment(a board), it seems fine. So I'll try the riscv target without vector extension, and maybe report it to qemu.


I can create a method in a box that overwrite a key

I was playing with the lib by subclassing it and I founds some funny behaviour. I know it's not the intended usage but though that was worth reporting. Feel free to close it if you think it's useless:
class toto(Box):

    def __init__(self, valeur):

        super().__init__({"valeur": [valeur, valeur], "print": "toto"})

    def print(self):
        print(self.valeur)
        return

toto("truc").print
>>>><bound method toto.print of Box({'valeur': ['truc', 'truc'], 'print': 'toto'})>

I didn't know what to expect but it's the method that have priority over the key.
but the key still exist
toto("truc")["print"]
>>>>'toto'

I think it would be handy to at least raise a warning ?

I don't have a section of the docs for other devs subclassing and what that could mean, but for this case you'll need to expaind the _protected_keys to include print in this case.
https://github.com/cdgriffith/Box/blob/master/box/box.py#L148
Here is how I do it with a subclass of box https://github.com/cdgriffith/Box/blob/master/box/config_box.py#L20

that's exactly what I needed, thanks a lot. I assume I'll leave the issue open for documenting


Recurring schedule hides configuration errors

When configuring recurring jobs I've noticed that SolidQueue is hiding configuration mistakes that make it difficult to catch potential errors:

Jobs that don't pass SolidQueue::RecurringTask's validation are silently filtered out without any error
Setting SOLID_QUEUE_RECURRING_SCHEDULE to a path that doesn't exist makes SolidQueue silently fall back to an empty schedule

Expected behaviour: above errors should trigger an exception.

I also noticed a very odd behaviour:

when I pass an invalid path via SOLID_QUEUE_RECURRING_SCHEDULE then SolidQueue defaults to the default config/recurring.yml configuration file
when I pass an invalid path via --recurring_schedule_file then SolidQueue is defaulting to an empty schedule (not reading the default config/recurring.yml)

when I pass an invalid path via SOLID_QUEUE_RECURRING_SCHEDULE then SolidQueue defaults to the default config/recurring.yml configuration file

Hmm... I'm not seeing this. When I pass an invalid path via SOLID_QUEUE_RECURRING_SCHEDULE, the default path in config/recurrring.yml is not used either.  The code path for both options (environment variable or CLI option) is the same, so it's strange the behaviour would be different.

I thought I'm just tired, but I can still replicate this behaviour - I'm testing it by adding a new task to config/recurrring.yml - it gets added every time when I'm adding SOLID_QUEUE_RECURRING_SCHEDULE=non_existing_path.yml, doesn't when using --recurring_schedule_file=non_existing_path.yml.
Also this will no longer be an issue when passing an invalid path in any of these methods will result in an error.

Could you copy the full command you're using to run Solid Queue in each case?

What version are you running? What you describe sounds a lot like what was fixed in https://github.com/rails/solid_queue/pull/345 🤔 This would have been included in 1.0.0.beta.

I'm on 0.9.0 😅
Indeed the env variable seems to be simply ignored by cli, I got confused by the combination of these few issues.

Ahhhh! So sorry about that! It should be fixed now 😅
About the issue with the configuration, yes, it's on my radar; I've got it in my plans to raise or print some warnings about invalid configuration.

About the issue with the configuration, yes, it's on my radar; I've got it in my plans to raise or print some warnings about invalid configuration.

Raise please 😅
Also reading the code I could see that recurring.yml supports nesting tasks under rails-env keys (production, staging, etc.) - knowing this would save me some time. It might be quite a common thing to have recurring tasks tied to specific environments, might make sense to add this example in the docs ✌

Also reading the code I could see that recurring.yml supports nesting tasks under rails-env keys (production, staging, etc.) - knowing this would save me some time.

Hmm... I think this is standard for all Rails configurations 🤔 But yes, I'll enhance the example with this.

Oops, I forgot to close this one. Invalid configuration now raises an error thanks to https://github.com/rails/solid_queue/pull/427


Modify the About this Release page

Description:
The About this Release page content should be modified to reflect the latest changes for APIM 4.3.0.

Added the changes to APIM 4.3.0 doc space via PR [1]. Need to add to the master branch space.
[1] https://github.com/wso2/docs-apim/pull/7835

Master branch fixes are added through https://github.com/wso2/docs-apim/pull/7862. Hence closing.


Add CentOS support

Right now, the installation script only supports debian based operating systems because it uses the apt-get command. Could CentOS support be added?

At this time, we don't have plans to modify the install script. The CLI should still be installable via the direct download link.


Update access-on-premises-resources.md

'Dyname' looks like a typo and should be 'Dynamic'. I could not find any references to 'Dyname' elsewhere.

@QueryCoordinator thanks for helping improve the docs!
#sign-off


Plugins master merge (again)

@jsenin more tweaks to make the imports cleaner. Also clean merge with master again.

@jsenin wow I reallly broke that. Please do not merge. :man_facepalming:


Bug 2082667: Bump MAPI dependency. Separate node drain controller.

Update mapi-operator dependency for supply separate
node drain controller.
For additional context see:

https://github.com/openshift/machine-api-operator/pull/1023

I've reviewed the CI logs and they look good as far as I can tell, I think we can remove the hold
/retest-require

/retest-required

/retest-required


add All Orgs demo

No code changes required. Just the two normal demo files.

Can you turn on Netlify deploy previews for this project? That way we can check it out without cloning and running it locally.


question: Conditional redirect to OIDC provider login page based on IP together with OIDC-enabled web app

Hi,
I would to use Caddy and Caddy-Security for the following use case:

I have a Django app that already uses OIDC internally for authentication, let's say at www.myapp.com, its OIDC callback redirect URL is www.myapp.com/oidc/callback/
I already used Caddy as the webserver for the app, and block any external IP from accessing it

www.myapp.com {

    # Block external access
    @external {
     not remote_ip [LIST OF IPs]
    }
    respond @external 403

    # Static files
    handle_path /static/* {
        file_server {
            root /home/myapp/staticfiles
        }
    }

    # Reverse proxy
    reverse_proxy <IP_ADDRESS>:9090
}

What I would like to do is to allow external IP addresses to access the app but, before they can do so, they would need to authenticate themselves with the same OIDC that the app uses. If the authentication is successful, they would be automatically redirected to the app. Internal IP addresses would not need this extra layer of protection.

I know there is a generic OIDC configuration at https://github.com/authp/authp.github.io/blob/main/assets/conf/oauth/generic/Caddyfile but I must admit I don't even know where to start. Would you have some pointers?
Some additional questions:

would I need to set up and additional callback URL for Caddy with my OIDC provider?
How can I make the redirection to my OIDC provider conditional on an IP address?
What is exactly the base_auth_url?

Thank you very much in advance.

You should never "allow" something just because it comes from a certain IP address or subnet, IP addresses are never to be trusted because of how easily they can be spoofed.

Ok, but that is only one part of the problem, which would be nonetheless interesting to know whether it is at all possible, at least theoretically. I would be fine if all traffic would have to be authenticated/authorized by the OIDC provider before it can access the web app. The main question remains as to whether I would need to have two callback URLs, one for Caddy and one for Django. And if so, how can the OIDC provider know when to use one and the other? And how would Caddy need to be configured for that?

@nicolazilio , Django would not work with OIDC in this scenario. Once you have authenticated, the JWT token you are carrying is the caddy-security token.

@greenpau, yes, that makes sense, but of course one could just (re)authenticate with the Django app once it got past Caddy. My ultimate aim is not to have the app exposed directly to the world.


Which one is the same within all groups, the numerator or the denominator?

You mentioned in the paper that the denominator is the same for all groups, but the code seems to suggest the opposite.

In the code, the numerators are the same for all groups.

@1-2-3-4-0 ！！！extrewly sorry for the typo in the paper！i willrevise and update the arxiv version right now！


Liveprog21

Adds a label to probes

Pull Request Test Coverage Report for Build 915029832

160 of 189 (84.66%)  changed or added relevant lines in 22 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+1.05%) to 61.967%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

packages/Babylonian-UI.package/BPProbeMorph.class/instance/newExpressionMorph.st
0
1
0.0%

packages/Babylonian-UI.package/BPProbeMorph.class/instance/updateFrom..st
0
1
0.0%

packages/Babylonian-UI.package/BPStyler.class/instance/unstyledTextFrom..st
0
1
0.0%

packages/Babylonian-UI.package/BPProbeMorph.class/instance/updateExpressionMorphIfNecessary.st
0
6
0.0%

packages/Babylonian-UI.package/BPProbeMorph.class/instance/newButtonMorphWithLabel.doOnClick..st
0
20
0.0%

Totals

Change from base Build 914898463:
1.05%

Covered Lines:
4430

Relevant Lines:
7149

💛  - Coveralls


UIImageView category calls progress block on random thread

Hi,
I use a method on UIImageView to preload image and update indicator view:
[self.imageView sd_setImageWithURL:url placeholderImage:nil options:SDWebImageRetryFailed | SDWebImageRefreshCached progress:^(NSInteger receivedSize, NSInteger expectedSize) {
    NSAssert([NSThread isMainThread], @"In UIKit we should be in main thread lands.");
    // do something useful
} completed:^(UIImage *image, NSError *error, SDImageCacheType cacheType, NSURL *imageURL) {
    NSAssert([NSThread isMainThread], @"In UIKit we should be in main thread lands.");
    // do something useful
}];

Turns out progress block is called on some random thread while completion block is called on main thread.
What's the point of that behavior? This is not reflected in documentation either. If you call something on random thread, please mention this in documentation especially for UIKit categories.

@pronebird the documentation for SDWebImageDownloader stated that the progressBlock is called on a background queue. Also added 62180db for more explicitly saying this for all APIs. (4.x branch).


App Crash Report

What happened?
App started successfully after dragging in cvs file. It stopped working  and never completed even thought it had already processed many of the rows. I reported this same problem a week or so ago. It has never been fixed, even thought the help desk sent me an email that said it had been resolved. I have a lot of addresses that I need to submit. When will this bug be fixed?
Cindy
Steps To Reproduce
try dragging a cvs file of addresses yourself and using the UT standard gis reference
App Version
1.5.7
Electron Version
17.0.0
Operating System
macOS Monterey
Relevant log output
Error message
Error invoking remote method 'geocode': TypeError: Cannot read properties of undefined (reading 'statusCode')
Stack trace
Error: Error invoking remote method 'geocode': TypeError: Cannot read properties of undefined (reading 'statusCode')

They resolved the ticket because we are tracking the issue in this repository. We haven't had time to work on this bug yet. But please follow along with the original issue which we are tracking at https://github.com/agrc/api-client/issues/122. Then you will know when we can get to it and ultimately resolve the issue.
If our timeline is not acceptable to you, we do have other ways to geocode. If you have arcgis software we have a toolbox available or if you know how to write code, the api is ready to handle your geocoding job.
Since this is a duplicate of #122 I am going to close this issue. Thank you for the reminder that people are having problems.

Thank you for your quick response, Love the App! you guys are awesome!
C


would be possible to implement a "waiting state" until all the worker nodes are ready ?

Hi I m using your excellent implementation, but i m hitting a feature of the terraform kubernetes call.
Whenever i want to modify the pool size expand/reduce the request is executed immediately ie the terraform scripts are setting up the new pool size, but the creation of the nodes is taking time and the new configuration is not ready yet meanwhile the terraform script has finished the excution.
If i have another scirpt which has to run on the new pool it will fail as the nodes are not ready yet.
would be possible to implement a flag like "keep_pooling_until_all_nodes_are_ready" so the execution of the terraform script will finish when the pool is having the new settings ?
thanks
eugene

done in #174


[docs] Add Typography example for MenuItem

[x] I have followed (at least) the PR section of the contributing guide.

Fixes #11380

Well done :)

@oliviertassinari Is it okay to change the wording slightly:
There is a flexbox bug that prevents text-overflow: ellipse to work in combination with a flexbox layout. You can use the Typography component to workaround the issue:
Would read:
There is a flexbox bug that prevents text-overflow: ellipse working in combination with a flexbox layout. You can use the Typography component to workaround the issue:

You are right. Should it be prevents x from working in combination ?

The word combination bloats the sentence so:
There is a flexbox bug that prevents text-overflow: ellipse from working in a flexbox layout. You can use the Typography component to workaround this issue:
Is probably easier to read.

👌

@joshwooding Do you want to open a pull request? :)

@joshwooding  Can you provide an example of the Typography workaround ?


Add Go counting to PHP

@camilopayan are you interested in completing this work? There have been some breaking changes in the directory structure and the continuous integration tests that would need some work to update.

@neenjaw I'll handle this one this weekend.

@neenjawTôi sẽ giải quyết vấn đề này vào cuối tuần này.

It's been a year... Are you still there?

Given this exercise is now severely out of date with respect to the changes that came with the current version of Exercism, I am converting this PR to draft as it is not ready for review/merge.  Will re-review at a later time, if no activity, will close.

No activity


[SUPPORT] 0.6.0 - using keytab authentication gives issues

Describe the problem you faced
While uplifting deltastreamer from 0.5.3 to 0.6.0, I can no longer use --principal and --keytab start parameters as it results in error
java.util.ServiceConfigurationError: org.apache.hadoop.security.token.TokenIdentifier: Provider org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier not found
To Reproduce
Steps to reproduce the behavior:

Start the job with the following
spark-submit --master yarn --deploy-mode cluster --conf spark.app.name=hudi_impl2 --conf spark.yarn.maxAppAttempts=4  --conf spark.yarn.am.attemptFailuresValidityInterval=1h  --conf spark.yarn.executor.failuresValidityInterval=1h  --conf spark.task.maxFailures=8  --principal iga@XXX  --keytab /home/director/XXX.keytab  --class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer  --jars hdfs:///app/ingestion/hudi/v0.3/hive-service-2.1.1-cdh6.1.0.jar,hdfs:///app/ingestion/hudi/v0.3/hive-jdbc-2.1.1-cdh6.1.0.jar  hdfs:///app/ingestion/hudi/v0.6.0/hudi-utilities-bundle_2.11-0.6.0.jar  --props hdfs:///app/ingestion/impl2/hudi-default.properties  --target-base-path adl://XXX.azuredatalakestore.net/impl2  --table-type COPY_ON_WRITE  --source-class org.apache.hudi.utilities.sources.AvroKafkaSource  --target-table impl2  --schemaprovider-class org.apache.hudi.utilities.schema.FilebasedSchemaProvider  --enable-hive-sync  --source-ordering-field commit_messageid

The job will fail, with ServiceConfigurationError errror. The same command, just with 0.5.3 (or earlier) hudi-utilities jar, will work fine.
The contents of hudi-default.properties:
hoodie.datasource.write.recordkey.field=commit_ts
hoodie.datasource.write.partitionpath.field=commit_partition

hoodie.deltastreamer.schemaprovider.source.schema.file=hdfs:///app/ingestion/impl2/flights.avsc
hoodie.deltastreamer.schemaprovider.target.schema.file=hdfs:///app/ingestion/impl2/flights.avsc
schema.registry.url=http://streaming-schema-registry:8081

# Kafka settings
bootstrap.servers=streaming-kafka-broker-1:9092,streaming-kafka-broker-2:9092,streaming-kafka-broker-3:9092
group.id=hudi_adls
hoodie.deltastreamer.source.kafka.topic=flights
enable.auto.commit=false
auto.offset.reset=earliest

#Hive settings
hoodie.datasource.hive_sync.database=data
hoodie.datasource.hive_sync.table=flights_impl2_tests
<EMAIL_ADDRESS>hoodie.datasource.hive_sync.assume_date_partitioning=false
hoodie.datasource.hive_sync.partition_fields=commit_partition
hoodie.datasource.hive_sync.useJdbc=false

hoodie.datasource.write.precombine.field=commit_partition

Expected behavior
A successful run
Environment Description
Hudi version : 0.6.0
Spark version : 2.4.0-cdh6.1.0
Hive version : 2.1.1-cdh6.1.0
Hadoop version : 3.0.0-cdh6.1.0
Storage (HDFS/S3/GCS..) : ADLS
Running on Docker? (yes/no) : no
Additional context
The 0.6.0 release was compiled with a change to hudi-utilities-bundle/pom.xml adding the following:
 <relocation>
                  <pattern>org.eclipse.jetty.</pattern>
                 <shadedPattern>org.apache.hudi.org.apache.jetty.</shadedPattern>
 </relocation>

Stacktrace
20/09/22 07:34:49 INFO yarn.Client: Application report for application_1600754659032_0016 (state: ACCEPTED)
20/09/22 07:34:50 INFO yarn.Client: Application report for application_1600754659032_0016 (state: ACCEPTED)
20/09/22 07:34:51 INFO yarn.Client: Application report for application_1600754659032_0016 (state: ACCEPTED)
20/09/22 07:34:52 INFO yarn.Client: Application report for application_1600754659032_0016 (state: ACCEPTED)
20/09/22 07:34:53 INFO yarn.Client: Application report for application_1600754659032_0016 (state: ACCEPTED)
20/09/22 07:34:54 INFO yarn.Client: Application report for application_1600754659032_0016 (state: ACCEPTED)
20/09/22 07:34:55 INFO yarn.Client: Application report for application_1600754659032_0016 (state: FAILED)
20/09/22 07:34:55 INFO yarn.Client:
         client token: N/A
         diagnostics: Application application_1600754659032_0016 failed 2 times in previous 3600000 milliseconds due to AM Container for appattempt_1600754659032_0016_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2020-09-22 07:34:54.678]Exception from container-launch.
Container id: container_1600754659032_0016_02_000001
Exit code: 1
Shell output: main : command provided 1
main : run as user is iga
main : requested yarn user is iga
Getting exit code file...
Creating script paths...
Writing pid file...
Writing to tmp file /data3/yarn/nm/nmPrivate/application_1600754659032_0016/container_1600754659032_0016_02_000001/container_1600754659032_0016_02_000001.pid.tmp
Writing to cgroup task files...
Creating local dirs...
Launching container...
Getting exit code file...
Creating script paths...

[2020-09-22 07:34:54.681]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
ovider org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier not found
        at java.util.ServiceLoader.fail(ServiceLoader.java:239)
        at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372)
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
        at org.apache.hadoop.security.token.Token.getClassForIdentifier(Token.java:150)
        at org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:171)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1$$anonfun$4.apply(HadoopFSDelegationTokenProvider.scala:116)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1$$anonfun$4.apply(HadoopFSDelegationTokenProvider.scala:116)
        at scala.collection.TraversableLike$$anonfun$filterImpl$1.apply(TraversableLike.scala:248)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:247)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.AbstractTraversable.filter(Traversable.scala:104)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1.apply(HadoopFSDelegationTokenProvider.scala:115)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1.apply(HadoopFSDelegationTokenProvider.scala:111)
        at scala.Option.flatMap(Option.scala:171)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider.getTokenRenewalInterval(HadoopFSDelegationTokenProvider.scala:111)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider.obtainDelegationTokens(HadoopFSDelegationTokenProvider.scala:53)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager$$anonfun$obtainDelegationTokens$2.apply(HadoopDelegationTokenManager.scala:132)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager$$anonfun$obtainDelegationTokens$2.apply(HadoopDelegationTokenManager.scala:130)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.MapLike$DefaultValuesIterable.foreach(MapLike.scala:206)
        at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
        at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager.obtainDelegationTokens(HadoopDelegationTokenManager.scala:130)
        at org.apache.spark.deploy.yarn.security.YARNHadoopDelegationTokenManager.obtainDelegationTokens(YARNHadoopDelegationTokenManager.scala:59)
        at org.apache.spark.deploy.yarn.security.AMCredentialRenewer$$anon$3.run(AMCredentialRenewer.scala:160)
        at org.apache.spark.deploy.yarn.security.AMCredentialRenewer$$anon$3.run(AMCredentialRenewer.scala:157)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
        at org.apache.spark.deploy.yarn.security.AMCredentialRenewer.obtainTokensAndScheduleRenewal(AMCredentialRenewer.scala:157)
        at org.apache.spark.deploy.yarn.security.AMCredentialRenewer.start(AMCredentialRenewer.scala:98)
        at org.apache.spark.deploy.yarn.ApplicationMaster.<init>(ApplicationMaster.scala:113)
        at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:806)
        at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)

[2020-09-22 07:34:54.682]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
ovider org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier not found
        at java.util.ServiceLoader.fail(ServiceLoader.java:239)
        at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372)
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
        at org.apache.hadoop.security.token.Token.getClassForIdentifier(Token.java:150)
        at org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:171)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1$$anonfun$4.apply(HadoopFSDelegationTokenProvider.scala:116)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1$$anonfun$4.apply(HadoopFSDelegationTokenProvider.scala:116)
        at scala.collection.TraversableLike$$anonfun$filterImpl$1.apply(TraversableLike.scala:248)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:247)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.AbstractTraversable.filter(Traversable.scala:104)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1.apply(HadoopFSDelegationTokenProvider.scala:115)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1.apply(HadoopFSDelegationTokenProvider.scala:111)
        at scala.Option.flatMap(Option.scala:171)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider.getTokenRenewalInterval(HadoopFSDelegationTokenProvider.scala:111)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider.obtainDelegationTokens(HadoopFSDelegationTokenProvider.scala:53)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager$$anonfun$obtainDelegationTokens$2.apply(HadoopDelegationTokenManager.scala:132)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager$$anonfun$obtainDelegationTokens$2.apply(HadoopDelegationTokenManager.scala:130)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.MapLike$DefaultValuesIterable.foreach(MapLike.scala:206)
        at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
        at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager.obtainDelegationTokens(HadoopDelegationTokenManager.scala:130)
        at org.apache.spark.deploy.yarn.security.YARNHadoopDelegationTokenManager.obtainDelegationTokens(YARNHadoopDelegationTokenManager.scala:59)
        at org.apache.spark.deploy.yarn.security.AMCredentialRenewer$$anon$3.run(AMCredentialRenewer.scala:160)
        at org.apache.spark.deploy.yarn.security.AMCredentialRenewer$$anon$3.run(AMCredentialRenewer.scala:157)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
        at org.apache.spark.deploy.yarn.security.AMCredentialRenewer.obtainTokensAndScheduleRenewal(AMCredentialRenewer.scala:157)
        at org.apache.spark.deploy.yarn.security.AMCredentialRenewer.start(AMCredentialRenewer.scala:98)
        at org.apache.spark.deploy.yarn.ApplicationMaster.<init>(ApplicationMaster.scala:113)
        at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:806)
        at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)

Seeing the error message is that the hbase verification failed, you can try to put the hbase-site.xml file in the resource and package it for execution. Currently, hbase connection does not open other configurations to pass in externally.

Is hbase used for hudi process? hbase service was added to the cluster only lately for the use of other project, and should not have anything to do with this stream.

Do you use hbase index of hudi? Can you show the configuration of hudi-default.properties?

I have added the properties file contents to issue description

@eigakow Can you provide the stack trace which shows the originator class. If the Hive-Sync is affected in 0.6.0, we should see this exception being raised from a hoodie hive-sync code path.

Below the stack trace of the failure. I have removed hive-sync option to simplify troubleshooting.
20/09/28 07:27:54 WARN deltastreamer.SchedulerConfGenerator: Job Scheduling Configs will not be in effect as spark.scheduler.mode is not set to FAIR at instantiation time. Continuing without scheduling configs
20/09/28 07:27:54 INFO spark.SparkContext: Running Spark version 2.4.0-cdh6.1.0
20/09/28 07:27:54 INFO spark.SparkContext: Submitted application: delta-streamer_impl2
20/09/28 07:27:54 INFO spark.SecurityManager: Changing view acls to: director,iga
20/09/28 07:27:54 INFO spark.SecurityManager: Changing modify acls to: director,iga
20/09/28 07:27:54 INFO spark.SecurityManager: Changing view acls groups to:
20/09/28 07:27:54 INFO spark.SecurityManager: Changing modify acls groups to:
20/09/28 07:27:54 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(director, iga); groups with view permissions: Set(); users  with modify permissions: Set(director, iga); groups with modify permissions: Set()
20/09/28 07:27:54 INFO util.Utils: Successfully started service 'sparkDriver' on port 38195.
20/09/28 07:27:54 INFO spark.SparkEnv: Registering MapOutputTracker
20/09/28 07:27:54 INFO spark.SparkEnv: Registering BlockManagerMaster
20/09/28 07:27:54 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
20/09/28 07:27:54 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
20/09/28 07:27:54 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-2d0abd3b-1755-43bc-9313-36a9cb19e802
20/09/28 07:27:54 INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MB
20/09/28 07:27:54 INFO spark.SparkEnv: Registering OutputCommitCoordinator
20/09/28 07:27:54 INFO util.log: Logging initialized @3357ms
20/09/28 07:27:54 INFO server.Server: jetty-9.3.z-SNAPSHOT
20/09/28 07:27:54 INFO server.Server: Started @3430ms
20/09/28 07:27:54 INFO server.AbstractConnector: Started ServerConnector@108d55c4{HTTP/1.1,[http/1.1]}{<IP_ADDRESS>:4040}
20/09/28 07:27:54 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2eaef76d{/jobs,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c6da8bb{/jobs/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3bae64d0{/jobs/job,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1fba386c{/jobs/job/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7e736350{/stages,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@36b310aa{/stages/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@76c387f9{/stages/stage,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@736048ed{/stages/stage/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1976f537{/stages/pool,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@45f421c{/stages/pool/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1816e24a{/storage,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6940f685{/storage/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@47b179d7{/storage/rdd,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@39008c9f{/storage/rdd/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@162b3d47{/environment,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@783115d9{/environment/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3402b4c9{/executors,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@95bb2a2{/executors/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@46d567cb{/executors/threadDump,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@363a3d15{/executors/threadDump/json,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@173f73e7{/static,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@387bf2d9{/,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@74aa9c72{/api,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c1fca2a{/jobs/job/kill,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5533dc72{/stages/stage/kill,null,AVAILABLE,@Spark}
20/09/28 07:27:54 INFO ui.SparkUI: Bound SparkUI to <IP_ADDRESS>, and started at http://biedva-worker-1.bigdatapoc.local:4040
20/09/28 07:27:54 INFO Configuration.deprecation: mapred.output.compression.codec is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.codec
20/09/28 07:27:54 INFO Configuration.deprecation: mapred.output.compression.type is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.type
20/09/28 07:27:54 INFO Configuration.deprecation: mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
20/09/28 07:27:54 INFO spark.SparkContext: Added JAR hdfs:///app/ingestion/hudi/v0.6.0/hudi-utilities-bundle_2.11-0.6.0.jar at hdfs:///app/ingestion/hudi/v0.6.0/hudi-utilities-bundle_2.11-0.6.0.jar with timestamp<PHONE_NUMBER>902
20/09/28 07:27:54 INFO util.Utils: Using initial executors = 0, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
20/09/28 07:27:55 INFO yarn.Client: Kerberos credentials: principal = iga@XXX, keytab = /home/director/XXX.keytab
20/09/28 07:27:55 INFO client.RMProxy: Connecting to ResourceManager at biedva-master-1.bigdatapoc.local/<IP_ADDRESS>:8032
20/09/28 07:27:55 INFO yarn.Client: Requesting a new application from cluster with 6 NodeManagers
20/09/28 07:27:55 INFO conf.Configuration: resource-types.xml not found
20/09/28 07:27:55 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
20/09/28 07:27:55 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (28672 MB per container)
20/09/28 07:27:55 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
20/09/28 07:27:55 INFO yarn.Client: Setting up container launch context for our AM
20/09/28 07:27:55 INFO yarn.Client: Setting up the launch environment for our AM container
20/09/28 07:27:55 INFO yarn.Client: Preparing resources for our AM container
20/09/28 07:27:55 INFO yarn.Client: To enable the AM to login from keytab, credentials are being copied over to the AM via the YARN Secure Distributed Cache.
20/09/28 07:27:55 INFO yarn.Client: Uploading resource file:/home/director/XXX.keytab -> hdfs://nameservice1/user/iga/.sparkStaging/application_1601272990574_0015/XXX.keytab
20/09/28 07:27:55 INFO yarn.Client: Source and destination file systems are the same. Not copying hdfs://nameservice1/app/ingestion/hudi/v0.3/hive-service-2.1.1-cdh6.1.0.jar
20/09/28 07:27:55 INFO yarn.Client: Source and destination file systems are the same. Not copying hdfs://nameservice1/app/ingestion/hudi/v0.3/hive-jdbc-2.1.1-cdh6.1.0.jar
20/09/28 07:27:55 INFO yarn.Client: Uploading resource file:/tmp/spark-431edee5-628f-4887-900b-ee4a354c574d/__spark_conf__2055697676300668872.zip -> hdfs://nameservice1/user/iga/.sparkStaging/application_1601272990574_0015/__spark_conf__.zip
20/09/28 07:27:55 INFO spark.SecurityManager: Changing view acls to: director,iga
20/09/28 07:27:55 INFO spark.SecurityManager: Changing modify acls to: director,iga
20/09/28 07:27:55 INFO spark.SecurityManager: Changing view acls groups to:
20/09/28 07:27:55 INFO spark.SecurityManager: Changing modify acls groups to:
20/09/28 07:27:55 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(director, iga); groups with view permissions: Set(); users  with modify permissions: Set(director, iga); groups with modify permissions: Set()
20/09/28 07:27:55 INFO conf.HiveConf: Found configuration file file:/etc/hive/conf.cloudera.hive/hive-site.xml
20/09/28 07:27:55 INFO security.HadoopFSDelegationTokenProvider: getting token for: DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_1511656334_1, ugi=iga@XXX (auth:KERBEROS)]]
20/09/28 07:27:55 INFO hdfs.DFSClient: Created token for iga: HDFS_DELEGATION_TOKEN owner=iga@XXX, renewer=yarn, realUser=, issueDate=1601278075790, maxDate=1601882875790, sequenceNumber=26153, masterKeyId=880 on ha-hdfs:nameservice1
20/09/28 07:27:55 INFO security.HadoopFSDelegationTokenProvider: getting token for: DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_1511656334_1, ugi=iga@XXX (auth:KERBEROS)]]
20/09/28 07:27:55 INFO hdfs.DFSClient: Created token for iga: HDFS_DELEGATION_TOKEN owner=iga@XXX, renewer=iga, realUser=, issueDate=1601278075804, maxDate=1601882875804, sequenceNumber=26154, masterKeyId=880 on ha-hdfs:nameservice1
20/09/28 07:27:55 INFO yarn.Client: Deleted staging directory hdfs://nameservice1/user/iga/.sparkStaging/application_1601272990574_0015
20/09/28 07:27:55 ERROR spark.SparkContext: Error initializing SparkContext.
java.util.ServiceConfigurationError: org.apache.hadoop.security.token.TokenIdentifier: Provider org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier not found
        at java.util.ServiceLoader.fail(ServiceLoader.java:239)
        at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372)
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
        at org.apache.hadoop.security.token.Token.getClassForIdentifier(Token.java:150)
        at org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:171)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1$$anonfun$4.apply(HadoopFSDelegationTokenProvider.scala:116)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1$$anonfun$4.apply(HadoopFSDelegationTokenProvider.scala:116)
        at scala.collection.TraversableLike$$anonfun$filterImpl$1.apply(TraversableLike.scala:248)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:247)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.AbstractTraversable.filter(Traversable.scala:104)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1.apply(HadoopFSDelegationTokenProvider.scala:115)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1.apply(HadoopFSDelegationTokenProvider.scala:111)
        at scala.Option.flatMap(Option.scala:171)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider.getTokenRenewalInterval(HadoopFSDelegationTokenProvider.scala:111)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider.obtainDelegationTokens(HadoopFSDelegationTokenProvider.scala:53)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager$$anonfun$obtainDelegationTokens$2.apply(HadoopDelegationTokenManager.scala:132)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager$$anonfun$obtainDelegationTokens$2.apply(HadoopDelegationTokenManager.scala:130)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.MapLike$DefaultValuesIterable.foreach(MapLike.scala:206)
        at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
        at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager.obtainDelegationTokens(HadoopDelegationTokenManager.scala:130)
        at org.apache.spark.deploy.yarn.security.YARNHadoopDelegationTokenManager.obtainDelegationTokens(YARNHadoopDelegationTokenManager.scala:59)
        at org.apache.spark.deploy.yarn.Client.setupSecurityToken(Client.scala:322)
        at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:1028)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:180)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:179)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:507)
        at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
        at org.apache.hudi.utilities.UtilHelpers.buildSparkContext(UtilHelpers.java:214)
        at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:457)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
20/09/28 07:27:55 INFO server.AbstractConnector: Stopped Spark@108d55c4{HTTP/1.1,[http/1.1]}{<IP_ADDRESS>:4040}
20/09/28 07:27:55 INFO ui.SparkUI: Stopped Spark web UI at http://biedva-worker-1.bigdatapoc.local:4040
20/09/28 07:27:55 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
20/09/28 07:27:55 INFO cluster.YarnClientSchedulerBackend: Stopped
20/09/28 07:27:55 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
20/09/28 07:27:55 INFO memory.MemoryStore: MemoryStore cleared
20/09/28 07:27:55 INFO storage.BlockManager: BlockManager stopped
20/09/28 07:27:55 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
20/09/28 07:27:55 WARN metrics.MetricsSystem: Stopping a MetricsSystem that is not running
20/09/28 07:27:55 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
20/09/28 07:27:55 INFO spark.SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.util.ServiceConfigurationError: org.apache.hadoop.security.token.TokenIdentifier: Provider org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier not found
        at java.util.ServiceLoader.fail(ServiceLoader.java:239)
        at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372)
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
        at org.apache.hadoop.security.token.Token.getClassForIdentifier(Token.java:150)
        at org.apache.hadoop.security.token.Token.decodeIdentifier(Token.java:171)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1$$anonfun$4.apply(HadoopFSDelegationTokenProvider.scala:116)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1$$anonfun$4.apply(HadoopFSDelegationTokenProvider.scala:116)
        at scala.collection.TraversableLike$$anonfun$filterImpl$1.apply(TraversableLike.scala:248)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:247)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.AbstractTraversable.filter(Traversable.scala:104)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1.apply(HadoopFSDelegationTokenProvider.scala:115)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider$$anonfun$getTokenRenewalInterval$1.apply(HadoopFSDelegationTokenProvider.scala:111)
        at scala.Option.flatMap(Option.scala:171)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider.getTokenRenewalInterval(HadoopFSDelegationTokenProvider.scala:111)
        at org.apache.spark.deploy.security.HadoopFSDelegationTokenProvider.obtainDelegationTokens(HadoopFSDelegationTokenProvider.scala:53)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager$$anonfun$obtainDelegationTokens$2.apply(HadoopDelegationTokenManager.scala:132)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager$$anonfun$obtainDelegationTokens$2.apply(HadoopDelegationTokenManager.scala:130)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.MapLike$DefaultValuesIterable.foreach(MapLike.scala:206)
        at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
        at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
        at org.apache.spark.deploy.security.HadoopDelegationTokenManager.obtainDelegationTokens(HadoopDelegationTokenManager.scala:130)
        at org.apache.spark.deploy.yarn.security.YARNHadoopDelegationTokenManager.obtainDelegationTokens(YARNHadoopDelegationTokenManager.scala:59)
        at org.apache.spark.deploy.yarn.Client.setupSecurityToken(Client.scala:322)
        at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:1028)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:180)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:179)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:507)
        at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
        at org.apache.hudi.utilities.UtilHelpers.buildSparkContext(UtilHelpers.java:214)
        at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:457)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
20/09/28 07:27:55 INFO util.ShutdownHookManager: Shutdown hook called
20/09/28 07:27:55 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-431edee5-628f-4887-900b-ee4a354c574d
20/09/28 07:27:55 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-e18d8494-7656-405c-87e2-0c3d456a08e6

Exactly the same start command, but using hudi 0.5.2 is working successfully, therefore I do not think it is related to the environment, but to the changes that were implemented between 0.5.2 and 0.6.0 versions of hudi.

@eigakow : 0.6. hudi bundle comes with shaded hbase jars whereas 0.5.3 does not have it. The version of hbase is : 1.2.3. I am not sure if this is causing the issue ? Can you try rebuilding Hudi with the version of hbase you are using and see if the problem goes away ?

I've seen this before. If you use a kerberized hdfs environmental, you need to add hbase-client jar to your spark lib.
Even if you don't use hBase as hudi index, hadoop also need to check whether this class exists. I feel that this may be a bug in Hadoop.

@eigakow  were you able to try this with a different base version and rebuilding Hudi ?

Since i deployed hudi jar in HIVE auxlib to be albe to request hive table without adding each time the JAR in the session. We have some clients who now systematicly get this token hbase error... even if hbase isnt installed on our cluster.
I think it might be a serious problem in hudi 0.6.0

@spyzzz thanks for letting us know. This will be in the next release

@bhasudha
I have a new problem. If the timeline server is deployed separately in the kerberized hdfs environment, and the service is called after the token refresh time of Kerberos, the authorization failure exception will appear.
WARN security.UserGroupInformation: Exception encountered while running the renewal command for<EMAIL_ADDRESS>(TGT end time:1609920920000, renewalFailures: org.apache.hadoop.metrics2.lib.MutableGaugeInt@7964ce9e,renewalFailuresTotal: org.apache.hadoop.metrics2.lib.MutableGaugeLong@7d5b456a)
ExitCodeException exitCode=1: kinit: KDC can't fulfill requested option while renewing credentials

I think this is a bug. If the timeline server is in a kerberized hdfs environment, it should refresh the credentials regularly. Otherwise, when the service is not called for a long time, the authorization will be invalid and the service will not be available.
If you have time, can you confirm this is a problem please. If it is a bug, I can propose an issue on JIRA and try to fix it.

@bhasudha
I have a new problem. If the timeline server is deployed separately in the kerberized hdfs environment, and the service is called after the token refresh time of Kerberos, the authorization failure exception will appear.
WARN security.UserGroupInformation: Exception encountered while running the renewal command for<EMAIL_ADDRESS>(TGT end time:1609920920000, renewalFailures: org.apache.hadoop.metrics2.lib.MutableGaugeInt@7964ce9e,renewalFailuresTotal: org.apache.hadoop.metrics2.lib.MutableGaugeLong@7d5b456a)
ExitCodeException exitCode=1: kinit: KDC can't fulfill requested option while renewing credentials

I think this is a bug. If the timeline server is in a kerberized hdfs environment, it should refresh the credentials regularly. Otherwise, when the service is not called for a long time, the authorization will be invalid and the service will not be available.
If you have time, can you confirm this is a problem please. If it is a bug, I can propose an issue on JIRA and try to fix it.

@bhasudha
I have a new problem. If the timeline server is deployed separately in the kerberized hdfs environment, and the service is called after the token refresh time of Kerberos, the authorization failure exception will appear.
WARN security.UserGroupInformation: Exception encountered while running the renewal command for<EMAIL_ADDRESS>(TGT end time:1609920920000, renewalFailures: org.apache.hadoop.metrics2.lib.MutableGaugeInt@7964ce9e,renewalFailuresTotal: org.apache.hadoop.metrics2.lib.MutableGaugeLong@7d5b456a)
ExitCodeException exitCode=1: kinit: KDC can't fulfill requested option while renewing credentials

I think this is a bug. If the timeline server is in a kerberized hdfs environment, it should refresh the credentials regularly. Otherwise, when the service is not called for a long time, the authorization will be invalid and the service will not be available.
If you have time, can you confirm this is a problem please. If it is a bug, I can propose an issue on JIRA and try to fix it.

@bhasudha
I have a new problem. If the timeline server is deployed separately in the kerberized hdfs environment, and the service is called after the token refresh time of Kerberos, the authorization failure exception will appear.
WARN security.UserGroupInformation: Exception encountered while running the renewal command for<EMAIL_ADDRESS>(TGT end time:1609920920000, renewalFailures: org.apache.hadoop.metrics2.lib.MutableGaugeInt@7964ce9e,renewalFailuresTotal: org.apache.hadoop.metrics2.lib.MutableGaugeLong@7d5b456a)
ExitCodeException exitCode=1: kinit: KDC can't fulfill requested option while renewing credentials

I think this is a bug. If the timeline server is in a kerberized hdfs environment, it should refresh the credentials regularly. Otherwise, when the service is not called for a long time, the authorization will be invalid and the service will not be available.
If you have time, can you confirm this is a problem please. If it is a bug, I can propose an issue on JIRA and try to fix it.

@cdmikechen are you deploying the timeline server separately on its own? That's interesting. Don't think anyone has tested it this way. cc @leesf as well, in case he has done something like this.

@cdmikechen are you deploying the timeline server separately on its own? That's interesting. Don't think anyone has tested it this way. cc @leesf as well, in case he has done something like this.

@vinothchandar Yes! I deployed the timeline server separately on its own and let multiple hudi services use this timeline server.
I used an unified timeline server to obtain hudi metadata and hudi task timeline on some customized services in dev and test environments.
At the same time, when the spark service is deployed in k8s environment, there will be some host problems when using the built-in timeline server under specific circumstances. So that I had to deploy the timeline server separately.
Another reason is that I'm considering doing some work about hudi-ui this year. I feel that the independent timeline service may be helpful in identifying hudi tables in a cluster.

@vinothchandar Yes! I deployed the timeline server separately on its own and let multiple hudi services use this timeline server.
I used an unified timeline server to obtain hudi metadata and hudi task timeline on some customized services in dev and test environments.
At the same time, when the spark service is deployed in k8s environment, there will be some host problems when using the built-in timeline server under specific circumstances. So that I had to deploy the timeline server separately.
Another reason is that I'm considering doing some work about hudi-ui this year. I feel that the independent timeline service may be helpful in identifying hudi tables in a cluster.

@n3nash @bhasudha : sorry the thread is bit long. I see some workarounds have been proposed and it worked. But do we need to fixes in Hudi in general? if yes, can you file a jira and close this out.

@n3nash @bhasudha : sorry the thread is bit long. I see some workarounds have been proposed and it worked. But do we need to fixes in Hudi in general? if yes, can you file a jira and close this out.

I feel that the independent timeline service may be helpful in identifying hudi tables in a cluster.
@cdmikechen This is actually very interesting to me too. Can we start a DISCUSS thread around this? we can also embed the Hive MetaStore instance within and make it very simple for everyone to deploy

I feel that the independent timeline service may be helpful in identifying hudi tables in a cluster.
@cdmikechen This is actually very interesting to me too. Can we start a DISCUSS thread around this? we can also embed the Hive MetaStore instance within and make it very simple for everyone to deploy

@cdmikechen : would you mind starting a DISCUSS thread as proposed. many might benefit from this.


error importing cacheable-response

Hi there,
I am facing this issue when trying to use this library:
ERROR in ./node_modules/iltorb/build/bindings/iltorb.node 1:0
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)
@ ./node_modules/iltorb/index.js 10:39-78
@ ./node_modules/compress-brotli/index.js
@ ./node_modules/cacheable-response/index.js
(it also seems iltorb is now depreciated  - and recommends using zlib
Not sure if they are related.)
Any help appreciated !

it seems it is not being used in compress-brotli - can it be removed ?

not sure why you are using webpack with this dependency since this is a backend-side thing 🤔
what's your cacheable-response version?

thx for your reply.
We are using angular universal.
For the server backend :
/**

*** NOTE ON IMPORTING FROM ANGULAR AND NGUNIVERSAL IN THIS FILE ***

If your application uses third-party dependencies, you'll need to
either use Webpack or the Angular CLI's bundleDependencies feature
in order to adequately package them for use on the server without a
node_modules directory.

However, due to the nature of the CLI's bundleDependencies, importing
Angular in this file will create a different instance of Angular than
the version in the compiled application code. This leads to unavoidable
conflicts. Therefore, please do not explicitly import from @angular or
@nguniversal in this file. You can export any needed resources
from your application's main.server.ts file, as seen below with the
import for ngExpressEngine.
*/

Do we need it in compress-brotli if it is depreciated ?

yes cause compress-brotli is not deprecated


hautelook_alice.resolver.bundle is hardcoded to use only default_entity_manager

I'm using multiple entity managers in my application and cannot persist entities that are not defined in the default manager because hautelook_alice.data_fixtures.loader service defines to use the default_entity_manager by hard-coding it in the configuration file.
Is there an easy way to force using mine without changing environments?

An easy way is to override the configuration: if you have one entity manager that can handle everything then override the service using that entity manager. If you don't and need to use different managers depending of the entity persisted you'll have to change two things:

The persister service
The purger

For the first, I think it's rather trivial: check the entity class and depending of it pick manager X or Y. For the later I don't have any idea as we actually rely on Doctrine purgers which takes a single entity manager as an argument. So maybe you'll need to use several purgers at once...
The theory should remain the same regardless of the version you are using, but if you need further help I need more info :)

My scenario covers the second case, every entity manager can handle a collection of different entities.
So if I'm not mistaken, I need to clone and specify my own entity manager:

fidry_alice_data_fixtures.persistence.persister.doctrine.object_manager_persister

and also:

fidry_alice_data_fixtures.doctrine.persister_loader

Which is the service that uses the persister.


adding element default check

Signed-off-by: Marco A. Gutierrez<EMAIL_ADDRESS>🎉 New feature
Related to #287
Summary
There's already a check for the Attribute class so adding a default attribute and check functions for the Element class. To keep consistency with the code from in the Attribute class, we set it to true if the element was set false if it was created by default.
Since there's many cases where elements are added the base case considers that the element was not set by default so the value is true. When an attribute is being added by default, the SetSet(bool _value) should be called and the set attribute set to false, as done in parser.cc.
Thinks to take into account:

elementDescriptions will have the set attribute to true

Checklist

[x] Signed all commits for DCO
[x] Added tests
[ ] Added example and/or tutorial
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[x] Code check passed (In source directory, run sh tools/code_check.sh)
[x] All tests passed (See test coverage)

FYI @seancurtis-tri


added on-http-api-2.0 to requirements.txt for tests

@dalebremner @brianparry @abildw @jlongever @srinia6

removed the accidental commit of the updated submodules

:+1:

:+1:
Thanks Geoff!!

+1

:+1:


share the same paper db with  two different  activites

how to share the same paper db with  two different  activites ?

try to initialize it in the application class

call Paper.init(context) in application class.
call Paper.book(). in whatever activity you want.

Thank you for answer me ,i well try it
נשלח מסמארטפון ה-Samsung Galaxy שלי.
-------- הודעה מקורית --------מאת: Aleksey Masny<EMAIL_ADDRESS>תאריך: 29.8.2017  09:27  (GMT+02:00) אל: pilgr/Paper<EMAIL_ADDRESS>עותק: basemmilhem<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>נושא: Re: [pilgr/Paper] share the same paper db with two different activites (#79)
call Paper.init(context) in application class.
call Paper.book(). in whatever activity you want.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/pilgr/Paper","title":"pilgr/Paper","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/pilgr/Paper"}},"updates":{"snippets":[{"icon":"PERSON","message":"@pilgr in #79: call Paper.init(context) in application class. \r\ncall Paper.book(). in whatever activity you want."}],"action":{"name":"View Issue","url":"https://github.com/pilgr/Paper/issues/79#issuecomment-325567625"}}}


Add duration as a type to JTS

Aligns with XSD, exists in SQL.

+1

+1
I added duration to my usage of JTS, but I'd prefer not to deviate from the spec.

@back5576 could you share an example of how you implemented it?

For spec here:
https://github.com/dataprotocols/dataprotocols/blob/gh-pages/json-table-schema/index.md
I named the new datatype "duration" and values are in days.

It's not a spec but we should probably have something that behave like Python Pandas to_timedelta
$ ipython
Python 3.4.3 |Anaconda 2.3.0 (x86_64)| (default, Mar  6 2015, 12:07:41)
Type "copyright", "credits" or "license" for more information.

IPython 4.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import pandas as pd

In [2]: pd.to_timedelta("0 days 01:03:30.24")
Out[2]: Timedelta('0 days 01:03:30.240000')

ISO 8601 also define a period (duration)
https://en.wikipedia.org/wiki/ISO_8601

+1
We needed a period type in an application visualizing statistical output tables. Actually many if not most statistical output tables contain periods as they concern quarters, months, years. For visualizing we wanted to differentiate between periodical figures belonging to one specific date (e.g. population the January 1) or belonging to a period (e.g. inflation).  We decided to use the regular date field allowing also the ISO 8601 period specifier referenced by @femtotrader. The first quarter of 2015 can then be represented using 2015-01/P3M; the first month by 2015-01 or 2015-01/P1M; the complete year 2015 by 2015 or 2015/P1Y.
What I call a period above is a period between two specific dates. It is not a pure period such as 2 days and 3 hours.
In our javascript we use moment.js and twix.js. We wrote a function that translates the input strings to twix objects.
We actually also accept 2015M[0]1 for months; 2015Q[0]1for quarters.

OK, I propose we go with the XSD style duration. @femtotrader would you like to submit a PR?
@djvanderlaan you are quite right re "points in time" vs duration. I like your method of being explicit though generally i would just interpret as a point in time e.g. 2015-Q1 or similar. Also, I'm interested to hear about your project - is it public?

Sorry @rgrp I'm quite busy these days (probably until december). So don't expect a PR from me. Sorry.

@femtotrader no problem - and thanks for all the contributions so far! We'll look to get that PR in.

OK, about to submit fix for this. Two things:

I'm adding duration
I'm not adding time interval atm - i.e. https://en.wikipedia.org/wiki/ISO_8601#Time_intervals

@djvanderlaan do you want time interval as well. If so can you open a separate issue and re-summarize the reason for adding


package dependencies as tarballs

Following up on the conversation at #8, this PR introduces the tarball as a file in the gemspec, so that it's delivered as part of the gem file download from rubygems.org.
mini_portile is smart enough to use this file instead of re-downloading it, which will be a better experience for users and save hosting costs for this project.

Update: I've added a commit to add the patches to the gemfile.

Hi @flavorjones,
Thank you!
I also believe that our custom clean-up should not cause any issues once the Ruby Gem is installed, very nice.
Krzysztof

Hi @flavorjones and @stanhu, sorry to common on the closed Pull Request!
Question to the two of you - are you also having some build time issues? For me, a build after a fresh checkout of the repository seems to require the archive to be always present to succeed, as per:
$ rake compile -- --use-system-libraries
Building ruby-magic using system libraries.
checking for ruby.h... yes
checking for rb_thread_call_without_gvl()... yes
checking for rb_thread_blocking_region()... no
checking for magic.h... yes
checking for -lmagic... yes
checking for magic_getpath()... yes
checking for magic_getflags()... yes
checking for utime.h... yes
checking for sys/types.h... yes
checking for sys/time.h... yes
checking for utime()... yes
checking for utimes()... yes
creating extconf.h
creating Makefile
compiling ../../../../ext/magic/functions.c
(...)
linking shared-object magic/magic.bundle
rake aborted!
Don't know how to build task 'ports/archives/file-5.39.tar.gz' (See the list of available tasks with `rake --tasks`)

Tasks: TOP => compile => compile:x86_64-darwin19 => compile:magic:x86_64-darwin19 => copy:magic:x86_64-darwin19:3.0.0 => tmp/x86_64-darwin19/stage/ports/archives/file-5.39.tar.gz
(See full trace by running task with --trace)

Krzysztof

Yes, sorry, I'm seeing this problem, too. I didn't test with the --use-system-libraries flag. I'll add a commit to #17 to fix.

Hi @flavorjones,
For some reason, I am still having the same problem when using --use-system-libraries locally.
Even with a fresh  checkout followed by bundle install, this problem will manifest itself. I am not sure why I have this issue but we haven't seen it in the CI.
Krzysztof

@kwilczynski I think the problem is happening in CI: https://github.com/kwilczynski/ruby-magic/runs/2337765182
I think this line is causing the issue: https://github.com/kwilczynski/ruby-magic/blob/af160d9f7ba7e7e7bd6e5c3ebf456c708627785b/ruby-magic.gemspec#L50
When --use-system-libraries is used, the .tar.gz isn't downloaded. Can we fix this by dynamically adding this dependency?

Hi @stanhu,
We should fix it, indeed. I am wondering, whether the approach that Nokogiri uses (dedicated Rake task) would be more appropriate here? What do you think @flavorjones?
Interestingly, I assumed that the Ruby Gem package task would not run (lazy execution) unless it's called, and when I only use clobber or compile then only these tasks would be used - which is fine for local development.
Krzysztof