runtime: ThreadSanitizer CHECK failed

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
$ go version
go version go1.11 linux/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/benesch/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/benesch/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build445193843=/tmp/go-build -gno-record-gcc-switches"

What did you do?
go get -u github.com/cockroachdb/cockroach
cd $(go env GOPATH)/src/github.com/cockroachdb/cockroach
make testrace PKG=./pkg/sql/logictest TESTS=TestLogic

This builds some C and C++ dependencies and ultimately runs:
go test -race -tags ' make x86_64_linux_gnu' -ldflags '-X github.com/cockroachdb/cockroach/pkg/build.typ=development -extldflags "" -X "github.com/cockroachdb/cockroach/pkg/build.tag=v2.2.0-alpha.00000000-793-g33c7d27d82-dirty" -X "github.com/cockroachdb/cockroach/pkg/build.rev=33c7d27d8216b543ac77f0fe39d440ebebfa9e70" -X "github.com/cockroachdb/cockroach/pkg/build.cgoTargetTriple=x86_64-linux-gnu"  ' -run "TestLogic"  -timeout 25m ./pkg/sql/logictest

What did you expect to see?
Either a successful test run or an actual race.
What did you see instead?
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:380 "((v)) > ((0))" (0x0, 0x0)

I tried to dig into this, but gotsan.cc is apparently generated deep inside the build process for the race detector and it wasn't clear to me how to generate a gotsan.cc that matched my Go version.
Totally possible that this our (CockroachDB's) bug, but there's not a lot here for me to go on, so figured I'd file here and see if someone with more expertise in the race detector had any insight.

And here's a different tsan failure I just observed with the same command:
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:828 "((get_block(bi))) == ((0))" (0x603f, 0x0)

And another slightly different presentation:
0x00000000026cb4e6 in __tsan::SyncClock::Unshare(__tsan::DenseSlabAllocCache*) ()
(gdb) bt
#0  0x00000000026cb4e6 in __tsan::SyncClock::Unshare(__tsan::DenseSlabAllocCache*) ()
#1  0x00000000026cb773 in __tsan::ThreadClock::ReleaseStore(__tsan::DenseSlabAllocCache*, __tsan::SyncClock*) ()
#2  0x00000000026dd43a in __tsan_go_atomic32_compare_exchange ()
#3  0x000000000062cfc3 in racecall () at /usr/local/go/src/runtime/race_amd64.s:381
#4  0x00007ffff6a906d0 in ?? ()

CC @dvyukov

These instructions fail for me:
cockroach$ go test -race -tags ' make x86_64_linux_gnu' -ldflags '-X github.com/cockroachdb/cockroach/pkg/build.typ=development -extldflags "" -X "github.com/cockroachdb/cockroach/pkg/build.tag=v2.2.0-alpha.00000000-793-g33c7d27d82-dirty" -X "github.com/cockroachdb/cockroach/pkg/build.rev=33c7d27d8216b543ac77f0fe39d440ebebfa9e70" -X "github.com/cockroachdb/cockroach/pkg/build.cgoTargetTriple=x86_64-linux-gnu"  ' -run "TestLogic"  -timeout 25m ./pkg/sql/logictest
# github.com/cockroachdb/cockroach/pkg/sql/lex
pkg/sql/lex/predicates.go:60:14: undefined: reservedKeywords
FAIL	github.com/cockroachdb/cockroach/pkg/sql/logictest [build failed]

This is on commit:
cockroach$ git show HEAD
commit 418b9169b547069cd43252055196204af8f18040 (HEAD -> master, origin/staging, origin/master, origin/HEAD)
Merge: a9c7295dd6 5504e3f9ae
Author: craig[bot]<EMAIL_ADDRESS>Date:   Fri Sep 14 01:24:32 2018 +0000

    Merge #29163

Yes, I’m afraid you’ll need to run the make test command I posted at
least once to generate some Go code and build some C/C++ dependencies.
After that, you can run just the go test command.
I apologize that the surface area of this reproduction is so large, but I
really don’t know how to whittle it down. We run the race detector on every
other CockroachDB package with no problems. Let me know if there’s anything
I can do to assist in tracking this down!
On Fri, Sep 14, 2018 at 6:00 AM Dmitry Vyukov<EMAIL_ADDRESS>wrote:

These instructions fail for me:
cockroach$ go test -race -tags ' make x86_64_linux_gnu' -ldflags '-X github.com/cockroachdb/cockroach/pkg/build.typ=development -extldflags "" -X "github.com/cockroachdb/cockroach/pkg/build.tag=v2.2.0-alpha.00000000-793-g33c7d27d82-dirty" -X "github.com/cockroachdb/cockroach/pkg/build.rev=33c7d27d8216b543ac77f0fe39d440ebebfa9e70" -X "github.com/cockroachdb/cockroach/pkg/build.cgoTargetTriple=x86_64-linux-gnu"  ' -run "TestLogic"  -timeout 25m ./pkg/sql/logictest
github.com/cockroachdb/cockroach/pkg/sql/lex
pkg/sql/lex/predicates.go:60:14 http://github.com/cockroachdb/cockroach/pkg/sql/lexpkg/sql/lex/predicates.go:60:14: undefined: reservedKeywords
FAIL	github.com/cockroachdb/cockroach/pkg/sql/logictest [build failed]
This is on commit:
cockroach$ git show HEAD
commit 418b9169b547069cd43252055196204af8f18040 (HEAD -> master, origin/staging, origin/master, origin/HEAD)
Merge: a9c7295dd6 5504e3f9ae
Author: craig[bot]<EMAIL_ADDRESS>Date:   Fri Sep 14 01:24:32 2018 +0000
Merge #29163

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/27660#issuecomment-421301789, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA15IByVPvtHvHaZVvkSEownHzmlif_tks5ua348gaJpZM4Wn-Lc
.

I did that. The error I posted happens during go test.

Are you sure the make testrace command succeeded? It spews a lot of output from parallel commands so the error can really get buried. I just created a fresh VM and verified that the instructions I posted work with commit 418b9169, so I'm not sure what else it could be. I can give you access to this VM if that would be helpful.

Right, it failed somewhere in the beginning:
cockroach$ IGNORE_GOVERS=1 make testrace PKG=./pkg/sql/logictest TESTS=TestLogic IGNORE_GOVERS=1
Running make with -j4
GOPATH set to /go
Detected change in build system. Rebooting Make.
Running make with -j4
GOPATH set to /go
cd /go/src/github.com/cockroachdb/cockroach/c-deps/jemalloc && autoconf
rm -rf /go/native/x86_64-linux-gnu/protobuf
bash: autoconf: command not found
Makefile:511: recipe for target '/go/src/github.com/cockroachdb/cockroach/c-deps/jemalloc/configure' failed
make: *** [/go/src/github.com/cockroachdb/cockroach/c-deps/jemalloc/configure] Error 127
make: *** Waiting for unfinished jobs....
go install -v uptodate
bin/prereqs ./pkg/cmd/uptodate > bin/uptodate.d.tmp
mkdir -p /go/native/x86_64-linux-gnu/protobuf
cd /go/native/x86_64-linux-gnu/protobuf && cmake -DCMAKE_TARGET_MESSAGES=OFF   -Dprotobuf_BUILD_TESTS=OFF /go/src/github.com/cockroachdb/cockroach/c-deps/protobuf/cmake \
  -DCMAKE_BUILD_TYPE=Release
mv -f bin/uptodate.d.tmp bin/uptodate.d
github.com/cockroachdb/cockroach/vendor/github.com/MichaelTJones/walk
-- The C compiler identification is GNU 7.3.0
github.com/cockroachdb/cockroach/vendor/github.com/spf13/pflag
github.com/cockroachdb/cockroach/pkg/cmd/uptodate
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8")
-- Configuring done
-- Generating done
-- Build files have been written to: /go/native/x86_64-linux-gnu/protobuf

Oh, yeah, you’ll need autoconf and CMake installed. It looks like you have
CMake but not autoconf.
On Fri, Sep 14, 2018 at 9:43 AM Dmitry Vyukov<EMAIL_ADDRESS>wrote:

Right, it failed somewhere in the beginning:
cockroach$ IGNORE_GOVERS=1 make testrace PKG=./pkg/sql/logictest TESTS=TestLogic IGNORE_GOVERS=1
Running make with -j4
GOPATH set to /go
Detected change in build system. Rebooting Make.
Running make with -j4
GOPATH set to /go
cd /go/src/github.com/cockroachdb/cockroach/c-deps/jemalloc && autoconf
rm -rf /go/native/x86_64-linux-gnu/protobuf
bash: autoconf: command not found
Makefile:511: recipe for target '/go/src/github.com/cockroachdb/cockroach/c-deps/jemalloc/configure' failed
make: *** [/go/src/github.com/cockroachdb/cockroach/c-deps/jemalloc/configure] Error 127
make: *** Waiting for unfinished jobs....
go install -v uptodate
bin/prereqs ./pkg/cmd/uptodate > bin/uptodate.d.tmp
mkdir -p /go/native/x86_64-linux-gnu/protobuf
cd /go/native/x86_64-linux-gnu/protobuf && cmake -DCMAKE_TARGET_MESSAGES=OFF   -Dprotobuf_BUILD_TESTS=OFF /go/src/github.com/cockroachdb/cockroach/c-deps/protobuf/cmake
-DCMAKE_BUILD_TYPE=Release
mv -f bin/uptodate.d.tmp bin/uptodate.dgithub.com/cockroachdb/cockroach/vendor/github.com/MichaelTJones/walk
-- The C compiler identification is GNU 7.3.0github.com/cockroachdb/cockroach/vendor/github.com/spf13/pflaggithub.com/cockroachdb/cockroach/pkg/cmd/uptodate
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8")
-- Configuring done
-- Generating done
-- Build files have been written to: /go/native/x86_64-linux-gnu/protobuf
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/27660#issuecomment-421362135, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA15INHhn9MKCMjIVRqDwGFG2k8XPr6aks5ua7J4gaJpZM4Wn-Lc
.

How long does it generally take to crash? I've run it twice and both times got:
cockroach$ go test -race -tags ' make x86_64_linux_gnu' -ldflags '-X github.com/cockroachdb/cockroach/pkg/build.typ=development -extldflags "" -X "github.com/cockroachdb/cockroach/pkg/build.tag=v2.2.0-alpha.00000000-793-g33c7d27d82-dirty" -X "github.com/cockroachdb/cockroach/pkg/build.rev=33c7d27d8216b543ac77f0fe39d440ebebfa9e70" -X "github.com/cockroachdb/cockroach/pkg/build.cgoTargetTriple=x86_64-linux-gnu"  ' -run "TestLogic"  -timeout 25m ./pkg/sql/logictest
I180914 14:16:05.303611 1 rand.go:75  Random seed: -5069250136795247745
E180914 14:19:23.006360 77 sql/logictest/logic.go:2228

testdata/logic_test/crdb_internal:295: SELECT node_id, network, regexp_replace(address, '\d+$', '<port>') as address, attrs, locality, regexp_replace(server_version, '^\d+\.\d+(-\d+)?$', '<server_version>') as server_version, regexp_replace(go_version, '^go.+$', '<go_version>') as go_version
FROM crdb_internal.kv_node_status WHERE node_id = 1
expected:
    node_id  network  address           attrs  locality            server_version    go_version
    1        tcp      <IP_ADDRESS>:<port>  []     {"region": "test"}  <server_version>  <go_version>

but found (query options: "colnames") :
    node_id  network  address           attrs  locality            server_version    go_version
    1        tcp      <IP_ADDRESS>:<port>  []     {"region": "test"}  <server_version>  devel +9f59918cae Thu Sep 13 21:34:56 2018 +0000

E180914 14:19:23.006776 77 sql/logictest/logic.go:2259  testdata/logic_test/crdb_internal:302: too many errors encountered, skipping the rest of the input
E180914 14:19:32.896862 50624 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:49210,user=root,scExec] error setting up flow: column "d" does not exist
E180914 14:19:34.520305 50624 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:49210,user=root,scExec] error setting up flow: error type checking constant value: could not parse "a" as type int: strconv.ParseInt: parsing "a": invalid syntax
E180914 14:19:35.740206 50624 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:49210,user=root,scExec] error setting up flow: could not parse "a" as type int: strconv.ParseInt: parsing "a": invalid syntax
E180914 14:19:36.830273 50624 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:49210,user=root,scExec] error setting up flow: unsupported binary operator: <int> / <int> (desired <int>)
E180914 14:24:01.186527 68088 storage/consistency_queue.go:128  [n1,consistencyChecker,s1,r1/1:/M{in-ax}] computing own checksum: could not dial node ID 1: initial connection heartbeat failed: rpc error: code = Canceled desc = context canceled
E180914 14:24:01.186855 68088 storage/queue.go:788  [n1,consistencyChecker,s1,r1/1:/M{in-ax}] computing own checksum: could not dial node ID 1: initial connection heartbeat failed: rpc error: code = Canceled desc = context canceled
E180914 14:26:08.285725 96089 storage/queue.go:788  [n1,raftlog,s1,r1/1:/M{in-ax}] result is ambiguous (server shutdown)
E180914 14:26:15.409641 64257 sql/logictest/logic.go:2228

testdata/logic_test/crdb_internal:295: SELECT node_id, network, regexp_replace(address, '\d+$', '<port>') as address, attrs, locality, regexp_replace(server_version, '^\d+\.\d+(-\d+)?$', '<server_version>') as server_version, regexp_replace(go_version, '^go.+$', '<go_version>') as go_version
FROM crdb_internal.kv_node_status WHERE node_id = 1
expected:
    node_id  network  address           attrs  locality            server_version    go_version
    1        tcp      <IP_ADDRESS>:<port>  []     {"region": "test"}  <server_version>  <go_version>

but found (query options: "colnames") :
    node_id  network  address           attrs  locality            server_version    go_version
    1        tcp      <IP_ADDRESS>:<port>  []     {"region": "test"}  <server_version>  devel +9f59918cae Thu Sep 13 21:34:56 2018 +0000

E180914 14:26:15.410048 64257 sql/logictest/logic.go:2259  testdata/logic_test/crdb_internal:302: too many errors encountered, skipping the rest of the input
E180914 14:26:27.787026 99569 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:36516,user=root,scExec] error setting up flow: column "d" does not exist
E180914 14:26:29.171953 99569 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:36516,user=root,scExec] error setting up flow: error type checking constant value: could not parse "a" as type int: strconv.ParseInt: parsing "a": invalid syntax
E180914 14:26:30.247536 99569 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:36516,user=root,scExec] error setting up flow: could not parse "a" as type int: strconv.ParseInt: parsing "a": invalid syntax
E180914 14:26:31.426094 99569 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:36516,user=root,scExec] error setting up flow: unsupported binary operator: <int> / <int> (desired <int>)
E180914 14:37:20.722139 157227 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:35082,user=root,scExec] error setting up flow: column "d" does not exist
E180914 14:37:22.977964 157227 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:35082,user=root,scExec] error setting up flow: error type checking constant value: could not parse "a" as type int: strconv.ParseInt: parsing "a": invalid syntax
E180914 14:37:26.350066 157227 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:35082,user=root,scExec] error setting up flow: could not parse "a" as type int: strconv.ParseInt: parsing "a": invalid syntax
E180914 14:37:28.390702 157227 sql/distsqlrun/server.go:440  [n1,client=<IP_ADDRESS>:35082,user=root,scExec] error setting up flow: unsupported binary operator: <int> / <int> (desired <int>)
panic: test timed out after 25m0s

goroutine 175652 [running]:
testing.(*M).startAlarm.func1()
	/src/go/src/testing/testing.go:1303 +0x13a
created by time.goFunc
	/src/go/src/time/sleep.go:172 +0x52

goroutine 1 [chan receive, 25 minutes]:
testing.(*T).Run(0xc00034e300, 0x2ee939f, 0x9, 0x3010de0, 0xc0006b7bd0)
	/src/go/src/testing/testing.go:886 +0x689
testing.runTests.func1(0xc00034e300)
	/src/go/src/testing/testing.go:1126 +0xa9
testing.tRunner(0xc00034e300, 0xc0006b7d40)
	/src/go/src/testing/testing.go:834 +0x163
testing.runTests(0xc000299ba0, 0x4f9b7a0, 0x5, 0x5, 0x0)
	/src/go/src/testing/testing.go:1124 +0x4ef
testing.(*M).Run(0xc0004c8580, 0x0)
	/src/go/src/testing/testing.go:1041 +0x2ef
github.com/cockroachdb/cockroach/pkg/sql/logictest.TestMain(0xc0004c8580)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/main_test.go:37 +0x15b
main.main()
	_testmain.go:48 +0x222

It's been reproducing for me in less than a minute. Oddly enough, it's failing to reproduce for me on the latest master. Could you try commit 33c7d27d8216b543ac77f0fe39d440ebebfa9e70? And just to double check, you are running go1.11, right? I'm also having trouble reproducing on go1.10.3.
In the meantime I'm going to bisect to see if there's something obviously wrong that we fixed.

Bisecting proved challenging thanks to some actual races that were introduced and fixed within the range of commits I was bisecting. Regardless, I can semi-reliably repro on 33c7d27d8216b543ac77f0fe39d440ebebfa9e70 on a fresh Ubuntu machine with Go 1.11.
The problem seems to occur early or not at all. In five runs, this is what I observed:

Test timed out after 2m.
ThreadSanitizer CHECK failed: ./gotsan.cc:746 "((v)) > ((0))" (0x0, 0x0) after 54.42s.
Test timed out after 2m.
ThreadSanitizer CHECK failed: ./gotsan.cc:380 "((v)) > ((0))" (0x0, 0x0) after 113.751s.
Test timed out after 2m.

I could have sworn this reproduced more reliably when I original filed the bug. Still, I'm hopeful that you might be able to reproduce if you run a few more trials with a short timeout. The magic incantation to thread in a timeout via the Makefile, if that's easier, is:
$ make testrace PKG=./pkg/sql/logictest TESTS=TestLogic TESTTIMEOUT=2m

Oh, cool, I didn't know about the -failure flag to stress.
Anyway, thanks again for looking into this. We upgrade our CI environment to go1.11 today and various other tests started producing assertions like this under the race detector. We also started seeing outright segfaults:
unexpected fault address 0x27400
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x27400 pc=0x2bf0df6]

goroutine 1487 [running]:
runtime: unexpected return pc for runtime.throw called from 0x361d16b
stack: frame={sp:0x7fc94e9fe990, fp:0x7fc94e9fe9c0} stack=[0xc000ee3000,0xc000ee3800)

I bisected this down to a commit in Go itself:<PHONE_NUMBER>d984491d8070ed30aeffc1abe344a2, titled "runtime/race: update most syso files to compiler-rt". That sure looks like our culprit, though I'm willing to believe we're still doing something wrong (e.g., corrupting memory, like you suggest) and the update to the race detector just happened to tickle this preexisting condition.
I'm going to try to get somewhere by bisecting compiler-rt itself. I may need your help in producing those syso files.

Whew, that was an exercise in patience. Bisecting was complicated by the fact that a number of revisions of compiler-rt between 68e15324 and fe2c72c5 did not compile with my GCC toolchain. Regardless, I managed to pinpoint the problem to this commit: https://github.com/llvm-mirror/compiler-rt/commit/12d16901ebba5bc7185298e9efa3c4d265fa52ce. Still unclear what the actual problem is. I'm not in the position to understand that tsan code, I'm afraid.
By the way, I'm now using this to repro:
make stressrace PKG=./pkg/ccl/backupccl TESTS=TestBackupRestoreResume STRESSFLAGS="-p 8" IGNORE_GOVERS=1 GO=~/dev/go/bin/go

The -p 8 avoids overloading my 24 CPU VM; each test execution seems to require about 3 CPUs. The IGNORE_GOVERS=1 GO=~/dev/go/bin/go is only necessary because I'm pointing at a custom version of Go. I've found that this reliably repros in under 5m.
I am once again feeling that this is more likely to be a bug in tsan than in CockroachDB. @dvyukov let me know if you disagree. My offer of providing you a VM that can reproduce this issue out of the box still stands.

I am once again feeling that this is more likely to be a bug in tsan than in CockroachDB.

Maybe. Hard to say. What's strange is that nobody reported any corruptions before and that I can't reproduce this on my machine. Can anybody else reproduce this?

Can anybody else reproduce this?

I'm going to check if any of my fellow Linux-using engineers can reproduce this issue on their laptop. Everyone else uses the same VM configuration (Ubuntu 18.04 on Google Compute Engine), so it's not all that interesting that they can reproduce. Here's the output of uname -r:
Linux gceworker-benesch 4.15.0-1018-gcp #19-Ubuntu SMP Thu Aug 16 13:38:55 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Are you willing to share the output of uname -r on your machine?

Ok, @mberhault was able to reproduce on his Linux desktop running Ubuntu 16.04 (kernel 4.4.0-135-generic) using the make stressrace PKG=./pkg/ccl/backupccl... with go1.11 and the latest CockroachDB master.

 If not, that would be the first thing I would do.

Ok, great, you can reproduce!
Do you know how to read that stack trace you posted?
unexpected fault address 0x2600
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x2600 pc=0x2bff9a6]

goroutine 5587 [running]:
runtime: unexpected return pc for runtime.throw called from 0x360c2ed
stack: frame={sp:0x7f7e059ec9b0, fp:0x7f7e059ec9e0} stack=[0xc0048c4000,0xc0048cc000)

runtime.throw(0x3, 0x3)
	src/runtime/panic.go:608 +0x72 fp=0x7f7e059ec9e0 sp=0x7f7e059ec9b0 pc=0x697d72
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	gopath/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:192 +0xc4

I don't understand how we go from stopper.go:192 (which launches a goroutine) to runtime.throw. (Other segfaults I've triggered in Go are understandable, by contrast.) I suppose the answer is probably memory corruption?

@benesch do you know if native part of cockroachdb was testing with asan/tsan/msan? If not, that would be the first thing I would do.

It's not tested with any of the sanitizers, unfortunately. Most of our serious testing is done in Go, so we exercise the C/C++ code via cgo. That makes it tricky to use any of the sanitizers. I've tried to get msan to work but it triggers a number of false positives due to structs with padding: https://github.com/golang/go/issues/26167. I suppose it should be possible to use asan and tsan on our native code by plumbing in -fsanitize during compilation. I'll see if I can find some time to do that today.
That said, there are only four C/C++ libraries that we link: snappy, protobuf, RocksDB, and jemalloc. I've verified that this reproduces without jemalloc, so I think we can rule that one out. Snappy and protobuf are both used extensively at Google, so I'd be surprised if they were to blame. RocksDB is extremely complicated, but they test upstream with asan/tsan/ubsan. So that leaves our first-party C/C++ that glues everything together. It's certainly possible that we've had a lurking memory bug there, but a) the code's not that complicated, and b) I feel like we'd have seen it crop up somewhere before. This is all speculation, of course.

Hi, @dvyukov . Any update on this issue? We got a similar problem in our project after we updated go to version 1.11. And I just confirm that this issue exists in 1.11.2 too.

@leventeliu update on what exactly you mean? It looks like the app has memory corruption. I don't plan to do anything with that.
I noticed that I missed a question:

Do you know how to read that stack trace you posted?

It says that there was a paging fault when some code tried to access address 0x2600.
The bad accesses happened in non-Go code, Go unwinder could not show a trace (maybe it's untraceable code at all). It also says that the goroutine where the crash happened was created by stop.(*Stopper).RunWorker. That's pretty much it.

It says that there was a paging fault when some code tried to access address 0x2600.
The bad accesses happened in non-Go code, Go unwinder could not show a trace (maybe it's untraceable code at all). It also says that the goroutine where the crash happened was created by stop.(*Stopper).RunWorker. That's pretty much it.

Oh, I see. I was misreading the line under "created at" as part of the fault backtrace, not as the backtrace of the creation of the goroutine. Thanks.

Any update on this issue? We got a similar problem in our project after we updated go to version 1.11. And I just confirm that this issue exists in 1.11.2 too.

@leventeliu do you see exactly the same sorts of internal tsan assertion failures? And is your project public? If so I'd love to take a look. CockroachDB is large enough that narrowing this down has proven extremely difficult.
Specifically, if the failures in your project disappear if you build a version of the race detector with https://github.com/llvm-mirror/compiler-rt/commit/12d16901ebba5bc7185298e9efa3c4d265fa52ce reverted, that's additional support for the theory that the problem is in tsan. If not, then it's likely that your project and CockroachDB both have rare memory corruption issues.

I was confused about why that stack trace from a few comments ago was so indecipherable given that the PC implicated in the segfault occurred was clearly in ThreadSanitizer code. After some digging I realized that Go butchers the stack on a segfault: https://github.com/golang/go/blob/9f78aa77443389d61a3ef910a1c3d4628999d82c/src/runtime/signal_unix.go#L358-L364
I guess this is nice when it works, but it's really getting in the way here. I built a custom version of Go with this feature disabled:
diff --git a/src/runtime/sigtab_linux_generic.go b/src/runtime/sigtab_linux_generic.go
index b26040b803..d02225201b 100644
--- a/src/runtime/sigtab_linux_generic.go
+++ b/src/runtime/sigtab_linux_generic.go
@@ -22,7 +22,7 @@ var sigtable = [...]sigTabT{
        /* 8 */ {_SigPanic + _SigUnblock, "SIGFPE: floating-point exception"},
        /* 9 */ {0, "SIGKILL: kill"},
        /* 10 */ {_SigNotify, "SIGUSR1: user-defined signal 1"},
-       /* 11 */ {_SigPanic + _SigUnblock, "SIGSEGV: segmentation violation"},
+       /* 11 */ {0, "SIGSEGV: segmentation violation"},
        /* 12 */ {_SigNotify, "SIGUSR2: user-defined signal 2"},
        /* 13 */ {_SigNotify, "SIGPIPE: write to broken pipe"},
        /* 14 */ {_SigNotify, "SIGALRM: alarm clock"},

And voila! Now GDB has a better stack trace:
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000002db2719 in __tsan::SyncClock::Resize(__tsan::DenseSlabAllocCache*, unsigned long) ()
(gdb) bt
#0  0x0000000002db2719 in __tsan::SyncClock::Resize(__tsan::DenseSlabAllocCache*, unsigned long) ()
#1  0x0000000002db2b23 in __tsan::SyncClock::Unshare(__tsan::DenseSlabAllocCache*) ()
#2  0x0000000002db2e33 in __tsan::ThreadClock::ReleaseStore(__tsan::DenseSlabAllocCache*, __tsan::SyncClock*) ()
#3  0x0000000002db3e7c in __tsan::ReleaseStore(__tsan::ThreadState*, unsigned long, unsigned long) ()
#4  0x00000000006ce983 in racecall () at /home/benesch/dev/go/src/runtime/race_amd64.s:381
#5  0x00007fb7abcf73d0 in ?? ()
#6  0x0000000000800000 in regexp.(*machine).alloc (i=0x7fb7877fe700, m=0x0) at /home/benesch/dev/go/src/regexp/exec.go:100
#7  regexp.(*machine).add (cap=..., cond=0 '\000', m=0x0, pc=3496985344, pos=0, q=0x0, t=0x0, ~r6=<optimized out>) at /home/benesch/dev/go/src/regexp/exec.go:296
#8  0x0000000000000000 in ?? ()

I suspect frames 5 - 8 are garbage; GDB seems to have trouble unwinding through racecall. Next stop is trying to get line numbers on the ThreadSanitizer frames.

Ok, plumbing a -g into the build process for the .syso seems to have worked. Here's a more complete stack trace:
#0  __tsan::SyncClock::Unshare (this=0x7f94eea294f8, c=0x7f95088b2920) at ./gotsan.cc:773
#1  0x0000000002db3fff in __tsan::ThreadClock::ReleaseStore (this=0x7f94e2aa8c60, c=<optimized out>, dst=0x7f94eea294f8) at ./gotsan.cc:565
#2  0x0000000002dc615f in AtomicCAS<unsigned int> (mo=__tsan::mo_acq_rel, fmo=__tsan::mo_acquire, v=<optimized out>, c=<synthetic pointer>, a=0xc00425c840, pc=7135484,
    thr=0x7f94e2aa8c08) at ./gotsan.cc:1550
#3  AtomicCAS<unsigned int> (fmo=__tsan::mo_acquire, mo=__tsan::mo_acq_rel, v=<optimized out>, c=0, a=0xc00425c840, pc=7135484, thr=0x7f94e2aa8c08) at ./gotsan.cc:1573
#4  __tsan_go_atomic32_compare_exchange (thr=0x7f94e2aa8c08, cpc=<optimized out>, pc=7135484, a=0xc002ec1bd8 "@\310%\004\300") at ./gotsan.cc:2077
#5  0x00000000006ce223 in racecall () at /home/benesch/dev/go/src/runtime/race_amd64.s:381

I put the corresponding gotsan.cc here: https://gist.github.com/benesch/b62181af5aa6d3d7efa0761e34e7df68.

@benesch Yes, we are having the exact same FATAL while testing go-sqlite3, which is a sqlite driver implemented via cgo. And it can be easily reproduced with stress tool at any Go1.11.x version.
Here is our public project. Go to xenomint/sqlite, run go test -v -race -o sqlite_test.o and stress ./sqlite_test.o, and see what happens.
I have done some digging on this, but totally have no clue to it.

Excellent, I can reproduce. It's definitely the same problem. Thanks so much for reporting, @leventeliu! This is strong evidence that the bug is in either the Go runtime or ThreadSanitizer itself.
@dvyukov FYI this reproduction also disappears if I build the Go race detector with
https://github.com/llvm-mirror/compiler-rt/commit/12d16901ebba5bc7185298e9efa3c4d265fa52ce reverted.

Thanks for all the debugging. I will take a closer look. But I am travelling for few next weeks, so can't promise when exactly it will happen.

Gotcha, no problem. I'll keep digging in the meantime. On a hunch I disabled clock sharing in ThreadSanitizer with the following diff:
diff --git a/lib/tsan/rtl/tsan_clock.cc b/lib/tsan/rtl/tsan_clock.cc
index ef984a45c..0e7f21ba0 100644
--- a/lib/tsan/rtl/tsan_clock.cc
+++ b/lib/tsan/rtl/tsan_clock.cc
@@ -238,7 +238,7 @@ void ThreadClock::ReleaseStore(ClockCache *c, SyncClock *dst) {
   DCHECK_LE(dst->size_, kMaxTid);
   CPP_STAT_INC(StatClockStore);

-  if (dst->size_ == 0 && cached_idx_ != 0) {
+  if (dst->size_ == 0 && cached_idx_ != 0 && false) {
     // Reuse the cached clock.
     // Note: we could reuse/cache the cached clock in more cases:
     // we could update the existing clock and cache it, or replace it with the
@@ -296,7 +296,7 @@ void ThreadClock::ReleaseStore(ClockCache *c, SyncClock *dst) {

   // If the resulting clock is cachable, cache it for future release operations.
   // The clock is always cachable if we released to an empty sync object.
-  if (cached_idx_ == 0 && dst->Cachable()) {
+  if (cached_idx_ == 0 && dst->Cachable() && false) {
     // Grab a reference to the ClockBlock.
     atomic_uint32_t *ref = ref_ptr(dst->tab_);
     if (atomic_load(ref, memory_order_acquire) == 1)

and the problem disappeared! So my best guess is that ThreadSanitizer is somehow corrupting its own internal state when it shares clocks.

Indeed, this seems to be a straightforward double free. I'm going to try to write up a targeted test case and figure out how to submit a patch to LLVM.

You debugged the root cause? That was brave!
You already know how to build it, right?
Here are some docs on contributing:
https://github.com/google/sanitizers/wiki/AddressSanitizerHowToContribute
https://llvm.org/docs/Contributing.html
https://llvm.org/docs/Phabricator.html
There is probably lots of irrelevant stuff. The main part is to send a Phabricator review.

And a test would be good.

I think I spoke to soon. I thought the bug was a simple reference counting error with shared SyncClocks, but the problem appears to be deeper than that. In the course of my investigation I added a bit of logging to the DenseSlabAllocator.  This is a guaranteed disaster.
curproc is the address of the proc struct returned by get_cur_proc(), while tid is the opaque ID reported pthread_self().
The problem seems to be that the proc struct 0x0279c798, which contains a non-threadsafe allocator cache, is simultaneously in use by goroutines e7fc700 and a1ffb700. (It's possible that those two goroutines were quickly scheduled and rescheduled on the same pthread, but I doubt it.)
So is it possible that the Go runtime is sometimes failing to update the proc struct when it moves a goroutine between pthreads? That's my best guess as to what's happening. It explains why this badness has only been observed in Go programs, and not any C++ programs.
Disclaimer that I'm not 100% confident in my instrumentation, so I might be a little bit off here.

I gathered some more evidence for the theory that the Go runtime is sharing processor data across multiple threads. I started by annotating several of the calls into tsan. Here's a snippet.
diff --git a/lib/tsan/rtl/tsan_rtl.h b/lib/tsan/rtl/tsan_rtl.h
index 5e2a745c9..130b33609 100644
--- a/lib/tsan/rtl/tsan_rtl.h
+++ b/lib/tsan/rtl/tsan_rtl.h
@@ -356,6 +356,7 @@ struct Processor {
   DenseSlabAllocCache sync_cache;
   DenseSlabAllocCache clock_cache;
   DDPhysicalThread *dd_pt;
+  bool inuse;
 };

 #if !SANITIZER_GO

diff --git a/lib/tsan/go/tsan_go.cc b/lib/tsan/go/tsan_go.cc
index 71a660683..ba0dd4a08 100644
--- a/lib/tsan/go/tsan_go.cc
+++ b/lib/tsan/go/tsan_go.cc
@@ -231,48 +231,84 @@ void __tsan_proc_destroy(Processor *proc) {
 }

 void __tsan_acquire(ThreadState *thr, void *addr) {
+  CHECK(!thr->proc()->inuse);
+  thr->proc()->inuse = true;
   Acquire(thr, 0, (uptr)addr);
+  CHECK(thr->proc()->inuse);
+  thr->proc()->inuse = false;
 }

The full diff is here: https://gist.github.com/benesch/cb3258a3eb4b573b5a3b7891db000e66
The reproductions of the original bug now fail with
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:234 "((!thr->proc()->inuse)) != (0)" (0x0, 0x0)

instead of the various assertion errors they would previously fail with. That strongly implies that the root cause of all the assertion failures is the incorrect sharing of thr->proc().  And programs which did not trigger the previous assertions do not trigger this new assertion.
I've started poking around the Go scheduler but I'm way out of my depth here. I'll note that this diff
to disable stealing of Ps in syscalls
diff --git a/src/runtime/proc.go b/src/runtime/proc.go
index f82014eb92..5993e2a6da 100644
--- a/src/runtime/proc.go
+++ b/src/runtime/proc.go
@@ -1081,7 +1081,7 @@ func stopTheWorldWithSema() {
 	// try to retake all P's in Psyscall status
 	for _, p := range allp {
 		s := p.status
-		if s == _Psyscall && atomic.Cas(&p.status, s, _Pgcstop) {
+		if s == _Psyscall && false && atomic.Cas(&p.status, s, _Pgcstop) {
 			if trace.enabled {
 				traceGoSysBlock(p)
 				traceProcStop(p)
@@ -1455,7 +1455,7 @@ func forEachP(fn func(*p)) {
 	// off to induce safe point function execution.
 	for _, p := range allp {
 		s := p.status
-		if s == _Psyscall && p.runSafePointFn == 1 && atomic.Cas(&p.status, s, _Pidle) {
+		if s == _Psyscall && false && p.runSafePointFn == 1 && atomic.Cas(&p.status, s, _Pidle) {
 			if trace.enabled {
 				traceGoSysBlock(p)
 				traceProcStop(p)
@@ -4457,7 +4457,7 @@ func retake(now int64) uint32 {
 		}
 		pd := &_p_.sysmontick
 		s := _p_.status
-		if s == _Psyscall {
+		if s == _Psyscall && false {
 			// Retake P from syscall if it's there for more than 1 sysmon tick (at least 20us).
 			t := int64(_p_.syscalltick)
 			if int64(pd.syscalltick) != t {

vastly decreases the probability that xenomint/sqlite will hit the tsan assertion. (I initially thought it had fixed the problem, but alas.) With the tsan patch but not the Go scheduler patch, running xenomint/sqlite's tests under race reliably hits the tsan assertion on every run. With both patches, I managed to get 187 runs before it failed. I'm not sure what to make of that.

If 2 threads use the same Processor, it would explain all kinds of weird memory corruptions in tsan runtime.
thr->proc() obtains current Processor from Go runtime as g.m.p.racectx. The firing CHECK means that either 2 goroutines run with the same g.m, or 2 m's runtime with the same m.p. Both of which looks badly wrong.
I am thinking of some way to check this condition reliably in Go runtime and catch it es early as possible. But so far have not come up with anything.

This does not happen with Go1.10, right? Then one option is to bisect Go from Go1.10 to Go1.11 to find the root cause.

Yep, xenomint/sqlite blows up on go1.10 if you add the additional assertions to the race detector. Looks like this has been a latent bug in the Go runtime.

Perhaps this isn't so bad. On a hunch, I added a bit more information to the new assertions:
diff --git a/lib/tsan/go/tsan_go.cc b/lib/tsan/go/tsan_go.cc
index 5f2507b7d..d3345c6b7 100644
--- a/lib/tsan/go/tsan_go.cc
+++ b/lib/tsan/go/tsan_go.cc
 void __tsan_acquire(ThreadState *thr, void *addr) {
+  auto inuse_goid = thr->proc()->inuse_goid;
+  if (inuse_goid != 0) {
+    __sanitizer::Printf("tsan_acquire(%p): proc in use by %d (this goid: %d)\n", addr, inuse_goid, thr->goid);
+    __sanitizer::Die();
+  }
+  thr->proc()->inuse_goid = thr->goid;
   Acquire(thr, 0, (uptr)addr);
+  CHECK_EQ(thr->proc()->inuse_goid, thr->goid);
+  thr->proc()->inuse_goid = 0;
 }

Then I noticed that all of the failures seemed to be triggered by a call to __tsan_acquire:
tsan_acquire(0x000005035be8): proc in use by 48 (this goid: 49)

Even more suspiciously, every failure was an acquisition of the exactly the same memory address, 0x5035be8. Kind of seems like a global variable...
$ nm sqlite.test | grep 000005035be8
0000000005035be8 s runtime.racecgosync

Looks like cgocall is just calling raceacquire(&racecgosync) before it allocates a P (via exitsyscall), and that's not valid.
I'm hopeful that I've actually found the root cause this time.

Just a minute ago I started looking at:
func endcgo(mp *m) {
	mp.incgo = false
	mp.ncgo--

	if raceenabled {
		raceacquire(unsafe.Pointer(&racecgosync))
	}
}

:)

It needs to do raceacquire after exitsyscall. I think we beaten by the dangling g.m.p pointer left in reentersyscall.

cgocallbackg is also badly broken.

@benesch do you want to fix yourself? Since you already done so much work on this, don't want to steal it from you.
I think we also need to change reentersyscall to leave the dangling p pointer in some shadow variable instead of the g.m.p. g.m.p set makes impression that m and p are wired when they are actually not. Then only exitsyscallfast and retake will use the shadow p variable. This should have been caught this bug.

Whoops, sorry, missed your messages! About to send a CL.
I like your idea of a shadow variable but I didn't want to embark on that refactor myself, so I took a less intrusive approach. Now that I have your blessing I'm happy to do the refactor instead since it will prevent future mistakes of this kind—it is extremely confusing that an M can point to a P that it doesn't currently own. But I'll send the CL for what I have in the meantime to get the discussion started.

Nice sleuthing! Gentle plea for this to be back-ported to go1.11 as this bug is preventing CockroachDB from upgrading to go1.11 (yes, it only affects race builds, but we run go test -race during CI).

@gopherbot Please open an issue to backport to 1.11


Feature request: driver installed (IDFGH-2267)

Request for a method in the APIs to check whether a driver has already been installed.
For example, if you have two objects, both using UART_NUM_1, then the object that is created second has no way of knowing if the UART driver is already installed and running, or not.
Would be very useful,
Many thanks

Hi, @howroyd
Thins feature is available on 4.2 ~ 4.0 release version.


Fix issue #514

This pull request attemps to fix the issue #514 related to center of mass computation.
The code was completely refactored and improved. The center of mass are not correctly computed.
An integrity test has also been added.

Wrong pull request. Should have been to dev, not master.


WIP: log how long it takes to satisfy the make-new-nodes request

No need to review: I'm just curious about collecting stats on the CI machines.

@rainersigwald  FYI on my machine.
##vso[task.logissue type=warning] 00:00:00.0164362 spent making 1 new nodes
##vso[task.logissue type=warning] 00:00:01.5292103 spent making 11 new nodes

Seems like not a lot. You had one with multiple seconds?

@AndyGerlicher Hmm. On both of the 24-core machines I have access to I get something like
##vso[task.logissue type=warning] 00:00:00.0363900 spent making 1 new nodes
##vso[task.logissue type=warning] 00:00:06.2637747 spent making 23 new nodes

The VSTS VMs are tiny and don't show any problems either. Maybe I've got something funky on my machines?

This has been stale for a very long time and I no longer remember why it seemed important. Closing because of that.


Coveralls

Add support for Coveralls CI service

@lgebhardt "This branch has conflicts that must be resolved"

@lgebhardt This has been sitting for several years. Do you want to update this and get it merged?


{"status":"ok","data":[{"confidence":0.932974,"text":""}]}

Hi,it still don't work well with the latest kaldi,so anyone can help?

@zhaoyi2 I was having the same issue. I was using asr-server from this repo which uses a fork of this repo.
My solution was to use the following to send audio to the server (using the demo model mentioned in the docs):
curl -H "Content-Type: application/octet-stream" --data-binary @/Users/my-user/Downloads/sound-file.wav http://localhost:8080/asr?endofspeech=1

In other words, I played around with the parameters specified in the docs. With the endofspeec=1 param, I now get the entire file transcribed (albeit, the sample model isn't super accurate with the input files I'm testing this with).


[#3390] support to check available applicationName and agentId on the…

… web
move IdValidateUtils for use on the web. (bootstrap-core -> common)

Codecov Report

Merging #3393 into master will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3393      +/-   ##
==========================================
+ Coverage   40.08%   40.11%   +0.02%
==========================================
  Files        2135     2135
  Lines       69736    69736
  Branches     9674     9674
==========================================
+ Hits        27956    27976      +20
+ Misses      39066    39051      -15
+ Partials     2714     2709       -5

Impacted Files
Coverage Δ

...ler/context/provider/AgentInformationProvider.java
65.21% <ø> (ø)
:arrow_up:

.../com/navercorp/pinpoint/bootstrap/IdValidator.java
0% <ø> (ø)
:arrow_up:

...avercorp/pinpoint/common/util/IdValidateUtils.java
63.33% <ø> (ø)

...p/pinpoint/profiler/sender/AbstractDataSender.java
97.05% <0%> (-2.95%)
:arrow_down:

...navercorp/pinpoint/rpc/common/SocketStateCode.java
82.92% <0%> (-2.44%)
:arrow_down:

...om/navercorp/pinpoint/rpc/codec/PacketDecoder.java
61.7% <0%> (-2.13%)
:arrow_down:

...m/navercorp/pinpoint/test/OrderedSpanRecorder.java
70.19% <0%> (-1.93%)
:arrow_down:

.../com/navercorp/pinpoint/profiler/context/Span.java
50% <0%> (-1.79%)
:arrow_down:

...point/rpc/client/DefaultPinpointClientHandler.java
71.29% <0%> (-0.31%)
:arrow_down:

...inpoint/profiler/sender/AsyncQueueingExecutor.java
77.17% <0%> (+1.08%)
:arrow_up:

... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d639e7d...62f4f2d. Read the comment docs.


zq option to specify a null input value

At the time this issue is being filed, Zed is at commit 6fcbee6.
Currently the zq usage docs explain:

In the case of a yield with no inputs, the query is run with a single input value of null. This provides a convenient means to run in a "calculator mode" where input is produced by the yield and can be operated upon by the Zed query, e.g.,
zq -z '1+1'
emits
2

However, issues like #5110 from the community indicate there's some potential ambiguity about when zq requires arguments, and this "calculator mode" exception is one such example that might create user confusion. During a recent group discussion, there was consensus that we could reduce some ambiguity by requiring the use of a new command line option to explicitly request the "null input". This would closely follow what many users know from jq:
$ jq --help
...
Command options:
  -n, --null-input          use `null` as the single input value;

We ended up addressing this from the opposite direction. The changes in #5346 made it such that now -c is used to explicitly precede query text, which reduces the "potential ambiguity" described above. The docs changes in #5481 also clarified the "calculator mode" in terms of how it's similar to how SQL behaviors, such that the text from the relevant section now reads:

When super is run with a query that has no FROM operator and no input arguments, the SuperSQL query is fed a single null value analogous to SQL's default input of a single empty row of an unnamed table. This provides a convenient means to explore examples or run in a "calculator mode", e.g.,
super -z -c '1+1'

emits
2

Note that SuperSQL's has syntactic shortcuts for interactive data exploration and an expression that stands alone is a shortcut for SELECT VALUE, e.g., the query text
1+1

is equivalent to
SELECT VALUE 1+1


Can't install on Mac

Used this guide for installation
Tried this solution also.
But still it does not work.
My Mac OS version is 12.3.1 Monterey.
Can you help me?

Share screenshots or a QuickTime video of all steps you are taking.

Can you show the Game App contents and ensure that Contents\Resources\Data\Managed has a sub folder UnityModManager and 2 files: UnityEngine.UIModule.dll and UnityEngine.UIModule.dll.original_?
Share a screenshot of this folder please.

What’s your game version? Can you see the UMM UI when starting the game?

Version is 1.4.33

Nope.

The DLL there has the same timestamp as all other game files. This tells me the merge didn’t happen correctly. Copy the same DLL I mentioned above from zip and overwrite.

Thanks.
I have copied this files in the Solasta directory not in the Contents. This was a reason why I can't use the mod.


Avto

Change the branch destination ( definitely not main) and branch naming,  ( including the ticket name)


Table statistics should be cached

Currently, table stats are obtained by executing HTTP requests. This is bad, because:

it's done during planning, slowing it down
it requires some predicates, like a value for owner and repo

Instead, stats should be cached with an optional TTL.

https://github.com/google/guava/wiki/CachesExplained

This will allow running queries like this:
SELECT *
FROM repos
JOIN workflows w ON w.owner = repo.owner_login AND w.repo = repos.name
WHERE repos.owner_login = 'nineinchnick'


Update README.md with documentation link updates

Motivation
Fix Broken Links
Have you read the Contributing Guidelines on pull requests?
(Write your answer here.)
Test Plan
Verified links work in the Preview of the md file.
Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/diem/diem/tree/latest/developers.diem.com, and link to your PR here.)
If targeting a release branch, please fill the below out as well

Justification and breaking nature (who does it affect? validators, full nodes, tooling, operators, AOS, etc.)
Comprehensive test results that demonstrate the fix working and not breaking existing workflows.
Why we must have it for V1 launch.
What workarounds and alternative we have if we do not push the PR.

/canary

:sunny: Canary successful

/land


Make the widget background transparent and include more information

Hi,
First of all, thank you for adding a widget to the app. I've quickly tried it, but I was wondering if there could be an option to make the background transparent.
Also, resizing the widget doesn't make it display more information.
Thanks!

Hi, as I already wrote in #1, all this is already in the plans. I am still developing in creating widgets and will add new features as I study.

Hi, as I already wrote in #1, all this is already in the plans. I am still developing in creating widgets and will add new features as I study.

Thanks!

@Leonavichus I did some research into widget transparency and achieved it using this code:

Would it be a good idea to let users pick the color for the widget using a color picker inside the app settings? And then call the update widget method in the controller?
I can work on a PR on this if you want! In this manner we would be giving lots of customizability rather than directly setting it transparent.

@shatanikmahanty, You are my salvation, you would help me a lot with this! I would be very grateful for your help! 🙏

@shatanikmahanty, You are my salvation, you would help me a lot with this! I would be very grateful for your help! 🙏

Would get back with a pr soon ☺


No module named 'passageidentity.openapi_client'

After installing passage-identity, I run main.py with a single line from passageidentity import Passage in it. I get this error:
Traceback (most recent call last):
  File "/Volumes/web/collection/main.py", line 1, in <module>
    from passageidentity import Passage
  File "/opt/homebrew/lib/python3.11/site-packages/passageidentity/__init__.py", line 2, in <module>
    from .passage import Passage
  File "/opt/homebrew/lib/python3.11/site-packages/passageidentity/passage.py", line 18, in <module>
    from .openapi_client.api import AppsApi, MagicLinksApi, TokensApi, UsersApi, UserDevicesApi
ModuleNotFoundError: No module named 'passageidentity.openapi_client'

Hi @bardt this should be fixed now in v2.3.1 with https://github.com/passageidentity/passage-python/pull/53

Confirmed, it's fixed. Thank you!


not work in slackware

pls dell text

Not sure why that is, it does work in many other linux distros. I believe this is related to your setup. Feel free to reopen again with more details if you're stilling having trouble.


collect role name/idx and vc info from inspect

vc info will be used in alert email so user can know where to find their jobs.

Coverage remained the same at 53.313% when pulling 7cca0d28a00df2907802469b69e4f7075be6fa63 on dixu/collect-vc into dea48ad2946e5251c74b4960208951ada948d678 on dixu/dlts.


Improve usage of OCR in rules and switch more to of

This is basically #671 but after the other refactors to keep it from being as overwhelming.
I tried to keep concerns here focused to one of these two things:

Manually updated rules to use .scan.ocr.raw when it makes sense
Switch from length(filter.scan.strings.strings, ...) to of when it makes more sense
Fixed or simplified logic in a few places in these files when I encountered them

hero! Just tested against a FP i was going to tune and this remediated it>


Add codacy coverage reporting

https://github.com/codacy/codacy-coverage-reporter#setup

Works now.


Runbook EC2-Disk-Cleanup

Describe the bug
The Runbook EC2-Disk-Cleanup.ipynb is not shown properly in the Github File Browser.

However, this Runbooks opens properly when opened using the Link in the Docker.
To Reproduce
Steps to reproduce the behavior:

Navigate to the Code
Click on AWS Folder -> EC2-Disk-Cleanup.ipynb

you will see the issue.
Expected behavior
Runbook should open like any others that are opened.
Screenshots

Desktop (please complete the following information):

OS: Mac OSX
Browser Safari
Version 16.0 (176<IP_ADDRESS>.10, 17614)

Smartphone (please complete the following information):
N/A
Additional context
Add any other context about the problem here.

https://github.com/unskript/Awesome-CloudOps-Automation/pull/21
This issue is already addressed as part of the PR above.


$apolloData.loading missing when testing components that use ApolloQuery

We were having some problems when testing components that make use of ApolloQuery in combination with @vue/test-utils because it's difficult to get the content of the scopedSlot to render.
With shallowMount and ApolloQuery on the top-level of a using component the render result is only <apolloquery query="[object Object]" variables="[object Object]" debounce="200" skip="true" tag=""></apolloquery>, so we opted to use mount like the following:
const localVue = vueTestUtils.createLocalVue()
localVue.use(VueApollo)
const wrapper = mount(SomeComponentWithApolloQuery, {
  localVue
})

We then get TypeError: Cannot read property 'loading' of undefined, which is coming from the following line:
https://github.com/Akryum/vue-apollo/blob/2293adb03f23429af75723cc441ce01916699a83/src/components/ApolloQuery.js#L200
We can make it work by mocking $apolloData:
const const mountConfig = mount({
   mocks: {
     $apolloData: {
       loading: false,
     },
  },
})

But it could be good to ensure $apolloData.loading is set in the plugin already if it relies on this data and to make sure others are not running into similar problems.
Thanks for the great work on this plugin!

I'm happy to open an MR by the way, but am not completely sure if checking $apolloData.loading on the ApollQuery level is appropriate, or if the initialisation should be ensured on a lower level.

I'm getting the same error [Vue warn]: Error in render: "TypeError: Cannot read property 'loading' of undefined" in my nuxt project when trying to use ApolloQuery.
Is it working for you when not testing components?

Yes, in the app it works fine. We are not using nuxt though, so not sure how to set it up there.

I noticed that the issue only occurs when rendering the component on the server.
Maybe this is related to your issue.

For me the issue was that I mounted apolloProvider wrong. Looking at your code it seems like you never mount an Apollo provider. Maybe there needs to be some kind of mock / testing Provider?


Please add -sqlite flag to command documentation

The tutorial to use EntityFramework, specifically the section to scaffold pages (https://docs.microsoft.com/en-us/aspnet/core/data/ef-rp/intro?view=aspnetcore-5.0&tabs=visual-studio-code#scaffold-student-pages) includes instruction for MacOS and Linux users to run this command: dotnet aspnet-codegenerator razorpage -m Student -dc ContosoUniversity.Data.SchoolContext -udl -outDir Pages/Students --referenceScriptLibraries -sqlite
All of the flags used in the command are documented here (that is, this page about the aspnet-codegenerator) except for the '-sqlite' flag. (I searched on the page (using CTRL-F) for the word "sqlite", "database", and "connection", and find no results.)
I found, by accident, this passing mention of what that flag does, "you need to add the --useSqlite or -sqlite flag for the tool to know sqlite is being used. Otherwise it looks for sqlserver." (https://github.com/dotnet/Scaffolding/issues/1393#issuecomment-683910709).
Please update this documentation to mention the use of this '-sqlite' flag.
Thanks.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 517fdd63-f70b-a52b-56fa-e5051a13ae00
Version Independent ID: 754e1c4d-4742-a83b-7f2d-5dead439e569
Content: dotnet aspnet-codegenerator command
Content Source: aspnetcore/fundamentals/tools/dotnet-aspnet-codegenerator.md
Product: aspnet-core
Technology: aspnetcore-fundamentals
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

@Rick-Anderson this issue is similar to https://github.com/dotnet/AspNetCore.Docs/issues/21789. I do think adding a section mentioning the --useSqlite flag would be a good idea.

@fiyazbinhasan  you're right, hopefully you can  PR this.


Cannot Update the Scope Validators for a SPA with Console

How to reproduce:

Create a SPA
Go to Protocol tab
Try to update scope validators by checking Role based or xacml based validators
It doesn't reflect the updated change

https://user-images.githubusercontent.com/31848014/162699145-51e39f70-c44d-477c-9ecc-f6e366b645e5.mp4
Environment information (Please complete the following information; remove  any unnecessary fields) :
IS 5.12.0 alpha 19

Fixed with https://github.com/wso2/identity-apps/pull/2951


Add caching

Issue #124
This PR introduces basic caching mechanism to reduce costs and increase effectiveness of running test suites.
Performance Boost: Achieves an average speedup of 400%-600%, automations like "Find Lionel Messi Wikipedia page" now completing about 5x faster
Flow diagram

Note: This PR is WIP and still on early stage. It's there early to gather feedback. Future tests, refinements and enhancements will be made to make it reliable and even more performant

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

async getComponentStringByCoords(x: number, y: number) {
    return await this.getPage().evaluate(
      ([x, y]) => {
        const elem = document.elementFromPoint(x, y);
        return elem?.outerHTML.trim().replace(/\s+/g, ' ');
      },
      [x, y]
    );
  }

The function above retrieves normalized component string given X and Y coordinates. The reason it's implemented this way is that currently, Playwright does not support selecting DOM elements using coordinates (see this closed issue). The only way I have found was to use native document func. If you have any other ideas in mind, please suggest 👍

Would be nice to resolve the conflicts and then I can test your branch locally.

Would be nice to resolve the conflicts and then I can test your branch locally.

@m2rads I have resolved conflicts now. You can check it out.

Refinements / Improvements:

Write new text execution to cache file only after it has been successfully completed. That way we can be sure cached tests are only successful ones
Delete cache entry if it fails to run (e.g UI change) and rewrite it

@gladyshcodes sorry we made some more changes. Please resolve the conflicts again and I will test soon. Thank you :)

@gladyshcodes sorry we made some more changes. Please resolve the conflicts again and I will test soon. Thank you :)

Hi there @m2rads. Those conflicts actually do not interfere with functionality added. I resolved them 👍 .

Seems like Vercel deployment is failing. I am also getting some build errors locally.
Steps to reproduce this error:
Remove these files:
pnpm-lock.json
node_modules
packages/shortest/node_modules
packages/shortest/dist
Then run pnpm install

@m2rads Build issue resolved, but Vercel pipeline does not re-run. I guess your approval is needed

Thanks the build issue is resolved now but I think there might be another issue. When running a test, it caches the steps successfully, but on a consecutive run, it deletes the cache and does the same process. It doesn't seem like the consecutive test is being executed from the cache as it takes the same amount of time to run.
https://github.com/user-attachments/assets/ba574583-5194-4827-8bb6-8550e636d3d6

Hmm that's weird. I have just tried running several tests it and it worked fine for me. I didn't try it with new dashboard test though. I will see now and let you know
shortest("Open Google");

I see. The new test needs setup with Mailosaur. Maybe try a more complicated test that has multiple steps and see if this issue happens.

@m2rads I have tried with this test, it works:
shortest("Open first article on Hackernews");

@m2rads I have tried with this test, it works:
shortest("Open first article on Hackernews");

Hhhm seems like this happens when you have multiple test cases. Can you try with more than one test case?

@m2rads Thank you for caching the bug! I've realized the problem was caused by several factors:

Missing sleep time after each browserTool.execute. Similar to how it's done in runner, we need to address this. However, I believe we should eliminate such code altogether since it affects the performance of both cached and uncached tests. Sometimes, waiting less than a second is sufficient for the next action, while other times, potentially, a longer wait might be needed
For dashboard.test.ts:

I managed to get it running by referencing to the source code of #183, as the relevant documentation appears to be missing. It would be helpful under the services configuration section section. Also, Clerk configuration needs updating: you’ll need to disable the “Require the same device and browser” checkmark in the Clerk dashboard for the tests with mailosaur to work (at least in my case)
Issue itself was the difference in  components from email letter as their href attrs differ every time. We now recursively clean up href attributes from the tags. The idea that if the URL changes and we need to click the link, the next step will fail

I ran several tests and they pass now:

Perf boost: 5.6x
@slavingia If you have time, take a look as well, maybe any suggestions?

Merged in auto fix which led to some new failures

Bug report:

however I've removed the button I was testing and it still pass

maybe I did something wrong, my test is just this:
shortest("Open the home page and signin via email").expect("A message to check your email");

what I did now is that I removed the "Signing via Email" link and it still pass

it seems the second time I run the tests they are run against the screenshots but don't hit the server

You can see the video here: https://x.com/madarco/status/1874141497404363128
I guess there needs to be some logic to destroy the cache, so that the spec starts to fail. We'll need to think more on the best way to do that dynamically; we can start by making it very easy (one command) to nuke the cache?


Error: import @nuxt/bridge-edge or @nuxt/bridge bring error: Cannot read properties of undefined (reading '_version')

Environment
dev
Reproduction
https://github.com/nuxt/nuxt
Describe the bug

Additional context
and other
windows11
node 18.18.2
npm 9.8.1
Logs
No response

We are approaching the Nuxt 2 EOL date (June 30, 2024) - see this article for more information. This is advance warning that I'm going to close this issue then, as it's currently marked as a Nuxt 2 related bug.
If it's a critical or security issue, please do comment and let me know, in case it is possible to address it before the EOL date.
If it's a an issue you think is relevant for Nuxt 3, please feel free to open a fresh issue (or just comment here so I can update labels, etc.). 🙏
Thank you for your understanding and for your contribution to Nuxt. ❤

It's the day, at last. Nuxt 2 is now marked end-of-life.
My apologies we never got round to resolving this bug in the 2.x branch, but we do have to draw a line somewhere. Again, if you think this is still relevant for Nuxt going forward, please feel free to open a fresh issue (or just comment here so I can update labels, etc.). 🙏


mem::uninitialized backport

Before 1.0.0, itoa used std::mem::uninitialized to use uninitialized u8s. This is now regarded as being undefined behaviour, and itoa correctly switched to MaybeUninit instead some time ago. But this switch came with a new major version, meaning crates that still depend on old itoa (like the current published version of csv) do not get that soundness fix.
See: https://github.com/rust-lang/rust/issues/66151 for more info around mem::uninitialized.
Therefore, I would suggest backporting a soundness fix for 0.4.x. We can't use MaybeUninit, since that would increase the MSRV which is probably not desired. I would suggest to zero-initialize it instead. While this can come at a minor perf impact, it's for an old and outdated version, and most people will likely have upgrade their dependencies already. This would also be a really simple change.
What's your opinion on this? Is this something we should do right now? Maybe at a later point in the future?

I don't have benchmarks, but if you're interested I can run some.

I scanned through the list of crates that depend on itoa and there's only a few that have a non-0.4 version number, so there's no real need to backport to 0.3 or lower.
(Those crates being: https://crates.io/crates/pretty_toa, https://crates.io/crates/rust-version . There's also https://crates.io/crates/parallel but that's entirely yanked so can just be ignored)

Running the benches that exist in the repo on commit de247d6 after making Buffer::new use [0_u8; I128_MAX_LEN]:
(I deleted the std fmt bench lines because they aren't important here)
test bench_itoa_fmt::bench_i16_0      ... bench:           6 ns/iter (+/- 0)
test bench_itoa_fmt::bench_i16_min    ... bench:          10 ns/iter (+/- 0)
test bench_itoa_fmt::bench_u64_0      ... bench:           4 ns/iter (+/- 0)
test bench_itoa_fmt::bench_u64_half   ... bench:          11 ns/iter (+/- 1)
test bench_itoa_fmt::bench_u64_max    ... bench:          17 ns/iter (+/- 1)
test bench_itoa_write::bench_i16_0    ... bench:           5 ns/iter (+/- 0)
test bench_itoa_write::bench_i16_min  ... bench:          12 ns/iter (+/- 0)
test bench_itoa_write::bench_u64_0    ... bench:           4 ns/iter (+/- 0)
test bench_itoa_write::bench_u64_half ... bench:          11 ns/iter (+/- 0)
test bench_itoa_write::bench_u64_max  ... bench:          16 ns/iter (+/- 0)

And with mem::uninit, how it is now.
test bench_itoa_fmt::bench_i16_0      ... bench:           4 ns/iter (+/- 0)
test bench_itoa_fmt::bench_i16_min    ... bench:           9 ns/iter (+/- 0)
test bench_itoa_fmt::bench_u64_0      ... bench:           4 ns/iter (+/- 0)
test bench_itoa_fmt::bench_u64_half   ... bench:          10 ns/iter (+/- 0)
test bench_itoa_fmt::bench_u64_max    ... bench:          16 ns/iter (+/- 0)
test bench_itoa_write::bench_i16_0    ... bench:           5 ns/iter (+/- 0)
test bench_itoa_write::bench_i16_min  ... bench:          10 ns/iter (+/- 0)
test bench_itoa_write::bench_u64_0    ... bench:           5 ns/iter (+/- 0)
test bench_itoa_write::bench_u64_half ... bench:          10 ns/iter (+/- 0)
test bench_itoa_write::bench_u64_max  ... bench:          16 ns/iter (+/- 0)

Looks marginal. Some increase, but it didn't utterly kill performance or anything. Though this is being ran on my desktop just running cargo bench, so not exactly the most scientific measurements.

Also, to be clear, this isn't urgent or anything, making mem::uninit panic for uninit u8s is likely to happen eventually, but is probably a fair way off because so many crates did it.
It's entirely possible that by the time we are able to do that, all the crates would have updated by then, but I think that's unlikely.

If history is anything to go by, code like this will be made UB by the addition of LLVM attributes before a PR which prevents that with panics is approved. So that is what I would be more concerned with.

(rust-lang/rust#99182)[https://github.com/rust-lang/rust/pull/99182] has now landed, removing all possible future LLVM UB from the way itoa uses mem::uninit. This greatly reduces the risk of using old itoa in practice, and therefore makes a backport less important. A backport would still improve the experience of people using crates with old itoa under Miri on MSAN, but I don't think it's worth it, therefore I'll close the issue.

People getting a FCW for using an ancient version of the library sounds like a good outcome to me.

There is some reluctance to adding an FCW that would show up for a large fraction of the ecosystem, with not even cargo update enough to fix the warning. So a semver-compatible update for itoa would still be very useful. itoa 0.x is still used a lot.

The code in 0.4 exactly follows best practice at the time that 0.4 was released. ManuallyDrop did not exist and the standard library documentation literally showed mem::uninitialized::<[Vec; 1000]>() as an example of correct use.

Yes, nobody is denying that.
But we're trying to fix that mess, and there are some people (including me) advocating for a strategy that aims to remove the function in future editions entirely, and there are other people that say that path only makes sense if we do an FCW in all editions.

you can make the function 0x1-init

Latest nightly already does that.

there are some people (including me) advocating for a strategy that aims to remove the function in future editions entirely

I am on board with this.

and there are other people that say that path only makes sense if we do an FCW in all editions.

My not backporting is a vote saying my strong belief that those people are misguided.

My not backporting is a vote saying my strong belief that those people are misguided.

Yeah, I can tell. ;) It's more of a veto though, in effect. So looks like either we convince t-libs to do an old-edition-only removal, or we're just stuck with having mem::uninitialized around forever (or at least for a long time)... :/


Error on some task lists from Nextcloud

First of all, thank you very much for this program. It really fills a gap for me!
Unfortunately, I am facing a sync bug when fetching some task lists from my Nextcloud instance. Errands connects to Nextcloud and fetches some lists without problems, but for two lists ("juni" and "personal" in the logs) shows no entries, although the list is not empty. From the logs (see below), it seems like it cannot find certain resources on the server. Now, these tasks exist and can be synced with e.g. the Tasks.org/DAVx5 app on my Android phone. I am not sure if this is coincidental or not, but these lists are the ones where I use more of Nextcloud Task's features like subtasks and the priority property as well as partial completion tracking. I was wondering if Errands has not yet implemented some of these things and therefore struggles to process the lists.
Here are the logs of my last start. I have redacted user names and the domain of the instance for privacy reasons, but that should not be relevant to the issue at hand. The lists with issues are "juni" and "personal". All others are synced as expected.
[DEBUG] Starting Errands 45.1.6
[DEBUG] Initialize GSettings
[DEBUG] Creating actions
[DEBUG] Loading tasks for 'personal'
[DEBUG] Loading tasks for 'mails'
[INFO] Add task:<PHONE_NUMBER>545962153
[INFO] Task: Set completed to 'True'
[INFO] Add task: 8988d8fb-1098-4c5b-8245-05256268afca
[INFO] Task: Set completed to 'True'
[INFO] Add task: f739944c-3330-4374-b7fd-a217e952d228
[INFO] Task: Set completed to 'True'
[INFO] Add task: e446bc87-630a-48df-a24f-0865e2e7b986
[INFO] Task: Set completed to 'True'
[INFO] Add task: 79c9e116-3888-4ccc-94d5-dc9232365438
[INFO] Task: Set completed to 'True'
[INFO] Add task: 020175ce-460c-491f-91fc-5ef8c0f6babe
[INFO] Task: Set completed to 'True'
[INFO] Add task:<PHONE_NUMBER>592808117
[INFO] Task: Set completed to 'True'
[INFO] Add task: d852e318-4e01-4177-991f-2e2bee84ea9c
[INFO] Task: Set completed to 'True'
[INFO] Add task: be4dd6f6-ef8a-409b-a5fa-59e9026b2b6f
[INFO] Task: Set completed to 'True'
[INFO] Add task: 949496cf-28d0-4c2b-b396-0974d246c4bb
[INFO] Task: Set completed to 'True'
[INFO] Add task: b7c0e1a7-9154-4021-9672-4b6d82e1b857
[INFO] Task: Set completed to 'True'
[INFO] Add task: c814cc54675894ea65de5e1113af632c782c7f81
[INFO] Task: Set completed to 'True'
[INFO] Add task: 35c200a7-c170-4557-ad01-fabc9f448554
[INFO] Task: Set completed to 'True'
[INFO] Add task: 02741167-0caa-4eb7-8bdd-0378f6f62f94
[INFO] Task: Set completed to 'True'
[INFO] Add task: 91702a80-6a3f-471b-b5c6-e0b8b78335f1
[INFO] Task: Set completed to 'True'
[INFO] Add task: b89d5173-8257-43c4-b9e0-f41a3093e493
[INFO] Task: Set completed to 'True'
[INFO] Add task: c840a9ce-f381-4164-85c2-37864c60f593
[INFO] Task: Set completed to 'True'
[DEBUG] Loading tasks for 'telefonate'
[INFO] Add task:<PHONE_NUMBER>31363585
[INFO] Task: Set completed to 'True'
[INFO] Add task: fdd5491d-4cc0-42a2-812b-486c077757c2
[INFO] Task: Set completed to 'True'
[INFO] Add task: 6c6a1cba-68c8-4ece-80be-6cd8dece71fd
[DEBUG] Loading tasks for 'juni'
[DEBUG] Loading tasks for 'zuknftige-projekte'
[INFO] Add task: c72ab5b6-2845-491c-900e-29c5ef542dbc
[INFO] Task: Set completed to 'True'
[INFO] Add task: 5e4e7d1b-15fc-40b7-afd2-0fca62d15cf3
[INFO] Task: Set completed to 'True'
[DEBUG] Loading tasks for 'personal_shared_by_<redacted>'
[DEBUG] Lists: Switch list
[DEBUG] Lists: Switch list
[INFO] Sync: Initialize sync provider
[INFO] Sync: Initialize 'Nextcloud' sync provider
[DEBUG] Sync: Checking credentials
[DEBUG] Sync: Checking URL
[DEBUG] Sync: URL is set to https://<nextcloud-instance>/remote.php/dav/
[DEBUG] Sync: Attempting connection
[INFO] Sync: Connected to Nextcloud server at 'https://<nextcloud-instance>/remote.php/dav/'
[INFO] Sync: Sync tasks with remote
[DEBUG] Sync: Getting tasks for personal
[DEBUG] Sync: Delete orphaned task: [vText('b'6542c8e5-4115-4ebe-aee0-96fda23b1503''), vText('b'a5879de6-e99b-4388-979d-a8868c7bf8a8''), vText('b'890bbc20-fed3-401c-8838-18e47377e13c''), vText('b'7c6bbc36-29fa-4e45-91b4-2137f270fe2f'')]
[ERROR] Sync: Can't get tasks from remote. NotFoundError at '[vText('b'6542c8e5-4115-4ebe-aee0-96fda23b1503''), vText('b'a5879de6-e99b-4388-979d-a8868c7bf8a8''), vText('b'890bbc20-fed3-401c-8838-18e47377e13c''), vText('b'7c6bbc36-29fa-4e45-91b4-2137f270fe2f'')] not found on server', reason no reason
[DEBUG] Sync: Getting tasks for mails
[DEBUG] Sync: Getting tasks for telefonate
[DEBUG] Sync: Getting tasks for juni
[DEBUG] Sync: Delete orphaned task: 3912e442-afc7-4360-a1d8-60735430f8e0
[ERROR] Sync: Can't get tasks from remote. NotFoundError at '3912e442-afc7-4360-a1d8-60735430f8e0 not found on server', reason no reason
[DEBUG] Sync: Getting tasks for zuknftige-projekte
[DEBUG] Sync: Getting tasks for personal_shared_by_<redacted>
[DEBUG] Lists: Update UI...
[DEBUG] Task list personal: Update UI
[DEBUG] Task list mails: Update UI
[DEBUG] Task list telefonate: Update UI
[DEBUG] Task list juni: Update UI
[DEBUG] Task list zuknftige-projekte: Update UI
[DEBUG] Task list personal_shared_by_<redacted>: Update UI
[DEBUG] Lists: Switch list
[DEBUG] Lists: Switch list
[DEBUG] Lists: Switch list
[DEBUG] Lists: Switch list

Judging by the logs, in this problematic lists tasks have parent task, but parent is not exists in the list (they are orphaned and because of that - deleted). Errands can read all properties of the task like priority and completion percent. Can you write an example how to replicate this task list? Maybe try to create new list with same tasks and properties and look how it behaves.

In the meantime I'll make that orphaned tasks is not deleted and just remove their parent property

I have not been able to reproduce the issues on a new task list when trying different things like deleting a task with unfinished subtasks or deleting some subtasks etc. But I have been able to fix the issue on the "personal" list (have not tried the other one yet) by having Nextcloud delete all finished tasks. I think it is likely that one of these old tasks was corrupted and once it was removed, Errands was able to sync without issue. So it mainly seems to be an issue with the source, but adding a workaround that prevents one corrupted task from not syncing the whole list seems to be a good idea. Thank you for your support and feel free to close the issue.

Thank you for reporting the issue! I already added fix for this 22c5ca3.


Support new MSBuild platform (at the .NET Core/Standard environments)

3b6510b099971163f959c0fa69331417c4499758
In 0.9.1 supported.


Immutable DB validation should check that chunk files don't have too many entries

Marking as low priority for now since it's extremely unlikely that disk corruption will add additional (valid!) blocks to a chunk (aka epoch) file.


[DH-335] Make default search type whole string.

It changes the query to look for exact phrases first and is giving lower priority to incomplete matches.

Codecov Report

Merging #268 into develop will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff            @@
##           develop     #268   +/-   ##
========================================
  Coverage    93.39%   93.39%
========================================
  Files           56       56
  Lines         2044     2044
  Branches       189      189
========================================
  Hits          1909     1909
  Misses         114      114
  Partials        21       21

Impacted Files
Coverage Δ

datahub/search/elasticsearch.py
98.38% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2583d86...9d5859a. Read the comment docs.


Data Science with Python: Least angle Regression (LARS)

Description
Welcome to the 'DSWP' Team, good to see you here.
With this issue, readers will get introduced to the core information about 'Least angle Regression (LARS)'.
To get assigned to this issue, add your serial numbers mentioned in the spreadsheet of "Data Science with Python", the approach one would follow and the choice you prefer (Documentation, Audio, Video). You can go with all three or any number of options you're interested to work on.
If you had referred any resources, add them up in "DS Resources". Similarly, if you had used datasets, include them in "DS Datasets".
Domain: Statistics
Mentors Assigned : @Pushpit07 / @aishanii
Points to Note :
The issues will be assigned on a first come first serve basis, 1 Issue == 1 PR.
"Issue Title" and "PR Title" should be the same. Include issue number along with it.
Changes should be made inside the Datascience_With_Python/ directory & Datascience_With_Python branch.
Follow Contributing Guidelines & Code of Conduct before start Contributing.
This issue is only for 'GWOC' contributors of 'DSWP' domain.
All the best. Enjoy your journey ahead. 😎
Domain
Datascience with Python
Type of Contribution
Audio, Video, Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

Issue assigned to @vishnupriya129 for documentation. Your approach is not sufficient. It should be detailed enough.

Greetings mam @prathimacode-hub, as this issue hasn't been solved since 13 days, I would like to work on the documentation part on my end.
Details -->
Name: Jay Gohil
Serial No: 124
Batch: 4
Issue Number: 6365
Type: Documentation
Approach: Create a detailed Jupyter Notebook with code as well as explanation of Least Angle Regression that provides all necessary details pertaining to LAR in a comprehensive manner.
Please assign this to me!

@prathimacode-hub mam I have worked on it but when I make PR then the changes are automatically adding in my old PR what should I do to create pull request for this issue.

Do you have any pending PR's? @vishnupriya129

Greetings mam @prathimacode-hub, as this issue hasn't been solved since 13 days, I would like to work on the documentation part on my end.
Details --> Name: Jay Gohil Serial No: 124 Batch: 4 Issue Number: 6365 Type: Documentation
Approach: Create a detailed Jupyter Notebook with code as well as explanation of Least Angle Regression that provides all necessary details pertaining to LAR in a comprehensive manner.
Please assign this to me!

Contributor work's is done, she is yet to do PR. @gohil-jay

@prathimacode-hub my another PR review is in pending mam

@prathimacode-hub my another PR review is in pending mam

from which domain? You can take help from mentors who are assigned to you. They will help you out. I could have but I have no data or software in my laptop to showcase you with the process. I lost all to laptop malfunctioning. @vishnupriya129

@prathimacode-hub In this domain only mam. issue no: #5977 Evaluating Algorithms. My PR is under your review only mam.

Changes has been requested. Is it done? @vishnupriya129

@prathimacode-hub I am working on it. I will complete it soon.

@prathimacode-hub Mam can you please review this PR

Hello, I am Deepthi M with serial number:172, Batch-5. I would like to do audio on this issue.
My approach: a detailed explanation

Issue assigned to @deepthi1107 for audio contribution

@prathimacode-hub , Mam
I am M. Ashish Reddy. I would like to do video on this issue.
Batch number: 5
serial number:174.
Approach: A detailed explanation with examples and real life uses

Issue assigned to @ashish-reddy-20-08 for video contribution


issue on commit 159b0c1: angular missing module

i tried to start the application but there is an angular error caused by geo.js 'cause it now imports an undefined module: geodashadmin.
only geo-dash.ftl includes the javascript in which is defined geodashadmin module.
i've fixed including the script also in the header.ftl but I don't know if it is ok:

:)

Sorry about that, i had done it wrong, just including in the other ftl.  I updated the geo-dash.ftl as well to reflect the change.

Thanks for catching that, Roberto. I fixed your fix, Billy. ;-)


Fix capatable for turbo0301 model

Just trying to understand the code. You made the properties optional because the API doesn't always return the properties? Is that why you kept the init non-optional?

@ronaldmannak yep

@KittenYang Can you provide example requests/responses?

Closed due to inactivity.


Initial skeleton for v2

Major simplifications and new abstractions, renamed napalm-srl --> napalm-srlinux, adding typing and reimplementing PoC methods for get_bgp_neighbors, get_lldp_neighbors, get_users.

CI/CD tests are failing, you can check locally using make run-tests (after deploying the test lab)

CI/CD tests are failing, you can check locally using make run-tests (after deploying the test lab)

Thanks mate, we’ll probably review the tests and look to improve them. We want to make the driver a bit more friendly to stub and test in general.

@hellt We probably don’t have to reinvent the wheel here, these types are already in napalm.base. https://github.com/napalm-automation/napalm/blob/develop/napalm/base/models.py#L25

yes, we can use the upstream types (and extend them when needed)


Pin recent version h5py and tifffile

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Pinning to take to pin h5py and tifffile, which are not compatible with hyperspy 1.6.0. See commit messages for relevant PRs.

[x] Pin dependencies until next release.
[x] Add pip check.
[x] Move optional dependencies (scikit-learn, mrcz, pyusid, mrcz, imagecodecs) from hyperspy-base to hyperspy package.
[x] Use dask-core to avoid installing unnecessary dependencies.
[x] Order dependency list in alphabetical order.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.
I do have some suggestions for making it better though...
For recipe:

It looks like the 'hyperspy' output doesn't have any tests.
It looks like the 'hyperspy-dev' output doesn't have any tests.

@conda-forge-admin, please rerender


Code style

[ ] All tests pass
[ ] I have run npm run doc

Description

References

The library uses airbnb base style.


Add unit tests for Simple Display output; improve either/or wording.

This PR includes 4 improvements:

It adds unit tests for the Display of Simple and tests a variety of Simple<char> display cases.
It sorts the display of expected tokens, so that they are stable in UIs and to facilitate parse error messages testing.
It changes expected display of a pair from one of "x", "y" to either "x" or "y".
It changes expected display of three or more from one of "x", "y", "z" to one of "x", "y", or "z".

Note that this includes a test case for custom errors, and asserts that the display is found end of input even though this is a bug! A potential future branch fixing that bug needs to update that test case appropriately.

Thanks. What's the purpose of using indoc/etc.?

I'm going to close this since it's very out of date. Happy to reopen it in the future if that changes though.


Vulnerability in a Go pkg

Describe the bug
➜  nginx-supportpkg-for-k8s git:(main) ✗ govulncheck -show verbose ./...
Scanning your code and 946 packages across 138 dependent modules for known vulnerabilities...

Fetching vulnerabilities from the database...

Checking the code against the vulnerabilities...

=== Symbol Results ===

Vulnerability #1: GO-2024-2687
    HTTP/2 CONTINUATION flood in net/http
  More info: https://pkg.go.dev/vuln/GO-2024-2687
  Module: golang.org/x/net
    Found in<EMAIL_ADDRESS>    Fixed in<EMAIL_ADDRESS>    Example traces found:
      #1: pkg/data_collector/data_collector.go:93:50: data_collector.NewDataCollector calls kubernetes.NewForConfig, which eventually calls http2.ConfigureTransports
      #2: cmd/nginx-supportpkg.go:96:27: cmd.Execute calls cobra.Command.Execute, which eventually calls http2.ConnectionError.Error
      #3: pkg/data_collector/data_collector.go:110:28: data_collector.DataCollector.WrapUp calls fmt.Sprintf, which eventually calls http2.ErrCode.String
      #4: pkg/data_collector/data_collector.go:110:28: data_collector.DataCollector.WrapUp calls fmt.Sprintf, which eventually calls http2.FrameHeader.String
      #5: pkg/data_collector/data_collector.go:110:28: data_collector.DataCollector.WrapUp calls fmt.Sprintf, which eventually calls http2.FrameType.String
      #6: cmd/nginx-supportpkg.go:96:27: cmd.Execute calls cobra.Command.Execute, which eventually calls http2.GoAwayError.Error
      #7: pkg/data_collector/data_collector.go:110:28: data_collector.DataCollector.WrapUp calls fmt.Sprintf, which eventually calls http2.Setting.String
      #8: pkg/data_collector/data_collector.go:110:28: data_collector.DataCollector.WrapUp calls fmt.Sprintf, which eventually calls http2.SettingID.String
      #9: cmd/nginx-supportpkg.go:96:27: cmd.Execute calls cobra.Command.Execute, which eventually calls http2.StreamError.Error
      #10: pkg/data_collector/data_collector.go:251:5: data_collector.DataCollector.QueryCRD calls rest.Request.Do, which eventually calls http2.Transport.NewClientConn
      #11: pkg/data_collector/data_collector.go:251:5: data_collector.DataCollector.QueryCRD calls rest.Request.Do, which eventually calls http2.Transport.RoundTrip
      #12: pkg/data_collector/data_collector.go:262:14: data_collector.DataCollector.AllNamespacesExist calls fmt.Printf, which eventually calls http2.chunkWriter.Write
      #13: cmd/nginx-supportpkg.go:96:27: cmd.Execute calls cobra.Command.Execute, which eventually calls http2.connError.Error
      #14: cmd/nginx-supportpkg.go:96:27: cmd.Execute calls cobra.Command.Execute, which eventually calls http2.duplicatePseudoHeaderError.Error
      #15: pkg/jobs/nic_job_list.go:80:22: jobs.NICJobList calls http2.gzipReader.Close
      #16: pkg/jobs/nic_job_list.go:74:26: jobs.NICJobList calls io.Copy, which eventually calls http2.gzipReader.Read
      #17: cmd/nginx-supportpkg.go:96:27: cmd.Execute calls cobra.Command.Execute, which eventually calls http2.headerFieldNameError.Error
      #18: cmd/nginx-supportpkg.go:96:27: cmd.Execute calls cobra.Command.Execute, which eventually calls http2.headerFieldValueError.Error
      #19: pkg/data_collector/data_collector.go:251:5: data_collector.DataCollector.QueryCRD calls rest.Request.Do, which eventually calls http2.noDialH2RoundTripper.RoundTrip
      #20: cmd/nginx-supportpkg.go:96:27: cmd.Execute calls cobra.Command.Execute, which eventually calls http2.pseudoHeaderError.Error
      #21: pkg/data_collector/data_collector.go:262:14: data_collector.DataCollector.AllNamespacesExist calls fmt.Printf, which eventually calls http2.stickyErrWriter.Write
      #22: pkg/jobs/nic_job_list.go:80:22: jobs.NICJobList calls http2.transportResponseBody.Close
      #23: pkg/jobs/nic_job_list.go:74:26: jobs.NICJobList calls io.Copy, which eventually calls http2.transportResponseBody.Read
      #24: pkg/data_collector/data_collector.go:110:28: data_collector.DataCollector.WrapUp calls fmt.Sprintf, which eventually calls http2.writeData.String

=== Package Results ===

No other vulnerabilities found.

=== Module Results ===

Vulnerability #1: GO-2024-2611
    Infinite loop in JSON unmarshaling in google.golang.org/protobuf
  More info: https://pkg.go.dev/vuln/GO-2024-2611
  Module: google.golang.org/protobuf
    Found in<EMAIL_ADDRESS>    Fixed in<EMAIL_ADDRESS>
Your code is affected by 1 vulnerability from 1 module.
This scan also found 0 vulnerabilities in packages you import and 1
vulnerability in modules you require, but your code doesn't appear to call these
vulnerabilities.

To Reproduce
Steps to reproduce the behavior:

Run govulncheck -show verbose ./...

Expected behavior

No fixed vulnerabilities reported.

Screenshots
N/A
Environment
➜  nginx-supportpkg-for-k8s git:(main) ✗ govulncheck --version
Go: go1.22.4
Scanner<EMAIL_ADDRESS>DB: https://vuln.go.dev
DB updated: 2024-06-20 18:18:26 +0000 UTC

Additional context
N/A
### Tasks
- [ ] https://github.com/nginxinc/nginx-supportpkg-for-k8s/pull/11

@jjngx , thanks for reporting this, we shall address this shortly.

Solved and merged in #11.


No love for Python 3?

python --version
    Python 3.4.2

    from decorators import *
ImportError: No module named 'decorators'

This should be fixed. I had already fixed it in master, but I hadn't pushed a release to pypi.

Thanks, I will check out master.


Rename _iroha_wasm_main() for triggers and validators

Originally suggested by @mversic here.
The idea is to rename _iroha_wasm_main() to:

_iroha_trigger_main() -- for triggers;
_iroha_validator_main() -- for validators.

This will require changes on both Iroha side and procedural macros side.

no, my suggestion was to have only one name _iroha_wasm_main instead of multiple different


 It can wait until next restart.

This is at least partly my fault for merging bumba's refactor so hastily.

Also, getIdentityWithStatus can now just be name or name + '@!', right?

Given that (afaik) we only use the status message in the userdetails popup, yes. If we want to include them in join/leave messages then we'd need to keep it.


support build in arm64 platform

Signed-off-by: xiang<EMAIL_ADDRESS>close #97
But the clock attack not supported in arm64 platform

/merge