“Continuous Loading indicator” is shown after tapping on “Don’t have an Invite Code” button from Invite Code page

Frequency: 100%
Repro App version:  Android Celo internal build 1.9.1
Does not Repro App version: IOS Test flight Celo integration build v1.9.1 (41)
Repro on device: Samsung Galaxy A5(7.0), Google pixel 3a(11.0), Samsung Grand Prime(5.1.1)
Does not Repro on device: iphone 7(13.5.1), iphone 11(14.0)
Repro Steps:

Launch the Application and Complete the signup process till Invite Code page
Tap on “Don’t have an Invite Code” button
Observe Continuous loading indicator is shown and user does not forward to Home page

Current Behavior: Continuous loading indicator is shown after tapping on “Don’t have an Invite Code” button
Impact: User is block to sign up without invite code
Expected Behavior: On tapping “Don’t have an Invite Code” button user should forward to Home page
Attachment: ContinuousLoadingindicatorIssue.mp4

Verified with Play store internal Valora build v1.9.4 and found that issue is fixed


Add Array#bsearch, Array#bsearch_index, Range#bsearch

Currently Crystal lacks binary search, which is a great feature of Ruby 2.0+.
I have (partially) ported https://github.com/marcandre/backports/blob/master/lib/backports/2.0.0/range/bsearch.rb .
The only point is that current port lacks Float support, which requires reinterpreting Float64 as Int64.
And, ->{ (pointerof(midpoint) as Pointer(Int64)).value }.call caused
Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/manastech/crystal/issues

Anyway, the Int version works flawlessly. How do you think of this pull request?
Currently it is also available as https://github.com/cielavenir/crystal-bsearch shard and the spec will be ported after approval (I'll need to fix the indent).

What's the use case of Range#bsearch?

I sometimes use (1..10**9).bsearch{|i|predicate(i)}.

Committed part1 fix (I'll rebase them into one commit after review is done)

I'm not sure I like Ruby's bsearch and its two modes. I'm reading the docs and find them very confusing. Having one method with different "modes" is a bad idea in my opinion. In some languages I know (C#, Java, Go) binary search returns the index of an element that matches some condition, or just equals some element. What's the reason for Ruby's bsearch to be so complex?

Yes. actually I don't use Numeric mode at all. I can remove it.

For Float binary-search, I wrote pseudo code. Although it doesn't compile (yet), you might know what I mean by using block.
    midpoint = 0 # placeholder
    if from.is_a?(Int) && to.is_a?(Int)
      convert = -> { midpoint }
    else
      map = ->(T,v) {
        result = (pointerof(v.abs) as Pointer(T)).value
        v<0 ? -result : result
      }
      from = map.call(UInt64,from.to_f)
      to = map.call(UInt64,to.to_f)
      convert = -> { map.call(Float64,midpoint) }
    end

I'd like to review this with @waj, so it'll have to wait at least a couple of weeks

Well, I wrote https://github.com/cielavenir/crystal-bsearch/blob/float/lib/bsearch.cr#L31 , but convert.call fails to assert type. If from.is_a?(Int) && to.is_a?(Int), convert.call must be Int, not (Int32|Int64|Float64). I think it would be very hard to fix, so I feel @MakeNowJust 's implementation is better in this point.

Closed in favor of #1871 because it also includes Float range. Thank you in any case!


replace AES ciphermode CFB

CFB is not supported on .net core.
see also https://github.com/dotnet/corefx/issues/4647

Looks like this function is not used anywhere yet so this should be just fine.


Parallelize stage 2 to reduce branches

I promised thoughts: here's a sketch.
Currently, we have a 2 stage parser: stage 1 scans the input produces a list of buffer indexes to each structural token, and then stage 2 iterates over that list once, branching to parse and validate atoms, structures, strings and numbers all in one series.
This is branch-heavy because stage 2 largely doesn't know what it will see next, and needs to do drastically different things depending on what it sees. Essentially, the problem is that we walk up to an index and then run 1 of 4 very different functions--requiring a real branch.
Proposal
In this proposal, instead of producing a single list of structural indexes, stage 1 puts each one in a different list depending on type (string, number, or atom). It also produces a full list of structural bytes (first character / type of each structural). Then stage 2 runs as 4 independent loops, each one handling a different job (string, number, atom, structure).
Scanner (Stage 1)
Scans for structure and values, and produces several lists:

Structural tokens, the closest thing to the original structural_indexes. This is a full list of structurals ({[]},: and the start of each string, atom and number), and does not contain JSON indexes.
String indexes, a list of start string indexes.
Number indexes, a list of start number indexes.
Atom indexes, a list of start atom indexes. (Possibly separate lists of true, false, and null.)
Other indexes, anything else. We only write to this buffer; nobody reads from it. It exists so we don't have to branch on write.

The scanner produces the index lists branchlessly, because the index arrays are themselves in an array indexed by character type: json_indexes[value_type(buf[idx])] = idx;. (There are still branches for the number of structurals in a block--that is unchanged.)
Parsers/Validators (Stage 2)
Stage 2 gets split into 4 separate loops:

String Parser: Parses all strings by iterating over strings indexes, scanning for \ and " and outputting the parsed strings.
Number Parser: Goes through all numbers, parses them, and outputs a list of parsed numbers.
Atom Validator: Validates atoms.
Structure Validator: Matches braces, and validates , and :. Iterates through structural tokens to do this. A tape just like the existing tape could be produced here, as long as number and string parsing is already complete.

Iterator
We could definitely produce a tape identical to what we currently have. However, it might be worth iterating the three separate outputs (structurals, strings, numbers) directly.
To do it, keep 3 pointers in your iterator instead of 1: next_structural, next_string, next_number. Whenever you reach a string, you return it and bump the next_string pointer. When you reach a number, you return it and bump the next_number pointer. And you always bump next_structural. This makes the parsed json smaller (at the cost of more iteration machinery / registers). It also eliminates indexes, allowing arbitrary-sized JSON to be represented.
It can be done either forward or backward (you can't start in the middle, though).
Pros/Cons
Pros:

Fewer branches: This being the original point.
Less to read: The structural validator doesn't have to read 32-bit indexes or access the buffer at all, so we'll read 32 bytes less there (and 1 read instead of 2). It's a bit of a wash with respect to values, because those parsers will read an index per value, and the structural validator will still read a byte. It's just split into different places.
Smaller primary tape: The primary tape for iteration uses one byte per structural. This means skipping through things can be a lot faster, and there's even SIMD potential there (load up 32 structural chars and process it all at once). The number and string tapes still exist, but you don't store indexes into them.
Arbitrary sized documents: The proposed iterator tape does not store structural indexes or JSON indexes. In fact, the only size-limiting thing there is the string length we store.

Cons:

+25% Memory Use: Because we store a byte for each structural as well as the index, the intermediate results will be 25% bigger. The number of reads will be the same, however, and we'll actually read fewer bytes than before since the structural validator iterates a byte at a time rather than a 32-bit index.
Possibly More Writes: Depending on how the write works, we'll end up with up to 2 writes per structural instead of 1 (the 1 byte and the 4-byte index), and probably a few extra writes per block to update the lengths.
Heavier Iterator: If you iterate over the proposed structure directly, without producing a combined tape, your iterator is tracking 3 pointers instead of 1.
Reads: This causes us to loop over the buffer 4 times, instead of our current 2. It does not translate to twice the reads, however: we still read the buffer once per index during parsing, just we start back at the beginning for each of the 3 loops. So this could have cache implications, but at least shouldn't increase read contention.

Further Possibilities
Strings and numbers in particular are still individually branchy (though hopefully the branches won't interfere with each other as much). They would both benefit from sending more indexes during scanning.

For strings, if a separate list of "bytes to copy" and "escapes to process" were produced branchlessly using masks, we could parse all strings in two loops (one copying string bytes, and one cranking over escape characters and writing them to the right place).
For numbers, sending each integer separately from the +-.eE characters lets you parse integers in one particularly efficient loop, and then combine them to form floats (which is what we already do, just with a bunch of branches).

If it's possible to produce "continuation indexes" at block boundaries, you could even unfold the loops in the parsers: for example, if you have a super long string, you could produce an index every 16-32 bytes, allowing the "string copy" to do a single SIMD load and store every iteration.
Collections and separator validation could use some optimization as well. All structure except brace matching and : can be validated during scanning, for example. I haven't fully thought through how to do this right, though.
We can also explore the benefits of storing values of the same type next to each other:

SIMD can be used to process multiple integers or floats, if you do it right.
If we store string length separate from string bytes, we can validate UTF-8 in all strings with maximal efficiency by doing it on the string tape.
Many arrays are single typed, making it theoretically possible for a user to consume (for example) an array of numbers directly from a pointer to the array, without having to pick apart values from type markers in the current tape.

Stage 2 can be usefully multithreaded, as well! Each parser has a lot of independent work to do. Has cache implications, but it's probably worth exploring at some point.
It's likely we'll be able to eliminate gotos from stage 2 as well, once we've gotten rid of enough branches. Not 100% on this though.

Yeah, that seems like a good plan. Making different iterators is the first thing we have to do, anyway.


Sample ID cross-checking

Modules 00b-0506 have multiple inputs of Array[File] or File that assume consistent sample IDs.
For example, Module 01 has manta_vcfs, delly_vcfs, wham_vcfs, melt_vcfs, del_bed, and dup_bed. The sample IDs in these files are never cross-checked for consistency, eg the sample ID in manta_vcfs[i] matches that of wham_vcfs[i] and also exists in the two CNV bed files.
We should implement some simple checks in these modules to catch input errors caused by the user.

This has mostly been addressed through the implementation of the GetSampleIdsFromVcf utility task to fetch sample IDs rather than providing them explicitly as an input. I'm closing this unless we start to experience this issue again.


HBASE-24964 Remove MetaTableAccessor.tableExists

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 27s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 24s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
3m 39s
master passed

+1 :green_heart:
checkstyle
1m 34s
master passed

+1 :green_heart:
spotbugs
3m  9s
master passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 14s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 22s
the patch passed

-0 :warning:
checkstyle
1m  7s
hbase-server: The patch generated 4 new + 176 unchanged - 10 fixed = 180 total (was 186)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
11m 18s
Patch does not cause any errors with Hadoop 3.1.2 3.2.1.

+1 :green_heart:
spotbugs
3m 37s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 38s
The patch does not generate ASF License warnings.

37m 42s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2330

Optional Tests
dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle

uname
Linux 0100d1212180 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / ef5da4a61d

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt

Max. process+thread count
94 (vs. ulimit of 12500)

modules
C: hbase-balancer hbase-server hbase-backup U: .

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 15s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
4m 31s
the patch passed

+1 :green_heart:
compile
2m  6s
the patch passed

+1 :green_heart:
javac
2m  6s
the patch passed

+1 :green_heart:
shadedjars
7m 44s
patch has no errors when building our shaded downstream artifacts.

-0 :warning:
javadoc
0m 18s
hbase-balancer in the patch failed.

-0 :warning:
javadoc
0m 43s
hbase-server in the patch failed.

-0 :warning:
javadoc
0m 21s
hbase-backup in the patch failed.

_ Other Tests _

+1 :green_heart:
unit
0m 26s
hbase-balancer in the patch passed.

+1 :green_heart:
unit
143m 47s
hbase-server in the patch passed.

+1 :green_heart:
unit
11m 39s
hbase-backup in the patch passed.

192m 43s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2330

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 2651ca1e798a 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / ef5da4a61d

Default Java
2020-01-14

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-backup.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-balancer.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-balancer.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-backup.txt

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/testReport/

Max. process+thread count
5009 (vs. ulimit of 12500)

modules
C: hbase-balancer hbase-server hbase-backup U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  2s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 23s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
4m  0s
master passed

+1 :green_heart:
compile
1m 36s
master passed

+1 :green_heart:
shadedjars
7m 12s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
1m  7s
master passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 14s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m 48s
the patch passed

+1 :green_heart:
compile
1m 37s
the patch passed

+1 :green_heart:
javac
1m 37s
the patch passed

+1 :green_heart:
shadedjars
7m  5s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
1m  6s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
0m 20s
hbase-balancer in the patch passed.

+1 :green_heart:
unit
213m 13s
hbase-server in the patch passed.

+1 :green_heart:
unit
13m 21s
hbase-backup in the patch passed.

258m 16s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2330

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux c0ca07353c11 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / ef5da4a61d

Default Java
1.8.0_232

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/testReport/

Max. process+thread count
3571 (vs. ulimit of 12500)

modules
C: hbase-balancer hbase-server hbase-backup U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2330/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.


Execute "export.sh" for esp32 in branch of release/4.3 failed

Problem
PC: macOS v10.15.6
bash: v5.1.8, installed by "brew"
When I using export.sh to setup my idf enviroment, it will be failed at the end of command.
PM80266089:esp-idf 80266089$ . ./export.sh
Detecting the Python interpreter
Checking "python" ...
Python 3.9.1
"python" has been detected
Adding ESP-IDF tools to PATH...
Using Python interpreter in /Users/80266089/work/chip/esp/toolchains/python_env/idf4.3_py3.9_env/bin/python
Checking if Python packages are up to date...
Python requirements from /Users/80266089/work/chip/esp/esp-idf/requirements.txt are satisfied.
Added the following directories to PATH:
/Users/80266089/work/chip/esp/esp-idf/components/esptool_py/esptool
/Users/80266089/work/chip/esp/esp-idf/components/espcoredump
/Users/80266089/work/chip/esp/esp-idf/components/partition_table
/Users/80266089/work/chip/esp/esp-idf/components/app_update
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:
idf.py build
-bash: complete: nosort: invalid option name
WARNING: Failed to load shell autocompletion!

如果没有解决，可以看看我的，小伙子你这个错误很神奇，没遇到过，但是有类似的错误，也是bash的，不保证有用哈，去你的export.sh下，159行增加LANG=en，我的bash问题是这样解决的 @Hugh-Bao

MacOS环境会出这个问题，也在export.sh 159行添加了LANG=en，但是目前不影响使用。（I have added "LANG=en" at line 159 in expot.sh, and this problem still exists, but it has no impaction for my testing for now）

We're in progress of reviewing older issues. Is this still an issue given how old the SDK this was tested against? Please re-open if so. Thanks!


encoding/binary: binary.Read doesn't respect byteOrder when []byte is passed

What version of Go are you using (go version)?

$ go version
go version go1.15 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/taka/.cache/go-build"
GOENV="/home/taka/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/taka/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/taka/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/taka/go/src/github.com/nokute78/go-bit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build405174845=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I tried to read 6 bytes using binary.Read in BigEndian.
https://play.golang.org/p/RUvRdkc0a0W
What did you expect to see?
ret(big)=ffeeddccbbaa
ret(lit)=aabbccddeeff

What did you see instead?
binary.Read returns same value and it seems to ignore byteOrder.
ret(big)=aabbccddeeff
ret(lit)=aabbccddeeff

Background:
The GUID Partition Table (GPT)  has mixed endian GUID.
Refer to https://developer.apple.com/library/archive/technotes/tn2166/_index.html#//apple_ref/doc/uid/DTS10003927-CH1-SECTION2
 the GUID for the ESP partition type is shown in the documentation as C12A7328-F81F-11D2-BA4B-00A0C93EC93B, but the bytes written to disk are 0x28 0x73 0x2a 0xc1 0x1f 0xf8 0xd2 0x11 0xba 0x4b 0x00 0xa0 0xc9 0x3e 0xc9 0x3b.

The GUID is encoded like

4bytes (Little Endian)
2bytes (Little Endian)
2bytes (Little Endian)
2bytes (Big Endian)
6bytes (Big Endian)

The point is the last 6bytes 0x00 0xa0 0xc9 0x3e 0xc9 0x3b. It is encoded as Big Endian.
How can we get using binary.Read ?
Does binary.Read support such case?

This is because the byte ordering is the byte ordering of the target type which has no meaning when the target is only 8 bits wide

@davecheney Thank you for comment.

the target is only 8 bits wide.

It is an element size (not an entire slice).
So binary.Read supports only 1,2,4,8,16 bytes which are the size of numeric types, right ?

guid is 16 bytes. That’s it, no endianness needed.

I think we should take care endianness.
MS supports  a GUID API to avoid Guid endianness issue. (refer to Remarks)
https://docs.microsoft.com/en-us/archive/blogs/openspecification/guids-and-endianness-endi-an-ne-ssinguid-or-idne-na-en-ssinguid

So binary.Read supports only 1,2,4,8,16 bytes which are the size of numeric types, right ?

it supports integer types of varying but sizes. If you decode into something larger than a byte then endianness comes into play.

binary.Read does not support what you are trying to do.
binary.Read ignores the endian argument if the data you're reading into is a byte slice. You have to read into a larger type, e.g. int32, for endianness to matter.
You can always reverse your byte slice after reading.
func reverse(b []byte) {
   for i := 0; i < len(b)/2; i++ {
       b[i], b[len(b)-1-i] = b[len(b)-1-i], b[i]
   }
}

@davecheney @randall77 Thank you for comment and example code !
I understood binary.Read doesn't support what I want to do.
So, I close this issue.


spring-cloud-contract-wiremock 2.2.3.RELEASE and above breaks test context reloading when using different base classes

Hi,
I'm upgrading from Hoxton.SR4 to Hoxton.SR9 and my contract tests started to break.  I have a project to illustrate this https://github.com/davidmelia/spring-cloud-contract-wiremock-issue
I have figure out the issue is localised to the changes in
		<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-contract-wiremock</artifactId>
				<!-- 				<version>2.2.2.RELEASE</version> WORKS -->
				<!-- 				<version>2.2.3.RELEASE</version> DOES NOT WORK -->
				<version>2.2.5.RELEASE</version>
				<scope>test</scope>
		</dependency>

where 2.2.2.RELEASE works fine but anything above this breaks my tests.
I am using the Wiremock API and have two base classes Dave1ContractBase and Dave2ContractBase.
Thanks

I don't now if its related, but we are facing an issue with Wiremock as well when updating from Hoxton.SR4 to a newer version. Our setup looks similar to yours.
I found out, that in our case during Wiremock bean creation "https-port" is being set to a random port, though we have explicitly configured.
@ActiveProfiles( profiles = { "test" } ) @AutoConfigureWireMock( port = 0 )
However, adding the following line to our application-test.yml fixed the issue. Seems like an odd workaround for me.
wiremock:   server:     https-port: -1


undefined reference to `MdfChannelCreateChannelComposition

I am building from main and configuring using the following command
cmake -S . -B build/desktop -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_PREFIX=pwd/opt/libs/mdflib/desktop -DMDF_BUILD_SHARED_LIB_EXAMPLE=ON -DMDF_BUILD_TOOL=ON -DMDF_BUILD_TEST=ON
I get a linker error for the above function. The functions seem to be declared using MACROS so it is a bit confusing on where I should be looking for a fix.

I will test your combination but set the MDF_SHARED_LIB=OFF. This disable the .NET mdflibrary build, assuming you not need the C# interface. The MDF_LIB_TOOL=OFF speed up the build time as the wxWidgets third-party library take some time to build. Note that wxWidgets currently doesn't build on Linux/MacOS. It is a VCPKG problem, so there is nothing I can do.
Can you supply me with which operating system you working with?

I tried and I also gets a lot of link errors that indicate that you mixing C++ and C# targets. Setting the MDF_SHARED_LIB=OFF will fix the "mixing" linkage problem but I suspect you will get some other problem with third-party library. If you running on Windows, the VCPKG solves most of the problems with third-party libraries. If you have VCPKG installed, you should add the -DCMAKE_TOOLCHAIN_FILE=<vcpkg-root>/scripts/buildsystems/vcpkg.cmake. Expect some extra build time at first build.

If you not using the VCPKG, I recommend to invest in a component directory (-DCOMP_DIR=path to third-party libraries). The ihedvall/components contains script files for most third-party libraries. Little bit more manual work but works on most operating systems.

@ihedvall By the way, is there any example for ChannelCompositions. I will add some test for it.

Channel compositions are typically used for in bus logging. If you want typically bus logger files, the CSS Electronics have some logger files available. There are also some unit tests as in TEST_F(TestWrite, Mdf4MlsdCanConfig).
Hint: If you run the unit test TestWrite, it creates a lot of MDF files in the %TEMP%/test/mdf/write directory. These files will be removed the next test run.

Tested the above cmake setting. The same build issue still exist. Reopen the issue.

I doesn't run without the VCPKG. You can build a shared library on the mdflib by changing the add_library(..STATIC to SHARED and add a set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON). This should eliminate the need of using the .NET interface export.cpp wrapper.
I do get some minor annoyance that I need to copy the dll to the excutable directory after build (as mdfviewer.exe).

I tried and I also gets a lot of link errors that indicate that you mixing C++ and C# targets. Setting the MDF_SHARED_LIB=OFF will fix the "mixing" linkage problem but I suspect you will get some other problem with third-party library. If you running on Windows, the VCPKG solves most of the problems with third-party libraries. If you have VCPKG installed, you should add the -DCMAKE_TOOLCHAIN_FILE=<vcpkg-root>/scripts/buildsystems/vcpkg.cmake. Expect some extra build time at first build.

Hi @ihedvall,
I am building on Ubuntu WSL 22.04. I am writing a pure C++ application. I use a custom toolchain file for CMake which includes the paths for wxwidgets. I can get the library to compile and build just fine. I can run the viewer and the library test application included in the project.
I am using an mf4 file generated from the css canedge for reference to setup my own program. I am logging canbus. I can send you screenshots tomorrow since I am out of office today. The problem occours when I try to use the CreateChannelComposition function. I am unsure of which option is causing just that one function to not being included in compilation. To verify that the problem is within the library itself I tried compiling with tests on and encountered the error again. If you point me in the right direction I will attempt to fix it myself and submit a PR.

I am impressed that you get the test_mdf and mdfviewer running. The boost and wxWidgets are little bit tricky. Both the mdfviewer and the test_mdf executable uses the static mdf.lib instead of the mdflibrary.dll. I recommend that you use the static library in your C++ projects.
The mdflibrary.dll is designed as a .NET (C++/CLI) interface and is intended to be used in C# projects. You can get  this DLL running on Linux but it require an extra wrapper that is not well tested. In your case it was just an extra character. I think Simplxss have fixed the error, so just a git pull would probably solve the above problem.
If you run the test_mdf.exe, there should be a number of MDF files in the /temp/test/mdf/write directory. Well until next OS restart at least.
If you want to read CAN bus traffic, you might be interested in the DBC library instead. The MDF file normally only store the the 8 byte data, Can Id and time. You need a DBC file to convert the data bytes to signals. With the DBC Viewer you may read in a DBC file and the import MDF file afterwards thus converting the bytes to signals and samples. The ihedvall/dbclib is relative new so it might have some unknown features.

If you are writing a CAN bus logger i.e. creating MDF files , you might need some meeting as the writing of MDF files are little bit tricky. You may check the Google Test TestWrite, Mdf4MlsdCanConfig for guidance. Please let me know if you need some meeting.

I am impressed that you get the test_mdf and mdfviewer running. The boost and wxWidgets are little bit tricky. Both the mdfviewer and the test_mdf executable uses the static mdf.lib instead of the mdflibrary.dll. I recommend that you use the static library in your C++ projects.
The mdflibrary.dll is designed as a .NET (C++/CLI) interface and is intended to be used in C# projects. You can get this DLL running on Linux but it require an extra wrapper that is not well tested. In your case it was just an extra character. I think Simplxss have fixed the error, so just a git pull would probably solve the above problem.
If you run the test_mdf.exe, there should be a number of MDF files in the /temp/test/mdf/write directory. Well until next OS restart at least.
If you want to read CAN bus traffic, you might be interested in the DBC library instead. The MDF file normally only store the the 8 byte data, Can Id and time. You need a DBC file to convert the data bytes to signals. With the DBC Viewer you may read in a DBC file and the import MDF file afterwards thus converting the bytes to signals and samples. The ihedvall/dbclib is relative new so it might have some unknown features.

@ihedvall The typo correction from @Simplxss fixes the issue. I was able to use the .so in a C++ application.
I have shown below what the intended file formatting should look like.

The image below is of my mdf file using CreateChannelComposition

The intended use for mdflib for my use case is to bus log files that ASASMMDF can view. Thank you for your offer for meeting.  Is there a work email I can reach you at.?

I have sent my e-mail address through Linkedin.The MdfWriter::CreateBusLogConfiguration() function creates a standard CAN bus configuration for you but you need to add the source information (SI). There are some different ways you can store bus traffic. It might be worth looking at the mdfwriter.h and the canmessage.h.
Running bus loggers might require some MS Teams meeting.


Render photos

Value proposition
As a user,
I want to have a list of my photos
so that I have a quick overview and don't have to remember all of my clothes.
Description

Acceptance criteria

[ ] Items (photos) are rendered
[ ] The button is always at the bottom
[ ] Photos scroll beneath the button
[ ] Items wrap

Complexity
Small
Tasks

[ ] Create branch
[ ] Add base styles
[ ] Setup Storybook for button
[ ] Implement button component
[ ] Write specs for button
[ ] Add button to app
[ ] Render photos in an unordered list

Hey Sarina,
nicely done.
My suggestions:

I would suggest to phrase the value proposition a bit more specific, for example:

As a user,
I want to have a list with photos of my clothes
so that I have a quick overview and don't have to remember all the clothes in my closet.

in your scribble you can indicate photos with a crossed out box :) As well, as I understand correctly the add-button will not have any functionality yet, therefore it should not be included in this story (neither in the scribble nor the acceptance criteria and tasks etc.)


Support for Google Authenticator

Twilo notify in below link that, we can support google authenticator through Authy itself
https://www.twilio.com/learn/account-security/authy-now-supports-google-authenticator
In Authy-ruby API also they have documented API which handle google authenticator.
https://github.com/twilio/authy-ruby
response = Authy::API.request_qr_code(id: user.authy_id, qr_size: 500, label: "My Example App")
    if response.ok?
      # qr code was generated
    else
      response.errors
    end

is this feature supported in Authy-Devise ?
Can you please guide me how to integrate Google Authenticator with Authy-Devise.

Hey @trustarun. Thanks for the question. This is not currently supported in Authy Devise, but we should support it. My focus is not currently on this gem for now, but if you wanted to try building it I'd be happy to support. Otherwise, I could get to this next month.

Hey there @philnash, have you thought any more about this request? I'd also appreciate this feature. Most of our internal users have 1Password already, which can store 2FA codes. I'd prefer to not make them install a second app.
I could potentially take a look at implementing this but I've never worked on a gem so it would probably take me a while.

Oops, you're right, I haven't returned to this and I definitely meant to. There are some other issues/PRs on this project that I need to address too.
I am busy with an event all the following week, but may be able to get to this the week after.

@philnash Thank you! Let me know if I can help test it or anything.

Hey, sorry for the delay, support for this is now released in version 2.1.0. See the documentation on it here.


Allow nmdc_data.py consumer to specify which variant of schema they want to access

In this branch, I updated nmdc_data.py so that the fuctions that provide access to the JSON Schema version of the schema, allow the caller to specify which variant of the schema they want (they currently only support the default variant and the nmdc_materialized_patterns variant).
That will allow callers (such as the Runtime and the migration notebooks) to access the nmdc_materialized_patterns.schema.json JSON Schema.
I also added doctests to all modified functions to demonstrate that they return values of the types I expect. The doctests can be run via $ poetry run python -m doctest nmdc_schema/nmdc_data.py (or, now, $ make test-python).

This looks pretty good to me. @pkalita-lbl and @eecavanna are you confident that nmdc_schema/nmdc_materialized_patterns.schema.json will be consistently bundled with the PyPI artifact?
I'm trying to convince myself by looking through .github/workflows myself now.
My interpretation is that build-n-publish from .github/workflows/pypi-publish.yaml aka "Publish Python Package" is run every time a release is created. The "Generate project files" step runs make all on GitHub's runner. That makes
project/nmdc_materialized_patterns.schema.json: project/nmdc_materialized_patterns.yaml
	$(RUN) gen-json-schema \
		--closed \
		--top-class Database $< > $@

cp project/nmdc_materialized_patterns.schema.json nmdc_schema/nmdc_materialized_patterns.schema.json

Thanks to this part of pyproject.toml, all files in nmdc_schema/ are included in the PyPI package.
packages = [
    { include = "nmdc_schema" }, # this is the package Poetry would have included by default
    { include = "migrators", from = "nmdc_schema" },
    { include = "adapters", from = "nmdc_schema/migrators" },
]

And then code like this, in nmdc_schema/nmdc_data.py, makes the contents of the bundled files avaialbe in any code that imports nmdc-schema
return io.BytesIO(pkgutil.get_data(__name__, filename))

@pkalita-lbl @sujaypatil96 couldn't we have written nmdc_schema/nmdc_data.py so that it wouldn't have needed all the derived files bundled? I could have just read in src/nmdc.yaml and done all of the derivations in real time?

@pkalita-lbl and @eecavanna are you confident that nmdc_schema/nmdc_materialized_patterns.schema.json will be consistently bundled with the PyPI artifact?

Yeah. Your analysis sounds right to me.

@pkalita-lbl @sujaypatil96 couldn't we have written nmdc_schema/nmdc_data.py so that it wouldn't have needed all the derived files bundled? I could have just read in src/nmdc.yaml and done all of the derivations in real time?

It's possible, but what would that accomplish? From a client's perspective the output of those functions will never change at runtime, so why not just read a pre-made file? That should be much more efficient than generating on the fly.

are you confident that nmdc_schema/nmdc_materialized_patterns.schema.json will be consistently bundled with the PyPI artifact?

As long the process that was used to generate the current package doesn't change, I expect the file to keep getting bundled into the package.


Value from --with-option=value is not recorded in INSTALL_RECEIPT.json

[x] ran brew update and can still reproduce the problem?
[x] ran brew doctor, fixed all issues and can still reproduce the problem?
[x] ran brew config and brew doctor and included their output with your issue?

What you were trying to do (and why)
I tried to write a formula for tmux with custom FPS value read from --with-fps=30.
This is the trick I learned from #1803.
    option "with-fps=", "FPS (default 10)"
    ...
    if ! (ARGV.value("with-fps").nil? || ARGV.value("with-fps").empty?)
      fps=ARGV.value("with-fps").to_i
      redraw_interval=(1000000/fps).round
      inreplace "tty.c" do |s|
        s.gsub! /^#define TTY_BLOCK_INTERVAL .*$/, "#define TTY_BLOCK_INTERVAL (#{redraw_interval} /* #{fps} fps */)"
      end
    end

What happened (include command output)
The option I passed in the command line --with-fps=30 is not fully recorded in INSTALL_RECEIPT.json. It only records --with-fps=.
This could be also confirmed from the verbose installation log as well.
❯ brew install tmux-options --with-fps=30 -v
==> Installing tmux-options from laggardkernel/tap
/usr/bin/sandbox-exec -f /private/tmp/homebrew20200229-48037-1msxno7.sb nice ruby -W0 -I $LOAD_PATH -- /usr/local/Homebrew/Library/Homebrew/build.rb /usr/local/Homebrew/Library/Taps/laggardkernel/homebrew-tap/Formula/tmux-options.rb --verbose --with-fps=30 --with-fps=
==> Downloading https://github.com/tmux/tmux/releases/download/3.0a/tmux-3.0a.tar.gz

What you expected to happen
I assume the value part should be recorded as well, which is helpful to dump installation detail by brew bundle dump.
Output of brew config and brew doctor commands
❯ brew config
HOMEBREW_VERSION: 2.2.6-67-gf37cd18
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: f37cd18e1a3605e7409c6dbf16185110e27263ad
Last commit: 34 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core.git
Core tap HEAD: 42c65ea0af06aff75feba40bea8b357129669bb0
Core tap last commit: 18 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_DEV_CMD_RUN: 1
HOMEBREW_FORCE_BREWED_CURL: 1
HOMEBREW_GITHUB_API_TOKEN: set
HOMEBREW_NO_ANALYTICS: 1
HOMEBREW_NO_AUTO_UPDATE: 1
HOMEBREW_NO_INSTALL_CLEANUP: 1
HOMEBREW_VISUAL: nvim
CPU: octa-core 64-bit skylake
Homebrew Ruby: 2.6.3 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3/bin/ruby
Clang: 10.0 build 1000
Git: 2.25.0 => /usr/local/bin/git
Curl: 7.68.0 => /usr/local/opt/curl/bin/curl
Java: 12.0.2
macOS: 10.13.6-x86_64
CLT: <IP_ADDRESS>.1.1539992718
Xcode: N/A
XQuartz: 2.7.11 => /opt/X11

ARGV.value should not be considered a public API so this is a WONTFIX, sorry.


add profile

Fix #1123

@qiujiayu please take a look at CI, thx

Looks good to me, we can merge it soon.

@qiujiayu Thank you very much, it has been merged.


Error: Building package bzip2:x86-windows failed with: BUILD_FAILED

I'm trying to install libmysql:x64-windows, but every time I do it in the standard command prompt, I'm stuck in a loop with the following errors:

error: mingw32: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown

error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely

error: mingw64: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown

error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely

error: msys: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown

error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely

error: database 'mingw32' is not valid (invalid or corrupted database (PGP signature))

error: database 'mingw64' is not valid (invalid or corrupted database (PGP signature))

error: database 'msys' is not valid (invalid or corrupted database (PGP signature))

When I ran the same installation command in Git CMD, I got an error saying:
Error: Building package bzip2:x86-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: bzip2:x86-windows
Vcpkg version: 2020.06.15-nohash
If anyone can help me with this, I'd appreciate it.

This issue should be fixed,  please update vcpkg and rebuild bzip2.
Thanks.

I still have the problem:
`
Computing installation plan...
The following packages will be built and installed:
bzip2[core]:x86-windows
Detecting compiler hash for triplet x86-windows...
Starting package 1/1: bzip2:x86-windows
Building package bzip2[core]:x86-windows...
Could not locate cached archive: C:\Users\faponte\AppData\Local\vcpkg\archives\54\547100163dd329fca01eca4a2cf079d390bed1a3.zip
-- Using cached C:/vcpkg/downloads/bzip2-1.0.6.tar.gz
-- Cleaning sources at C:/vcpkg/buildtrees/bzip2/src/1.0.6-8838f21f8e.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source C:/vcpkg/downloads/bzip2-1.0.6.tar.gz
-- Applying patch fix-import-export-macros.patch
-- Applying patch fix-windows-include.patch
-- Using source at C:/vcpkg/buildtrees/bzip2/src/1.0.6-8838f21f8e.clean
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Fixing pkgconfig
-- Acquiring MSYS Packages...
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:72 (message):
Command failed: C:/vcpkg/downloads/tools/msys2/msys64/usr/bin/bash.exe --noprofile --norc -c "pacman -S --noconfirm --needed pkg-config"
Working Directory: C:/vcpkg/downloads/tools/msys2
Error code: 1
See logs for more information:
C:\vcpkg\buildtrees\bzip2\msys-pacman-x86-windows-err.log
Call Stack (most recent call first):
scripts/cmake/vcpkg_acquire_msys.cmake:153 (vcpkg_execute_required_process)
scripts/cmake/vcpkg_fixup_pkgconfig.cmake:281 (vcpkg_acquire_msys)
ports/bzip2/portfile.cmake:51 (vcpkg_fixup_pkgconfig)
scripts/ports.cmake:79 (include)
Error: Building package bzip2:x86-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: bzip2:x86-windows
Vcpkg version: 2020.06.15-nohash
Additionally, attach any relevant sections from the log files above.
The error log is this.
error: mingw32: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown
error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely
error: mingw64: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown
error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely
error: msys: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" is unknown
error: key "4A6129F4E4B84AE46ED7F635628F528CF3053E04" could not be looked up remotely
error: database 'mingw32' is not valid (invalid or corrupted database (PGP signature))
error: database 'mingw64' is not valid (invalid or corrupted database (PGP signature))
error: database 'msys' is not valid (invalid or corrupted database (PGP signature))
`

Please delete VCPKG_PATH/downloads/tools/msys2 and try again.

> Please delete VCPKG_PATH/downloads/tools/msys2 and try again.
It works. Thank you.
Does this mean that vcpkg uses an "embedded" msys2 (mingw) installation for building its libraries? Even if the system has Visual Studio installed?

Yes,  it's build process requires msys2.


fix: use image of container in MigratorSpec

Previous to this commit, the operator would inject an waiter container
that would poll for the image of the first container in the target Pod
rather than the image of the specified container. This is surprising
behaviour and also potentially problematic when combined with Istio's
holdApplicationUntilProxyStart functionality, which injects the Istio
sidecar at the beginning of the list of containers.
This commit ensures the operator uses the container specified in the
MigratorSpec, if supplied and present. If the target container name is
specified and not present, it falls back to using the image from the
first container in the Pod (the previous behaviour). If the target
container name is not present, it also uses the previous behaviour.

Good catch, thanks!


Update installation script

warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.

Thanks for doing this 😄 The less characters to getting started the better 😉


HDFS-16419. Make HDFS data transfer tools cross platform

Description of PR
The source files for hdfs_copyToLocal and hdfs_moveToLocal uses getopt for parsing the command line arguments. getopt is available only on Linux and thus, isn't cross platform. We need to replace getopt with boost::program_options to make these tools cross platform.
How was this patch tested?
In progress.
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
37m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 52s

trunk passed

+1 :green_heart:
compile
3m 16s

trunk passed

+1 :green_heart:
mvnsite
0m 31s

trunk passed

+1 :green_heart:
shadedclient
58m 38s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 17s

the patch passed

+1 :green_heart:
compile
3m  8s

the patch passed

+1 :green_heart:
cc
3m  8s

the patch passed

+1 :green_heart:
golang
3m  8s

the patch passed

+1 :green_heart:
javac
3m  8s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
0m 17s

the patch passed

+1 :green_heart:
shadedclient
18m 42s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
32m 22s

hadoop-hdfs-native-client in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

153m 53s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3873

Optional Tests
dupname asflicense compile cc mvnsite javac unit codespell golang

uname
Linux 355837b91bca 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk /<PHONE_NUMBER>001e192b47bcb9146cc216978edb2a

Default Java
Red Hat, Inc.-1.8.0_312-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/testReport/

Max. process+thread count
543 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/console

versions
git=2.9.5 maven=3.6.3

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
24m  0s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
23m 29s

trunk passed

+1 :green_heart:
compile
3m 24s

trunk passed

+1 :green_heart:
mvnsite
0m 45s

trunk passed

+1 :green_heart:
shadedclient
49m 50s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 23s

the patch passed

+1 :green_heart:
compile
3m 19s

the patch passed

+1 :green_heart:
cc
3m 19s

the patch passed

+1 :green_heart:
golang
3m 19s

the patch passed

+1 :green_heart:
javac
3m 19s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
0m 26s

the patch passed

+1 :green_heart:
shadedclient
20m 51s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
32m 36s

hadoop-hdfs-native-client in the patch passed.

+1 :green_heart:
asflicense
0m 47s

The patch does not generate ASF License warnings.

134m 41s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3873

Optional Tests
dupname asflicense compile cc mvnsite javac unit codespell golang

uname
Linux 0f58924ebff6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk /<PHONE_NUMBER>001e192b47bcb9146cc216978edb2a

Default Java
Red Hat, Inc.-1.8.0_312-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/testReport/

Max. process+thread count
699 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/console

versions
git=2.27.0 maven=3.6.3

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
12m  7s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
28m 45s

trunk passed

+1 :green_heart:
compile
2m 55s

trunk passed

+1 :green_heart:
mvnsite
0m 21s

trunk passed

+1 :green_heart:
shadedclient
59m  3s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 17s

the patch passed

+1 :green_heart:
compile
3m  4s

the patch passed

+1 :green_heart:
cc
3m  4s

the patch passed

+1 :green_heart:
golang
3m  4s

the patch passed

+1 :green_heart:
javac
3m  4s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
0m 22s

the patch passed

+1 :green_heart:
shadedclient
27m 30s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
32m 23s

hadoop-hdfs-native-client in the patch passed.

+1 :green_heart:
asflicense
0m 33s

The patch does not generate ASF License warnings.

137m 38s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3873

Optional Tests
dupname asflicense compile cc mvnsite javac unit codespell golang

uname
Linux 3290e4277db8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk /<PHONE_NUMBER>001e192b47bcb9146cc216978edb2a

Default Java
Debian-11.0.13+8-post-Debian-1deb10u1

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/testReport/

Max. process+thread count
720 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/console

versions
git=2.20.1 maven=3.6.0

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 44s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
21m 24s

trunk passed

+1 :green_heart:
compile
3m 10s

trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
3m 12s

trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
mvnsite
0m 27s

trunk passed

+1 :green_heart:
shadedclient
47m 58s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 17s

the patch passed

+1 :green_heart:
compile
3m  7s

the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
cc
3m  7s

the patch passed

+1 :green_heart:
golang
3m  7s

the patch passed

+1 :green_heart:
javac
3m  7s

the patch passed

+1 :green_heart:
compile
3m  7s

the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

+1 :green_heart:
cc
3m  7s

the patch passed

+1 :green_heart:
golang
3m  7s

the patch passed

+1 :green_heart:
javac
3m  7s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
0m 19s

the patch passed

+1 :green_heart:
shadedclient
19m 19s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
32m  8s

hadoop-hdfs-native-client in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

109m 45s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3873

Optional Tests
dupname asflicense compile cc mvnsite javac unit codespell golang

uname
Linux 61dc9f89e9ee 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk /<PHONE_NUMBER>001e192b47bcb9146cc216978edb2a

Default Java
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/testReport/

Max. process+thread count
549 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3873/1/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.


Bump chrono and iana-time-zone; fix use of deprecated function

This PR bumps chrono from 0.4.26 to 0.4.38 and iana-time-zone from 0.1.56 to 0.1.60. The latter is updated to avoid  having two versions of some dependencies it has in common with chrono. The PR also replaces the use of the deprecated NaiveDateTime::from_timestamp_opt() with DateTime::from_timestamp().

Changes since last push:

updated iana-time-zone to 0.1.61


Use GitHub App

Resolves #130.

Need to change the format of the private key used.

Waiting for https://github.com/jenkins-infra/helm-charts/pull/347


Deprecate low-level api from aiohttp namespace

Deprecate
aiohttp.HttpMessage
aiohttp.Response
aiohttp.Request
aiohttp.RawRequestMessage
aiohttp.RawResponseMessage
aiohttp.HttpRequestParser
aiohttp.HttpResponseParser
aiohttp.HttpPayloadParser
aiohttp.HttpProcessingError
aiohttp.BadHttpMessage
aiohttp.HttpBadRequest
aiohttp.BadStatusLine
aiohttp.LineTooLong
aiohttp.InvalidHeader
aiohttp.ClientDisconnectedError

I agree with this.

will be removed in 2.1

ok

@fafhrd91 what are your exact steps in process of deprecation of library's public names?
How to inform users about deprecation?
Also I suspect that dropping deprecated names in aiohttp 2.1 is too fast for smooth transition period.

technically all those names are private, I can not imaging anybody uses those names.
I think specifically for this set of names dropping early should be safe.

maybe except aiohttp.ClientDisconnectedError, but we can keep this name longer

I do not have strong opinion on this issue. if you think we should keep this names for longer, let's keep them longer.

tbh if it was me I would have crammed aiohttp into far less modules. But I am not sure if everyone would like it though so yeah. (Not to mention it is not always a good idea.)

here names that aiohttp2.0 is missing compared to 1.3
# we should not export this names in any case
CIMultiDict
CIMultiDictProxy
MultiDict
MultiDictProxy
istr
upstr
multidict

# this are gone after refactoring, and in any case they are too low-level
ParserBuffer
StreamParser
StreamProtocol
parsers

# deprecated
ProxyConnector
HttpPrefixParser

# this are removed, but we can make backward compatibility names
DisconnectedError
ClientHttpProcessingError
HttpMethodNotAllowed
MsgType

# this long time deprecated
delete
get
head
options
patch
post
put
ws_connect

# modules
errors
protocol
web_reqrep
_ws_impl

we still export this names for backward compatibility
HttpMessage
Request
Response

# http parser, nobody should've use those, to aiohttp specific
# I am not sure if there is any value to keep this names, as api completely changed after
# parser redesign
HttpRequestParser
HttpResponseParser
HttpPayloadParser

# low-level
RawRequestMessage
RawResponseMessage

# this are http parsing specific exceptions and should not escape aiohttp low-level usage
HttpProcessingError
BadHttpMessage,
HttpBadRequest
BadStatusLine
LineTooLong
InvalidHeader

# just backward compatibility
ClientDisconnectedError
ClientTimeoutError
FingerprintMismatch
HttpProxyError
ProxyConnectionError

I am not sure about aiohttp.request(...)

Hi, I'm new to aiohttp and have some code written using aiohttp low level server that needs to be ported to latest standards. Are there any docs that deal specifically with the porting? What things do we need to change?

Are you talking about https://docs.aiohttp.org/en/stable/web_lowlevel.html ?
If yes -- you are not affected.


⚠ Bofrost.be has degraded performance

In e324aa7, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 2314 ms

Resolved: Bofrost.be performance has improved in dcba337.


Finalized State Upload Script

Using the beacon node API (finalized event and debug state endpoint),
Create a script that fetches every new finalized state, signs and uploads it to S3 or some public space.
We can then start using these states to launch our beacon node and bypass having to sync from genesis.

By script, I meant it could be cli command :)

By script, I meant it could be cli command :)

To make it more secure, we can create BeaconState envelope (SignedBeaconState {message: BeaconState, signature: BLSSignature}) so that we can have some trusted Chainsafe bls key signing those and we can hardcode that key it in lodestar client so we can verify downloaded state is really produced by Chainsafe.

To make it more secure, we can create BeaconState envelope (SignedBeaconState {message: BeaconState, signature: BLSSignature}) so that we can have some trusted Chainsafe bls key signing those and we can hardcode that key it in lodestar client so we can verify downloaded state is really produced by Chainsafe.

Update on progress:
https://github.com/ChainSafe/beacon-state-upload

Update on progress:
https://github.com/ChainSafe/beacon-state-upload

oh, how come it's not part of cli?

Update on progress:
https://github.com/ChainSafe/beacon-state-upload

oh, how come it's not part of cli?

I think the goal it to have a script client-agnostic that all clients can run on their CI

Don't think that's suppose to be ran in ci 😄

Don't think that's suppose to be ran in ci

Right I meant some (cloud) instance on a cron job

linked with #2177

See https://github.com/ChainSafe/beacon-state-upload

closing due to favored strategy in #2571


Merges latest changes from release/9.5.0 into future/10

Summary
Merges latest changes from release/9.5.0 into future/10

Meh, will have to submit another PR with a manual merge...


validator.js throws an error when user passes in an array value

Hi, I'm having this problem where validator.js throws This library (validator.js) validates strings only when user passes in an array value in the query.
Query: mydomain.com/some/path?new=true&new=true
Router code:
router.get('/some/path', check('new').isBoolean(),
function(req, res, next) {
  // Some code
})

The data type for new is Boolean in our API docs. If user tries to pass in an array like this, we just want to return a 400 error, not crashing the whole server.
I know the error is thrown by the validator.js lib, but is there any safe way to prevent it from express-validator, or maybe catch the error and handle it ourselves with the new express-validator API?
Thank you.

Same error with sanitize etc. when used with checkboxes having the same name since multiple selections will result in an array:
i.e. req.sanitize('checkbox') works fine with single selection but fails with multiple, as req.body.checkbox returns a string for single selection and an array for multiple.

Do you guys have any suggestion on what to do in these cases?
I guess the correct behaviour is to validate/sanitise all fields the same way. See related issue: #331
Anyway, whatever direction we head to will imply a breaking change.

@gustavohenke @codynguyen @nkalvi  I think wildcards() takes care of an array value.
I have successfully achieved the desired functionality just by using the wildcards().
https://github.com/mdn/express-locallibrary-tutorial/pull/31

@VaibhavAjayGupta Thank you. I'll check it out later.


No such file or directory -- .../gems/rspec-core-2.7.0/bin/rspec (LoadError)

Hi rspec group!
I saw this error happen when I run autotest -n (and ruby version 1.9.2-p180) on terminal and after that it's showing me all my specs and this error message.
I went to the folder rspec-core-2.7.0 and I can't see any folder called bin. So, I suppose that exist a problem in the .gemspec inside the project.
When I saw the .gemspec file, I didn't see any reference to bin folder. But here I can see the new bindir, however it's looking for a folder called bin.
I need execute some command, run something? Before this gem update, I was using 2.6.4 version and all of my specs were working fine.
I'm using this gems cause this help to identify this problem.
Thanks for your help.

@faianoalmeida - thanks for the report. I see what the problem is - I moved the bindir to exe so we could bundle install --binstubs into bin for local development. When I do that, it puts the rspec command in bin, so I would never have seen this bug in action :(
Fix coming soon.

@dchelimsky - thank you for the quick reply. And about your self-punishment (lol), this happens, right?
I have a question in your reply... Where can I execute this command (bundle install --bunstubs)? Whether I understood, I could this commend inside the bin folder in .../gems/rspec-core-2.7.0/, that's correct?
Cause this answer is "yes", I need a Gemfile file to execute this command. How can you see, I don't understand where I could execute this command.
Thanks one more time.
Regards.

@fabianoalmeida - bunstubs are in a completely different industry :) It's binstubs. When you run bundle install --binstubs in the project root directory, it creates a bin folder and puts stub executables for any gems that ship with executables. This helps to avoid version conflicts (i.e. if you have both rake 0.8.7 and 0.9.2 installed, but this project needs 0.8.7).
If you're looking for a workaround until this is released, the best bet is to rewrite the SPEC_PROGRAM variable so it looks like https://github.com/rspec/rspec-core/blob/96fa1191fcbd707fc27522fe74f1103626d103b7/lib/autotest/rspec2.rb#L8.
HTH,
David

@dchelimsky - thank you so much!
I never create a gem, so you help me and I learned a bit more about it. I'll describe what I did to solve this "bug", right?
I ran bundle install --binstubs and after that was created a bin folder on the root application folder.
After that, I tryed to run autotest -n and was necessary to install all missed gems. So, bundle install. That's created a folder called rspec-code.
The last step => I copied the bin folder to rspec-code/ruby/1.9.1/gems/rspec-core-2.7.0/.
Done! I ran autotest -n and all worked fine.
I hope that could help anyone with the same problem.


Implement service supervision?

The main argument of systemd fanboys against OpenRC is that it doesn’t support supervision, i.e. crashed services are not automatically restarted. I agree that it’s indeed a nice feature, that comes handy in some situations, although it cannot (and should not) replace a monitoring system.
I’m convinced that it can’t be hard to implement a simple supervising in OpenRC. What do you think about it?

I was quite busy in late February scripting an agnostic supervision backend framework which is quite feature complete with CGROUP support and simple service (need) dependencies. No complicated dependecies tree scheduled because of the supervision service scan model.
What to say... I guess the approach described on the bug would ultimately fails because... First, automatic service generation would never workd because start-stop-daemon (ssd) forks long lived process (daemon) by design while particular supervision backend runs a supervised process in the foreground. There's absolutely no way to make a simple replacement of ssd by introducing a new backend without adding at least an extra option to hold the forground argument for every single init script to function properly.
Who's going to do that extra compatibility work? I guess this is manageable to some extend.
Second, service scan model makes it hard to start new services in a racy free condition but to... start new service as down state and then start them. This could be manageable as well if a particular backend behave properly--runit/s6 does this.
So the blocker is the first point because the second could resolved with that trick or possibly more or less dirty hacks. However, the dynamic {,log/}{run,finish} is not that good an idea... it will be certainly better to have a proper '/service/' directory and work with making writing a supervision backend to replace ssd not that simple to do because of logging choice and so forth. Well, I tried this first and completly abandoned the idea.
So I am nicely leaving the hot potato to others. Note, 1 has LSB compatible commands {start,stop,reload,status} plus a particularly command to add services and can be used as a starting point for more adventurous ones.
Good luck!
PS: Maybe closing this issue would be a good idea!
PS2:
Almost forget to mention that the framework has [Open]RC support to use functions.sh and service config files. This is not that usefull because there aren't that many environment variables to share besides the command and command_args... but the latter one is problematic because of the first point (above.)
Second, Init-Stage-{1,2,3} for Runit/S6 do use OpenRC to not have to write system boot/halt from scratch! I don't feel like wasting my time here when there's a good portable alternative to take care of it.

I'm going to close this issue since we now have multiple supervisors supported internally in OpenRC. See runit-guide.md, s6-guide.md and supervise-daemon-guide.md in the distribution. The idea of these is to hook into runit or s6 or use supervise-daemon  (OpenRC's internal supervisor) to supervise long-running services and leave the rest to OpenRC.


Allow using API from python

I would like to use versioneer like this:
pip install versioneer
mymodule/__init__.py or mymodule.py:
from versioneer import get_version
__version__ = get_version(__file__)
del get_version

Versioneer could now use the following assumptions:
def get_version(package_file, style='pep440'):
    package_file = Path(package_file)
    assert package_file.name == '__init__.py'
    package_name = package_file.parent.name
    project_dir = package_file.parent.parent

Versioneer would then:

Try from-vcs, e.g. project_dir / '.git'
Try from-file project_dir / '_version.py'
Try from-parentdir project_dir.name
Try from-metadata pkg_resources.get_distribution(package_name)

The last step is new and can be used for installed packages instead of from-file, eliminating the need for a build step.

Indeed point 4. above is needed to retrieve the version of pip-installed packages. So it would be great if it could be incorporated into versioneer.


[#7] Fetch posts from markdown files

https://github.com/nimblehq/til-web/issues/7
What happened 👀

Fetch posts from markdown files
Fetch the individual post by slug
The detail includes: slug, content of the file, and the frontmatter fields

Insight 📝
As I haven't set up the test yet, then I will add the test in another PR
Proof Of Work 📹
The sample posts should be shown on the home page:
https://deploy-preview-8--nimble-til.netlify.app/

❌ Deploy Preview for nimble-til failed.
Name	Link
🔨 Latest commit	56f1cc8
🔍 Latest deploy log	https://app.netlify.com/sites/nimble-til/deploys/628cbba34ff4a00009f3de84

🥲

Can you add Resolves before #7?


Improve Atom startup time

Don't activate until the user actually opens an HTML file.
This saves Atom about 40ms of startup time.
For more details, search for activationHooks on
http://flight-manual.atom.io/hacking-atom/sections/package-word-count/

Is there a way to manually install it with this patch?


dts: Add a bunch of properties to base.yaml

This is based on #18156

Caught up in the type checking stuff, but looks like nice cleanups (status is already handled in the code, and gives a more explicit error message, so that one might be redundant though).
One thing that irks me a bit is that more and more property names get hardcoded in the code. Was hoping we could move in the other direction.
If we need to suppress output for lots of properties, then maybe it should be an option in the binding files instead (like output: false, where a missing output is treated the same as output: true).
Or maybe it could be reorganized in some other way.

Maybe the type checking stuff will help now that I think about it, because it will tell you e.g. whether a property value has a phandle reference in it (gives compound type).

Caught up in the type checking stuff, but looks like nice cleanups (status is already handled in the code, and gives a more explicit error message, so that one might be redundant though).

It is, but felt that having it in the binding made sense even if we special case it in the code.

One thing that irks me a bit is that more and more property names get hardcoded in the code. Was hoping we could move in the other direction.
If we need to suppress output for lots of properties, then maybe it should be an option in the binding files instead (like output: false, where a missing output is treated the same as output: true).
Or maybe it could be reorganized in some other way.

Yeah, something to think about.

Approved after the interrupt-parent thing (could be removed, since there's already an early return for type: phandle).

removed interrupt-parent & interrupt-extended


cosmwasm_std: remove dependency on DefaultHasher in tests

DefaultHasher is part of std module and is not available on no_std
builds.  Remove dependency on that type to pave the path for no_std
support.
This is done by adding test_utils::check_hash_impl helper function
which tests whether type implements Hash trait.  That function uses
crc32fast::Hasher instead of DefaultHasher thus eliminating need
for std.
Furthermore, Binary and HexBinary tests checking AsRef implementation
call AsRef::as_ref directly and explicitly check the result rather
than unnecessarily hashing the data.
As an added bonus, this diff has negative delta. \o/

I wonder, could you use crc32fast::Hasher::new() as a cosmwasm-std dev dependency? We use that in cosmwasm-vm already. Then we don't need to implement a hasher on our own.

Sure, I could, but does that actually give us anything? It adds a dependency and we don’t really care whether the TestHasher is an actual sensible hasher or not.

I wonder, could you use crc32fast::Hasher::new() as a cosmwasm-std dev dependency? We use that in cosmwasm-vm already. Then we don't need to implement a hasher on our own.

Done.

My idea is to have the same solution in both places in the codebase. We have it in the dependencies anyways already.
But I have another question: why is no_std support in test code relevant? To run cosmwasm-std tests on a no_std CI?

But I have another question: why is no_std support in test code relevant? To run cosmwasm-std tests on a no_std CI?

It really isn’t.  It’s more a matter of some vague sense of ‘pureness’.  If a test can be made no_std, than that’ll save us a conditional compilation later on.


Update dependencies identified by dependabot

This PR contains dependency updates

PR required because dependabot does not have access to repository secrets.


Bump service catalog in Provisioner

Description
Changes proposed in this pull request:

Bump service catalog in Provisioner to version v0.3.0

Resolved in https://github.com/kyma-project/control-plane/pull/168


connection of leaderboard to frontend

self-explanatory issue :>

well it works. database is displayed on frontend. its a leaderboard so users shouldn't be able to change anything, so basically everything is done.


Refactor Dynamic survey

[] Use clear name for the variables and functions
[] Reorganize the function so that code flows better.
[] Write comments where to refactor the code.

Currently, working branch is. Started working on this on July-17/2020
https://github.com/StatisticalReinforcementLearningLab/SARAv2/tree/harvard/mash-issue-146


"A source file could not be found"

doc.deno.land is telling me a source file could not be found.

Yet the file exists<EMAIL_ADDRESS>Others accessed the site with no issues, except for me and one other person:
Messages:

A user named Yamboy was able to access the page though:

This issues only occurs with me when I try to access the version tagged v0.3.1. All the other versions work as they should on my side. I tried accessing the pages with different devices and different browser. The only luck I had was using a VPN to the USA (I'm in the Canadian region), in which only then did the page load:

Just tried, and I was able to access<EMAIL_ADDRESS>in the UK 🇬🇧 😃

I'm guessing it may be an issue with cloud flare caching an old file in the Canadian region (originally I pushed the version, but the imports were broken, so I quickly deleted it, made the fix, and uploaded it)

You correctly concluded on Discord that this is Cloudflare caching. We do have a button on the site that lets you clear the cache, but its only visible once the page loads (not on error screen ). Also we should stop caching failed requests... I'll look into it

Thanks!


Google Kubernetes Engine

I'd thought this would be easy, but damn Google makes this confusing :rage1:
Important considerations:

Users can pick different kinds of clusters (standard v/s autopilot)
autopilot clusters must pick a release channel, but standard clusters can pick a kubernetes minor version
standard clusters not enrolled in a release channel will still get automatic control plane upgrades. You can delay them or limit them by defining maintenance windows, but you can't stop them.
node upgrades are in your control only if you picked a minor version.
Even then, google can decide to upgrade your nodes in select cases (See last link)

Note that there are no EoL dates or active support dates for release channels since they will always be supported. Static releases do have a  "release date" (when the version was made available in static), but I had to remove it to keep it consistent.
Preview: https://deploy-preview-314--endoflife-date.netlify.app/gke
Pages consulted:

https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview
https://cloud.google.com/kubernetes-engine/versioning
https://cloud.google.com/kubernetes-engine/docs/release-notes
https://cloud.google.com/kubernetes-engine/docs/release-notes-nochannel
https://cloud.google.com/anthos/clusters/docs/security-bulletins
https://cloud.google.com/kubernetes-engine/docs/concepts/release-channels
https://cloud.google.com/kubernetes-engine/docs/release-schedule
https://cloud.google.com/kubernetes-engine/upgrades

Automation Notes
Since GKE releases are quite frequent, it is very hard to keep the latest field updated regularly. Hence, I tried to automate it.
GKE has a Get Server Config API which returns the data gke.json.zip
However, the API needs authentication, so I've created a service account with zero permissions. There's 2 new scripts:
_auto/gke.rb
Authenticates to Google APIs, calls the get_server_config API and stores a parsed output in _data/gke.json
_plugins/liquify.rb
Jekyll doesn't permit YAML frontmatter to contain liquid tags. So we keep the latest tag as a liquid string, and then parse it with a liquify filter in the layout file instead.
An option, perhaps down the line would be to move the gke.json file (and corresponding code) to a separate repo, which can perhaps be fetched at build-time instead.

Forgot to rebase against master (icon_slug changed to iconSlug there). Should be good now


Client side rendering of .ipynb has not been ported from gogs

Description
The client-side notebook preview functionality has not been backported from the original gogs repo.
https://try.gogs.io/niklas/ipynb-test/src/master/README.ipynb
https://try.gitea.io/memetb/test-ipynb/src/branch/master/Untitled.ipynb
Issue #3025 mentions using RENDER_COMMAND = jupyter nbconvert --to html --stdin --stdout however, there are multiple reasons why this can be problematic:

server side-rendering of jupyter notebooks requires the installation of anaconda. This defeats the purpose of having a lightweight executable that can be deployed on a raspberry pi.
executing jupyter on large notebooks is slow.
having to execute a python interpreter on user data is not safe and would require chrooting the whole thing if a sys-admin needs to be thorough.

Gogs uses notebookjs for displaying jupyter notebooks. I still need to research whether notebookjs allows the execution of arbitrary code (like JS code), and in the case it does, it's a no-go and we should instead consider making (or finding) a parser that only displays markup and does not run any user code.

Ok, I've uploaded a version of the notebook with javascript and style sheet injection. The quick answer is that, as configured, it does allow for execution of code. However, this doesn't constitute an audit of the code, perhaps there's a way to disable it.
https://try.gitea.io/memetb/test-ipynb/src/branch/master/Untitled.ipynb
https://try.gogs.io/testbbar/test-ipynb/src/master/Untitled.ipynb

Yeah, it seems like code generated from notebookjs should be run through an HTML cleaner/purifier/sanitizer first.

nbpreview is doing something to this effect. The thing works as a standalone html file, so there may be client side library parameters which allow for this.
Either that or it's buggy and not behaving the way it wants to.

Further info:
This is the feature being proposed on gogs.
nbviewer seems to do what we're wanting. (Other link leads to a broken preview).
To test it, just download the ipynb from the above repos and drag and drop.
Fyi, I've attached what the original file looks like (fwiw, none of the three previews so far have rendered the style sheet correctly)

It would seem jupyter themselves (c.f. page 34) advocate for the use of Google Caja. So far, from my searching, I haven't found any other library which does what Caja claims to do.
It would also appear caja has a standalone js sanitizer.
Let me know if you require further research.

is it planned yet ?

It would be relatively easy to add ipynb support through an external markup renderer (e.g. using jupyter nbconvert) when #3025 is resolved.
A vanilla version would call the command every time the notebook has to be rendered. Next (and very reasonable) step would be to cache the rendered file until the source file is changed (e.g. scoping the cache to the source file hash). I think the approach is valid for all custom markup rendering, not only jupyter.

"Rendering of jupyter notebook 'rich output' is still missing some capabilities"
https://gitlab.com/gitlab-org/gitlab-ce/issues/32784#note_97778959

https://gitlab.com/gitlab-org/notebooklab (VueJS)
https://github.com/kokes/nbviewer.js
...
https://components.nteract.io/ (ReactJS)

...

[ ] Link to https://nbviewer.jupyter.org for the current file

You may find https://github.com/go-gitea/gitea/pull/12261 and this draft tutorial helpful.

@HarvsG using nbconvert is not a satisfiable solution, due to all the reasons that were mentioned in the first post (security, performance, dependencies). A solution built around a JS library like nbviewer.js would be preferable.


Icons do not display on github pages

Updated links for icons to secure link, updated README file to remove comments about deployment error issues.


Support Translation Word Links only

Remove translation words as a resource and keep only the links which are capable of syncing with the user text position.

Removed translation words as a resource in Bible and OBS
Words are syncing with the user text position in Bible and OBS

@vipinpaul, This PR is working as expected, This PR can be merged.


fix chromap format

chromap's read_format is 0-index based. This pull request fixes the end positions of the barcode and read strings in the seqspec index subcommand.

@sidwekhande can you make this PR against devel? I had accidentally merged into main and just reverted it.


Conditional layout

I'm trying to achieve conditional layout, is it possible ATM?
{{#if isLightLayout}}
    {{#extend "lightLayout"}}
{{else}}
    {{#extend "darkLayout"}}
{{/if}}

It's been a while since I've worked on this project. I don't know off the top of my head. Give it a try!

@shannonmoeller Thanks for the quick response!
I think I'll just pass the layout name

Even better!


Fix warnings in BudgetRestController

Description

Fixed the code warnings in BudgetRestController
Type of change

[ ] New Feature
[ ] Bug Fix
[X] Refactor
[ ] Documentation Update
[ ] Style
[ ] Performance Improvements
[ ] Test Update
[ ] Build/CI Update

Testing

[ ] Unit Tests Added/Updated
[ ] Integration Tests Added/Updated
[ ] Manual Testing Completed

Test Cases

Deployment Instructions (optional)

QA Instructions (optional)

Breaking Changes

[ ] This PR introduces breaking changes

Dependencies

Related Tickets & Documents

Screenshots/Recordings

Checklist

[ ] Code follows style guidelines
[ ] Self-review completed
[ ] Tests pass locally
[ ] Documentation updated
[ ] No new warnings generated

Additional Notes

:tada: Snyk checks have passed. No issues have been found so far.
:white_check_mark: security/snyk check is complete. No issues have been found. (View Details)


Make sure tooltips point to headers

Currently not pointing at the text

Semantic UI is such a life saver


Can't access Identity or Profile APIs

I am using the core and git clients and they work perfectly. However, I can't find a way to get the identity of the current signed in user.
If I try and use the profile client, I get messages like this:
profile_client.get_profile('me')

ValueError: API resource location f83735dc-483f-4238-a291-d45f6080a9af is not registered on https://XXX.visualstudio.com.
Or with the identity client:
identity_client.get_self()

ValueError: API resource location 4bb02b5b-c120-4be2-b68e-21f7c50a4b82 is not registered on https://XXX.vssps.visualstudio.com.
However, if I manually make the same request in the browser, it works fine (for example https://app.vssps.visualstudio.com/_apis/profile/profiles/me?api-version=5.0).
It seems to only be requests related to the current users's profile which don't resolve. The following request goes through successfully:
identity_client.list_groups()

The client is authenticated with OAuth2 with scope vso.code_full vso.code_status vso.graph vso.identity vso.authorization_grant.

@slishak I will investigate.  In the meantime, try this alternate approach to getting the information:
location_client = connection.clients_v5_0.get_location_client()
connection_data = location_client.get_connection_data()
authenticated_user = connection_data.authenticated_user
pprint.pprint(authenticated_user.dict)

Thanks very much, that snippet of code is exactly what I needed!

Partial fix for Profile issue: #212

The profile client talks to the user service, which at this point does not allow PATs.  You would need to use an AAD or MSA credentials to talk to directly to the user service.  Fix #212 is also required.


Update failure

@Chanfeechen commented on Thu Apr 11 2019
vscode-inno-updater-1555038148.log

Environment data

VS Code version: VSCodeUserSetup-x64-1.33.0
Extension version (available under the Extensions sidebar): XXX
OS and version: Windows 10 1803
Python version (& distribution if applicable, e.g. Anaconda): 3.7.1
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): anaconda
Relevant/affected Python packages and their versions: none

Expected behaviour
autoupdate
Actual behaviour
fail and uninstall automatically
Steps to reproduce:

close vs code after use

Logs
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)
Not applicable

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)
Not applicable

Note, this is not specific to the Python extension

@Chanfeechen Not sure if I understood the issue. Are you having issues auto updating an extension or VS Code?


[FEATURE] Azure Cloud Shell integration

Is your feature request related to a problem? Please describe.
Since Azure Cloud Shell ships the PNP m365 CLI by default, we'd love to include the PNP PowerShell module too. (I'm the engineering manager for Cloud Shell.) Ideally this would also include supporting the implicit authentication that Cloud Shell enables via exposing a Managed Identity endpoint which can provide oauth tokens based on the logged-in user's credentials.
Describe the solution you'd like
Submit a PR to github.com/Azure/CloudShell which adds this module to the list of preinstalled PowerShell modules. It would probably make most sense to do this when the module is no longer considered preview.
In case of questions feel free to ask!

The moment we go with GA I'll create a PR on the cloudshell repo.

Connect-PNPOnline -ManagedIdentity currently doesn't work in cloud shell in my testing currently, the context doesn't exist. That should be fixed, and then some cloud shell detection logic should be added (ENV:ACC_VERSION can be used) so that connect-pnponline just works by entering the uri.
connect-pnponline -url https://mytenant.sharepoint.com -PnPManagementShell works
There should also be guidance for users who haven't approved the app yet, because currently the linux logic uses bash xclip which doesn't exist in cloud shell so it throws an error on register-pnpmanagementshellaccess

The current implementation of the built-in service using in the cloud shell does not allow us to acquire an access token for a SharePoint tenant, resulting in us not being able to create a context. Unless that is fixed on their end, the actual added value of -ManagedIdentity is rather low. We can acquire an accesstoken for the graph, and all graph based cmdlets (Teams/Planner/Microsoft 365 groups) cmdlets should work.
Regarding the Register-PnPManagementShellAccess cmdlet: that has been fixed and that fix is available since 0.3.10-nightly.


[Improve][Connector-V2-kafka] Support for dynamic discover topic & partition in streaming mode

Search before asking

[X] I had searched in the feature and found no similar feature requirement.

Description
subtask of #2755
Usage Scenario
No response
Related issues
No response
Are you willing to submit a PR?

[x] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Will you submit a pr for this?

yes,I will try

Can multiple topics be filtered out for consumption through regular expressions or other ways?

@IeohMingChan Yes, it is supported by regular expressions

@Carl-Zhou-CN Hello, is there any difficulty? If you have any questions, you can give feedback to the community

@IeohMingChan @ashulin Sorry, there is some delay in time, I will work hard for this direction, encounter difficulties I will contact the community in time

@Carl-Zhou-CN Hello, is there any difficulty? If you have any questions, you can give feedback to the community

Thank you. I want to use regular expressions for log execution, I will give feedback to the community if I have any questions.

@IeohMingChan @ashulin Sorry, there is some delay in time, I will work hard for this direction, encounter difficulties I will contact the community in time

Thank you for your enthusiasm!


Address depriciation warnings in the notebooks

Proposed changes
Changes in the notebooks

Replace xarray.open_rasterio with rioxarray.open_rasterio
Replace pandas.append with pandas.concat
Replace xr.ufuncs.log10 with np.10g10
Replace xr.ufuncs.isfinite with np.isinf

Changes to the deafrica-tools

Update spatial.py xr_vectorize function
Update temporal.py by replacing xr.ufuncs.isnan with np.isnan
Update data_handling.py by replacing xr.ufuncs.logical_or with np.logical_or

Checklist (replace [ ] with [x] to check off)

[ ] Remove any unused Python packages from Load packages
[ ] Remove any unused/empty code cells
[ ] Remove any guidance cells (e.g. General advice)
[ ] Ensure that all code cells follow the PEP8 standard for code. The jupyterlab_code_formatter tool can be used to format code cells to a consistent style: select each code cell, then click Edit and then one of the Apply X Formatter options (YAPF or Black are recommended)
[ ] Include relevant tags in the first notebook cell and re-use tags if possible
[ ] Ensure appropriate colour schemes  have been used to maximise accessibility for vision impairment. Test your images or learn more with Coblis or TPGI
[ ] Clear all outputs, run notebook from start to finish, and save the notebook in the state where all cells have been sequentially evaluated

Closes issues (optional)

Closes Issue #000

Do we need to worry about this error?
/usr/local/lib/python3.8/dist-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.8.0-CAPI-1.13.1 ) is incompatible with the GEOS version PyGEOS was compiled with (3.10.1-CAPI-1.16.0). Conversions between both will be slow.
  warnings.warn(

Also with the changes in Tools, shall we up the version to 0.1.4?
Otherwise, look good to me to merge.

@fangfy Based on the explanation given here on why the warning occurs, I don't think we need to worry about it.


Snap remove lxd fails with error message

Required information

Distribution: Ubuntu
Distribution version: 18.10
The output of "lxc info" or if that fails:

Kernel version:
LXC version:
LXD version: 3.9
Storage backend in use: btrfs

Issue description
Cannot remove LXD using snap. Fails with remove /var/snap/lxd/common/lxd/storage-pools/default/images/35f6bff57c250208c6dc73445eefe8c1020853a1bc8571ebd8bfb1fe3a239504/metadata.yaml: read-only file system
Re-installing LXD shows no storage pools. Removing snap again fails with the same error message as above.
Steps to reproduce

Install lxd using snap install lxd
Create container
lxc launch $DISTRO:$DISTRO_VERSION -p default -p $USER $CONTAINER_NAME
Add storage device
lxc config device add $CONTAINER_NAME device0 disk source=$VOL_SRC_DIR path=$VOL_DST_DIR
Remove LXD from the system
sudo snap remove lxd

Completion of step 4 generates the error message.
Information to attach

[ ] Any relevant kernel output (dmesg)
[ ] Container log (lxc info NAME --show-log)
[ ] Container configuration (lxc config show NAME --expanded)
[ ] Main daemon log (at /var/log/lxd/lxd.log or /var/snap/lxd/common/lxd/logs/lxd.log)
[ ] Output of the client with --debug
[ ] Output of the daemon with --debug (alternatively output of lxc monitor while reproducing the issue)

None of the suggested steps for additional information are working at this time, either because it is now disabled, and many of the paths are not applicable to snap installs.

Similar to your other issue, this isn't an issue with LXD itself, therefore I'm closing this issue.
In this case, it looks like you're attempting to remove a package that still has a number of active kernel resources in play, in this case a bunch of btrfs subvolumes which cannot be deleted through normal filesystem operations. I suspect that if you had first deleted all containers and images from your LXD installation, the package removal would have worked.

Doesn't change the fact that the behavior exists, and the user has no idea that this action will be detrimental to their system. Had you chosed a packaging system that didn't have these obvious flaws, such as apt, the user could expect to have those resources purged when the package was removed.
Blaming this on upstream when you are well aware of the issue is unacceptable, as this bug has been an issue upstream for several years now.

Therefore it's not an upstream issue then...
On Wed, Feb 20, 2019 at 08:45 Stéphane Graber<EMAIL_ADDRESS>wrote:

APT and RPM actually both do the exact same thing in this case.
All package managers effectively attempt to do a recursive wipe of the
directory and all will fail when you've been using btrfs subvolumes, this
has absolutely nothing to do with the packaging format.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/lxc/lxd/issues/5513#issuecomment-465632401, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ApBWZtNv6nPTm3u_NFmE8NTW0rfdSRiUks5vPW2jgaJpZM4bEb8T
.

...seriously, I checked and tested this myself. There's no reason that this can't be done with a script in APT. The only issue here is a poor packaging implementation. Packaging using the debian package manager format would intrinsically resolve this isssue.


Consider "soft" dependencies?

Great extension!
One observation: the install_requires has a lot of heavy hitters... perhaps it's worth considering making these a bit more modular, either with [extras] or a -core package with subpackages that provide individual features via traitlets/entry_points, with the main package keeping all of them?

Thanks @bollwyvl for the suggestion, indeed I want to make voila related stuffs optional.


Forecast memory

When loading 15 minutely satellite data, the forecast runs out of memory.
We could up the memory, but would prefer to tackle why the satellite data is different

How much memory does the interpolation use? That might cause a bigger bump than just loading the outputs from the RSS. Might need to do the interpolation in Satip if that's the case

I'll add some logging, thats a good point though