Data retention and deletion: user data should be deleted on a rolling basis

I'd suggest adding that user data should be deleted on a rolling basis, not just after users terminate their account.
A loyal user could generate years of data which becomes an increasingly private and risky thing for a manufacturer to be holding on to.

Added to agenda for May 24th review meeting.

Thanks for your contribution. We discussed this at the May 24th meeting and wonder if you have any ideas about how to define the type of data that should be deleted on a rolling basis? For many services (email, cloud storage, social media, etc.) retention of user generated data is clearly something users want.

@paulfurley Thanks again for your contribution! Do you have any ideas about how to define the type of data that should be deleted on a rolling basis?

I was thinking along the lines of analytics and event data — "you watched x on channel y".
Perhaps linking this back to the purpose that the data was collected — for example, if a device was logging my viewing activity for the purpose of making better recommendations, as that data got older it would be harder to justify needing it.
It seems to me that most data has some sort of useful timespan, usually shorter than 'forever' (otherwise new users would always be doomed to always have a bad experience). Manufacturers should get in the habit of assessing what that useful timespan is and deleting old data, rather than holding onto it "just in case".
(Oh and of course much data protection legislation requires timely deletion anyway, and GDPR will make it more explicit)

Thanks, @paulfurley --any suggestion on specific language and where it belongs in the Standard?

See pull request #124

This would be a good fit under:

Privacy / Data retention / Data retention and deletion / The company retains data only as long as relevant and reasonably necessary to provide service to me. / The company on its own deletes outdated and unnecessary personal information.

See https://thedigitalstandard.org/standard/privacy/?version=v1.3.0#data_retention_and_deletion
There are two procedures in place that either could be expanded, or a new procedure could be added.


HADOOP-18516: [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication

Description of PR
Jira: https://issues.apache.org/jira/browse/HADOOP-18516
Changes ported from PR: https://github.com/apache/hadoop/pull/5148
This PR introduces a new configuration for Fixed SAS Tokens: "fs.azure.sas.fixed.token"
Using this new configuration, users can configure a fixed SAS Token in the account settings files itself. Ideally, this should be used with SAS Tokens that are scoped at a container or account level (Service or Account SAS), which can be considered to be a constant for one account or container, over multiple operations.
The other method of using a SAS Token remains valid as well, where a user provides a custom implementation of the SASTokenProvider interface, using which a SAS Token are obtained.
When an Account SAS Token is configured as the fixed SAS Token, and it is used, it is ensured that operations are within the scope of the SAS Token.
The code checks for whether the fixed token and the token provider class implementation are configured. In the case of both being set, preference is given to the custom SASTokenProvider implementation. It must be noted that if such an implementation provides a SAS Token which has a lower scope than Account SAS, some filesystem and service level operations might be out of scope and may not succeed.
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m 25s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 5 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m 11s

trunk passed

+1 :green_heart:
compile
0m 35s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 32s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 31s

trunk passed

+1 :green_heart:
mvnsite
0m 39s

trunk passed

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 33s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  3s

trunk passed

+1 :green_heart:
shadedclient
32m 41s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 28s

the patch passed

+1 :green_heart:
compile
0m 28s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 28s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 19s

the patch passed

+1 :green_heart:
mvnsite
0m 29s

the patch passed

-1 :x:
javadoc
0m 24s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)

-1 :x:
javadoc
0m 24s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08 with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)

+1 :green_heart:
spotbugs
1m  2s

the patch passed

+1 :green_heart:
shadedclient
32m 13s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
1m 57s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

135m 23s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 2530f81620ee 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / db03c5fd4049a916c180f2cb33467c1d886ec27c

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/1/testReport/

Max. process+thread count
739 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m 43s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 5 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
43m 42s

trunk passed

+1 :green_heart:
compile
0m 40s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 36s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 32s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 39s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 35s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  5s

trunk passed

+1 :green_heart:
shadedclient
32m 55s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 30s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 30s

the patch passed

+1 :green_heart:
compile
0m 27s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 27s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 20s

the patch passed

+1 :green_heart:
mvnsite
0m 29s

the patch passed

-1 :x:
javadoc
0m 27s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)

-1 :x:
javadoc
0m 25s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08 with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)

+1 :green_heart:
spotbugs
1m  4s

the patch passed

+1 :green_heart:
shadedclient
33m  0s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
1m 58s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

137m  8s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux e1125968539d 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 92cb6717eba8eee97373982cabc6db9f40f60c1f

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/2/testReport/

Max. process+thread count
665 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
12m 46s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m  0s

trunk passed

+1 :green_heart:
compile
0m 37s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
compile
0m 36s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 33s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 38s

trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javadoc
0m 34s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
32m 45s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 29s

the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04

+1 :green_heart:
javac
0m 29s

the patch passed

+1 :green_heart:
compile
0m 27s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 27s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 19s

the patch passed

+1 :green_heart:
mvnsite
0m 30s

the patch passed

-1 :x:
javadoc
0m 26s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04.txt
hadoop-tools_hadoop-azure-jdkUbuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)

-1 :x:
javadoc
0m 25s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08.txt
hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_392-8u392-ga-1~20.04-b08 with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)

+1 :green_heart:
spotbugs
1m  3s

the patch passed

+1 :green_heart:
shadedclient
33m 13s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m  1s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

138m 23s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux bfa437800130 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2912980eefe39cd43a3dcc0c7a7f8ac8608ecdaa

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/3/testReport/

Max. process+thread count
558 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m  0s

Docker mode activated.

-1 :x:
patch
0m 19s

https://github.com/apache/hadoop/pull/6552 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/4/console

versions
git=2.34.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m 10s

trunk passed

+1 :green_heart:
compile
0m 38s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 35s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 32s

trunk passed

+1 :green_heart:
mvnsite
0m 40s

trunk passed

+1 :green_heart:
javadoc
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
34m 54s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
35m 14s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 28s

the patch passed

+1 :green_heart:
compile
0m 29s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 29s

the patch passed

+1 :green_heart:
compile
0m 27s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 27s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 19s
/results-checkstyle-hadoop-tools_hadoop-azure.txt
hadoop-tools/hadoop-azure: The patch generated 2 new + 7 unchanged - 0 fixed = 9 total (was 7)

-1 :x:
mvnsite
0m 27s
/patch-mvnsite-hadoop-tools_hadoop-azure.txt
hadoop-azure in the patch failed.

-1 :x:
javadoc
0m 25s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt
hadoop-tools_hadoop-azure-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  4s

the patch passed

+1 :green_heart:
shadedclient
35m 16s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 12s

hadoop-azure in the patch passed.

-1 :x:
asflicense
0m 35s
/results-asflicense.txt
The patch generated 1 ASF License warnings.

141m 51s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux a0a84b9a3195 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / e91c4dc8eba6dad380856e13f0a7320bdc8d199e

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/6/testReport/

Max. process+thread count
566 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:::: AGGREGATED TEST RESULT ::::
HNS-OAuth
[INFO] Results:
[INFO]
[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   ITestAbfsTerasort.test_120_terasort:262->executeStage:206 » IO The ownership o...
[INFO]
[ERROR] Tests run: 340, Failures: 0, Errors: 1, Skipped: 55
HNS-SharedKey
[INFO] Results:
[INFO]
[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 3
[INFO] Results:
[INFO]
[WARNING] Tests run: 340, Failures: 0, Errors: 0, Skipped: 41
NonHNS-SharedKey
[INFO] Results:
[INFO]
[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 9
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:181 Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
[ERROR] Errors:
[ERROR]   ITestAzureBlobFileSystemChooseSAS.testOnlyFixedTokenConfigured:145 » AccessDenied
[INFO]
[ERROR] Tests run: 607, Failures: 1, Errors: 1, Skipped: 268
[INFO] Results:
[INFO]
[WARNING] Tests run: 340, Failures: 0, Errors: 0, Skipped: 44
AppendBlob-HNS-OAuth
[INFO] Results:
[INFO]
[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   ITestAbfsStreamStatistics.testAbfsStreamOps:140->Assert.assertTrue:42->Assert.fail:89 The actual value of 99 was not equal to the expected value
[ERROR] Errors:
[ERROR]   ITestAbfsTerasort.test_120_terasort:262->executeStage:206 » IO The ownership o...
[INFO]
[ERROR] Tests run: 340, Failures: 1, Errors: 1, Skipped: 79
Time taken: 25 mins 14 secs.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
11m 46s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
45m 29s

trunk passed

+1 :green_heart:
compile
0m 38s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 35s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
checkstyle
0m 31s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
32m 52s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
33m 13s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 29s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 29s

the patch passed

+1 :green_heart:
compile
0m 27s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
javac
0m 27s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 20s

the patch passed

-1 :x:
mvnsite
0m 29s
/patch-mvnsite-hadoop-tools_hadoop-azure.txt
hadoop-azure in the patch failed.

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08

+1 :green_heart:
spotbugs
1m  4s

the patch passed

+1 :green_heart:
shadedclient
32m 37s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 14s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

138m 43s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux f80ee1d270a6 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 817f7cbc3bf21f013c61ff0fdabb9fadde02149f

Default Java
Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/7/testReport/

Max. process+thread count
695 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 29s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
43m 29s

trunk passed

+1 :green_heart:
compile
0m 38s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 37s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 34s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 40s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
33m 55s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
34m 16s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 28s

the patch passed

+1 :green_heart:
compile
0m 29s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 29s

the patch passed

+1 :green_heart:
compile
0m 28s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
0m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 20s

the patch passed

-1 :x:
mvnsite
0m 30s
/patch-mvnsite-hadoop-tools_hadoop-azure.txt
hadoop-azure in the patch failed.

-1 :x:
javadoc
0m 25s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt
hadoop-tools_hadoop-azure-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 generated 2 new + 15 unchanged - 0 fixed = 17 total (was 15)

-1 :x:
javadoc
0m 26s
/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt
hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 generated 2 new + 15 unchanged - 0 fixed = 17 total (was 15)

+1 :green_heart:
spotbugs
1m  4s

the patch passed

+1 :green_heart:
shadedclient
34m 38s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 14s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 38s

The patch does not generate ASF License warnings.

128m 58s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 5a01db91f7a2 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / a65cdd7862059f0cb3d594689483f697f35aa018

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/8/testReport/

Max. process+thread count
694 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
45m 48s

trunk passed

+1 :green_heart:
compile
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 35s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 34s

trunk passed

+1 :green_heart:
mvnsite
0m 43s

trunk passed

+1 :green_heart:
javadoc
0m 41s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  9s

trunk passed

+1 :green_heart:
shadedclient
33m 57s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
34m 19s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 29s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 29s

the patch passed

+1 :green_heart:
compile
0m 28s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
0m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 20s

the patch passed

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 26s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  4s

the patch passed

+1 :green_heart:
shadedclient
33m 52s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 13s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 39s

The patch does not generate ASF License warnings.

130m 43s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/10/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux e708fd9cbad4 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d06fe415497f1ca98787119b5f7e8680afed2547

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/10/testReport/

Max. process+thread count
689 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/10/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

_ Prechecks _

+1 :green_heart:
dupname
0m 03s

No case conflicting files found.

+0 :ok:
spotbugs
0m 00s

spotbugs executables are not available.

+0 :ok:
codespell
0m 00s

codespell was not available.

+0 :ok:
detsecrets
0m 00s

detect-secrets was not available.

+0 :ok:
markdownlint
0m 01s

markdownlint was not available.

+1 :green_heart:
@author
0m 00s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m 00s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
123m 47s

trunk passed

+1 :green_heart:
compile
7m 35s

trunk passed

+1 :green_heart:
checkstyle
7m 10s

trunk passed

+1 :green_heart:
mvnsite
7m 53s

trunk passed

+1 :green_heart:
javadoc
7m 18s

trunk passed

+1 :green_heart:
shadedclient
216m 15s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
220m 11s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

-1 :x:
mvninstall
3m 54s
/patch-mvninstall-hadoop-tools_hadoop-azure.txt
hadoop-azure in the patch failed.

-1 :x:
compile
3m 32s
/patch-compile-hadoop-tools_hadoop-azure.txt
hadoop-azure in the patch failed.

-1 :x:
javac
3m 32s
/patch-compile-hadoop-tools_hadoop-azure.txt
hadoop-azure in the patch failed.

+1 :green_heart:
blanks
0m 00s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
3m 12s

the patch passed

-1 :x:
mvnsite
3m 29s
/patch-mvnsite-hadoop-tools_hadoop-azure.txt
hadoop-azure in the patch failed.

+1 :green_heart:
javadoc
3m 20s

the patch passed

-1 :x:
shadedclient
221m 00s

patch has errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
8m 14s

The patch does not generate ASF License warnings.

596m 44s

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
MINGW64_NT-10.0-17763 2dbfdbef420f 3.4.10-87d57229.x86_64 2024-02-14 20:17 UTC x86_64 Msys

Build tool
maven

Personality
/c/hadoop/dev-support/bin/hadoop.sh

git revision
trunk / d06fe415497f1ca98787119b5f7e8680afed2547

Default Java
Azul Systems, Inc.-1.8.0_332-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6552/1/testReport/

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6552/1/console

versions
git=2.44.0.windows.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 55s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
47m 20s

trunk passed

+1 :green_heart:
compile
0m 35s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 35s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 30s

trunk passed

+1 :green_heart:
mvnsite
0m 40s

trunk passed

+1 :green_heart:
javadoc
0m 38s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 32s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
34m 24s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
34m 45s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 28s

the patch passed

+1 :green_heart:
compile
0m 29s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 29s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 19s

the patch passed

+1 :green_heart:
mvnsite
0m 30s

the patch passed

+1 :green_heart:
javadoc
0m 26s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 23s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  5s

the patch passed

+1 :green_heart:
shadedclient
33m 45s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m  9s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 36s

The patch does not generate ASF License warnings.

132m 44s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/12/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 86397bc683d5 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b01a229fcbe160a9a89b91a243e3430c7c81fb98

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/12/testReport/

Max. process+thread count
709 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/12/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 6 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
46m  0s

trunk passed

+1 :green_heart:
compile
0m 34s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 34s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 29s

trunk passed

+1 :green_heart:
mvnsite
0m 39s

trunk passed

+1 :green_heart:
javadoc
0m 35s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 31s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  5s

trunk passed

+1 :green_heart:
shadedclient
36m 57s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
37m 18s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 30s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 30s

the patch passed

+1 :green_heart:
compile
0m 26s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
0m 26s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 20s

the patch passed

+1 :green_heart:
mvnsite
0m 29s

the patch passed

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  7s

the patch passed

+1 :green_heart:
shadedclient
37m  6s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m  6s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 34s

The patch does not generate ASF License warnings.

136m 45s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/13/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux f5aba3b9be2a 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7c9d5b45ec896f5a6acdb418d6445fafde8e8cbe

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/13/testReport/

Max. process+thread count
552 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/13/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:::: AGGREGATED TEST RESULT ::::
============================================================
HNS-OAuth
[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 2
[WARNING] Tests run: 620, Failures: 0, Errors: 0, Skipped: 76
[WARNING] Tests run: 380, Failures: 0, Errors: 0, Skipped: 54
============================================================
HNS-SharedKey
[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 620, Failures: 0, Errors: 0, Skipped: 28
[WARNING] Tests run: 380, Failures: 0, Errors: 0, Skipped: 41
============================================================
NonHNS-SharedKey
[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 604, Failures: 0, Errors: 0, Skipped: 268
[WARNING] Tests run: 380, Failures: 0, Errors: 0, Skipped: 44
============================================================
AppendBlob-HNS-OAuth
[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 2
[WARNING] Tests run: 620, Failures: 0, Errors: 0, Skipped: 78
[WARNING] Tests run: 380, Failures: 0, Errors: 0, Skipped: 78
Time taken: 56 mins 59 secs.

Thanks @steveloughran for the review.
This is good to be merged now.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 54s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 7 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m 42s

trunk passed

+1 :green_heart:
compile
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 36s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 33s

trunk passed

+1 :green_heart:
mvnsite
0m 42s

trunk passed

+1 :green_heart:
javadoc
0m 38s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 35s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
33m 31s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
33m 52s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 30s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 30s

the patch passed

+1 :green_heart:
compile
0m 28s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
0m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 20s
/results-checkstyle-hadoop-tools_hadoop-azure.txt
hadoop-tools/hadoop-azure: The patch generated 1 new + 9 unchanged - 0 fixed = 10 total (was 9)

+1 :green_heart:
mvnsite
0m 30s

the patch passed

+1 :green_heart:
javadoc
0m 26s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  5s

the patch passed

+1 :green_heart:
shadedclient
33m 24s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 10s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 38s

The patch does not generate ASF License warnings.

129m  7s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/14/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 0058a3fedccc 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 5db5372240aeb83bd48ab3e774273e507820cdf2

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/14/testReport/

Max. process+thread count
562 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/14/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

+1 pending that change to abfs.md that @anmolanmol1234 spotted

Which change are you referring to??
I can see a comment regarding the typo which anmol spotted and already fixed that.
Am I missing any other comment?
Thanks

I'm afraid you are very unlucky in that I have been staring at the ABFS auth code for diagnostics purposes today and so these lines of code are now very familiar.

getPasswordString/getTrimmedPasswordString() must be used for lookup. that way you can keep the secret in a JCEKS file.
proposed a way to avoid double wrapping exceptions

Thanks for the pointers here.
Please let me know if changes made looks good.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  2s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 7 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m 16s

trunk passed

+1 :green_heart:
compile
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 35s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 34s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 40s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
33m 31s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
33m 52s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 29s

the patch passed

+1 :green_heart:
compile
0m 31s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 31s

the patch passed

+1 :green_heart:
compile
0m 29s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
0m 29s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 19s
/results-checkstyle-hadoop-tools_hadoop-azure.txt
hadoop-tools/hadoop-azure: The patch generated 3 new + 9 unchanged - 0 fixed = 12 total (was 9)

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 27s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  4s

the patch passed

+1 :green_heart:
shadedclient
33m 40s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m  9s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

129m  3s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/15/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 24be86ac3909 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ecd151a7fac10629fb93f8f25b3a8567caf172ac

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/15/testReport/

Max. process+thread count
552 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/15/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

all of which are trivial

Done.
Thanks

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 55s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 7 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m  9s

trunk passed

+1 :green_heart:
compile
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 37s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 33s

trunk passed

+1 :green_heart:
mvnsite
0m 42s

trunk passed

+1 :green_heart:
javadoc
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  7s

trunk passed

+1 :green_heart:
shadedclient
33m  3s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
33m 24s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 31s

the patch passed

+1 :green_heart:
compile
0m 29s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 29s

the patch passed

+1 :green_heart:
compile
0m 27s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
0m 27s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 19s

the patch passed

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 27s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 25s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  4s

the patch passed

+1 :green_heart:
shadedclient
33m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 10s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

128m 21s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/16/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 300cd269972c 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 820e2e0d0c98b3441a71f0101915990911052959

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/16/testReport/

Max. process+thread count
705 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/16/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

_ Prechecks _

+1 :green_heart:
dupname
0m 03s

No case conflicting files found.

+0 :ok:
spotbugs
0m 00s

spotbugs executables are not available.

+0 :ok:
codespell
0m 00s

codespell was not available.

+0 :ok:
detsecrets
0m 00s

detect-secrets was not available.

+0 :ok:
markdownlint
0m 00s

markdownlint was not available.

+1 :green_heart:
@author
0m 00s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m 00s

The patch appears to include 7 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
93m 50s

trunk passed

+1 :green_heart:
compile
5m 06s

trunk passed

+1 :green_heart:
checkstyle
4m 56s

trunk passed

+1 :green_heart:
mvnsite
5m 12s

trunk passed

+1 :green_heart:
javadoc
4m 53s

trunk passed

+1 :green_heart:
shadedclient
154m 16s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
156m 46s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 47s

the patch passed

+1 :green_heart:
compile
2m 22s

the patch passed

+1 :green_heart:
javac
2m 23s

the patch passed

+1 :green_heart:
blanks
0m 00s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
2m 09s

the patch passed

+1 :green_heart:
mvnsite
2m 30s

the patch passed

+1 :green_heart:
javadoc
2m 14s

the patch passed

+1 :green_heart:
shadedclient
167m 11s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
5m 47s

The patch does not generate ASF License warnings.

439m 35s

Subsystem
Report/Notes

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
MINGW64_NT-10.0-17763 ca0f186c13b4 3.4.10-87d57229.x86_64 2024-02-14 20:17 UTC x86_64 Msys

Build tool
maven

Personality
/c/hadoop/dev-support/bin/hadoop.sh

git revision
trunk / 820e2e0d0c98b3441a71f0101915990911052959

Default Java
Azul Systems, Inc.-1.8.0_332-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6552/3/testReport/

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6552/3/console

versions
git=2.45.0.windows.1

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Hi @steveloughran @mukund-thakur
Requesting you to kindly review the latest changes and merge if they looks good.
Thanks

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 7 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
43m 45s

trunk passed

+1 :green_heart:
compile
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
0m 37s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
0m 34s

trunk passed

+1 :green_heart:
mvnsite
0m 42s

trunk passed

+1 :green_heart:
javadoc
0m 39s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  6s

trunk passed

+1 :green_heart:
shadedclient
33m 40s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
34m  1s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 30s

the patch passed

+1 :green_heart:
compile
0m 31s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
0m 31s

the patch passed

+1 :green_heart:
compile
0m 28s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
0m 28s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 19s

the patch passed

+1 :green_heart:
mvnsite
0m 31s

the patch passed

+1 :green_heart:
javadoc
0m 26s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
0m 24s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
1m  6s

the patch passed

+1 :green_heart:
shadedclient
33m 46s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 10s

hadoop-azure in the patch passed.

+1 :green_heart:
asflicense
0m 38s

The patch does not generate ASF License warnings.

128m 40s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/17/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6552

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint

uname
Linux 0bf4c8d5d4fb 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 298640f65c486f62db490d2570b1db9402189cc9

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/17/testReport/

Max. process+thread count
552 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6552/17/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@anujmodi2021 merged to trunk. Can you do the branch-3.4 cherrypick, rerun the tests and then push up as a PR to merge there too? thanks

@anujmodi2021 merged to trunk. Can you do the branch-3.4 cherrypick, rerun the tests and then push up as a PR to merge there too? thanks

Thanks a lot Steve.
Here is the Backport PR for branch_3.4
https://github.com/apache/hadoop/pull/6855


breaking: update to ember-local-storage v2

Breaking
Update to ember-local-storage v2 (#167)
See changes here https://github.com/funkensturm/ember-local-storage/blob/master/CHANGELOG.md

Is it really a breaking change?


which is better OpenSSL version for debian buster (raspbian/raspberry armv7l)

I'm on project that implementation of OpenSSL modification on my Raspberry Pi 3 Model B+ that used armv7l with OS Raspbian within Debian Buster. On the official debian website https://packages.debian.org/buster/openssl , it says that the OpenSSL 1.1.1d is the stable version for Debian Buster, but from the OpenSSL project, the newest version is OpenSSL 1.1.1k. Which version is the best for my project, and what's the impact if I used the 1.1.1k version, while the 1.1.1d is the stable version for debian buster?

The only difference between 1.1.1d and 1.1.1k are bug and security fixes. The Debian version will be 1.1.1d plus backports of important security fixes. Usually it is sufficient to use the version of OpenSSL provided by your distro.

Answered, closing.


[13.0.x] ISPN-13633 dev guide title

https://issues.redhat.com/browse/ISPN-13633
Need to include the Dev Guide in the index for 13.0.x

Thanks @oraNod


Make the Curve25519 call-graph indexed over a predicate

Instead of being parameterized over a field representation, the generic
code is now parameterized over a field representation and a global
invariant.
type field_spec = M51 | M64
let index = field_spec & Type

Therefore, at instantiation-time, we now do:
let scalarmult_vale64 = mk_scalarmult (M64, Vale.CPU_Features.(adx /\ bmi2))
let scalarmult_hacl64 = mk_scalarmult (M64, True)
let scalarmult_hacl51 = mk_scalarmult (M51, True)

where mk_scalarmult is the higher-order, tactic-rewritten function
that generates a specialized instance of scalarmult for:

a given field representation
a global invariant that is carried throughout the whole call-graph
a given set of functions that require the global invariant and
implement the core operations for that field representation.

M64 has two implementations; M51 only has one. This is modular: further
implementations possibly requiring new fancy invariants may be added at
any time without having to touch the curve implementation.
This fixes a previous limitation of the curve code, wherein the hacl64
version would still inherit the Vale precondition because the code had
no way of referring to an abstract precondition that would be
instantiated depending on what the core operations need.
The implementation works and has the nice property that it didn't
require modifying the tactic. However, the result is somewhat verbose,
and achieving proper reduction behavior was a struggle. Indeed, having
the index be a pair now requires us to be very careful when destructing
and/or constructing.

In types, intermediary let-bindings where the left-hand side is a
pattern block reduction and prevent calls from type-checking. Therefore,
we do:

inline_for_extraction
let f_t (i: index) =
  let s = fst i in
  let p = snd i in
  ... ->
  felem s ->
  ... ->
  Stack (fun _ -> requires p /\ ...) (...)

instead of let s, p = i.

The default fst projector is not marked as inline_for_extraction,
so we shadow it and only use our version marked as
inline_for_extraction. In code, we use [@inline_let] to define s.

The presence of a pair seems to block reduction. This does not work:

inline_for_extraction
let i = M51, True

val f: #i:index -> f_t i

but this does:
val f: #i:index -> f_t (M51, True)

The index being a pair seems to interact poorly with implicit
argument inference and many call-sites had to be annotated explicitly.

Unless the predicate is annotated at Type0, universe binding problems
arise, as the tactic assumes that the code being rewritten is not
universe-polymorphic.

There may be other solutions; for instance, one might imagine a second
index that is not touched by the tactic, e.g:
val g_t (s: field_spec) =
  #p:Type ->
  ... ->
  felem s ->
  ... ->
  Stack (fun _ -> requires p /\ ...) ...

val g: #s:field_spec -> g_t s

This is only marginally more verbose than the solution in this commit;
it might also reduce struggles when it comes to dealing with pairs.
However, the implicit argument will have to be explicitly provided,
unless...
... unless we write a tactic to infer p by just doing a lookup in the
context and finding a p that is an argument to the current function,
and has type Type. But this is also high-tech and we may have blown
the complexity budget for that.

I'm leaving this out for discussion with @s-zanella and @R1kM ... I'm not super satisfied, but this may be the only reasonable solution for now. The alternative I'm outlining might be marginally better, I don't know... thoughts welcome.

Did you consider internalizing the invariant in the definition of the index type?
type implementation = | Hacl | Vale
type index = M51 of implementation | M64 of implementation | M64 of implementation

Or even
type index = M51_Hacl | M64_Hacl | M64_Vale

This only requires changing the definition of limb, nlimb, wide, nwide to pattern match on the new type.
That would be less verbose and avoid the problems with pairs. Am I missing something obvious?

Yes. This was ruled out early on, because:

it would introduce conversions when calling spec, e.g. == Spec.Curve25519.ecdh (spec_alg_of_impl_alg a) ...
it would introduce implementation details -- so far, Hacl.Impl.Curve25519 is oblivious to whether there is one or many implementations for a given representation
adding an implementation would require touching Hacl.Impl, i.e. we would lose modularity

I really like the property that Hacl.Impl.Curve25519 is completely core-field-implementation-agnostic and this is what I'm hoping to preserve. But the main pain stems from the use of a pair in the index.
Maybe doing: #s:field_spec -> #p:Type -> ... requires p (like I suggest near the end) would be much simpler and the same amount of complexity as we have now. I just haven't had a chance to experiment with it -- we'd also need a little bit of tactics to make sure the implicit argument is instantiated sensibly.

Got it. Thanks for the detailed explanation.
Then I don't see other solution than modifying the tactic in some way.
The solution you described in a comment of parametrizing the call graph by a global invariant looked promising, but it requires the tactic to rewrite and fiddle with types.
If this pattern will be common enough, it may be worth to modify the tactic to take an invariant as a second parameter (or add a new one, to avoid modifying the existing algorithms that don't use this feature) .

I can try that. Thanks for the in-depth look and helpful advice!

commit 6d73fabf280012a86ff5c3a1626643179ffd76f9 (master)
Author: Jonathan Protzenko<EMAIL_ADDRESS>Date:   Mon Mar 2 13:15:29 2020 -0800
Rewrite higher-order tactic to insert preconditions.

This follows Santiago's suggestion in https://github.com/project-everest/hacl-star/pull/243

Essentially, the tactic, upon encountering a node in the call-graph,
unfolds its type (because most declarations are of the form `let f: #i:index -> f_t i`),
then pushes a precondition `p` in its requires clause, which then gets
threaded all throughout the call-graph.

This generates code that type-checks slower than before, owing to the
need to unfold and refold types; furthermore, situations where a type
could be left as-is because no extra parameters were needed now gets
rewritten systematically.

commit 447128b0dfc60fad30bedf3373e37ebaeef63acc (HEAD -> protz_pre, origin/protz_pre)
Author: Jonathan Protzenko<EMAIL_ADDRESS>Date:   Mon Mar 2 14:03:11 2020 -0800
Adapt the code to the updated tactic.

There are some unpleasant side-effects; for instance, type abbreviations
normally wouldn't need to mention the extra precondition that gets
inserted by the tactic; however, in the interest of sharing types
between precondition-unaware, generic code and specialized instances of
base combinators that *do* require a precondition, I added the parameter
to a few types.

Other than that, works fine.

@protz
Update other algorithms too
2e40c92
This updated tactic seems to yield code that type-checks slower,
possibly owing to type abbrevations being less shared between various
variants of the function.

So this passes CI, but the code seems to type-check less efficiently. (I don't have hard evidence, just based on what I saw on my laptop.)
This is also quite a fair amount of extra complexity, but better than the previous iteration of this patch (which I left in the commit history via git revert).
If Curve25519 was the only algorithm for which we need this, I'd be inclined to do it manually; however, we might need it for other algorithms in which case it might turn out to be convenient. But for the purposes of sharing type abbreviations, the user is somehow required to anticipate on the insertion of the precondition by the tactic, which is weird.

This looks great and much cleaner than before!
My original suggestion was to keep both versions of the tactic and use the simpler version when the invariant is trivial. That way we pay for the extra typechecking cost only when it's needed.
I realize maintaining two versions of the tactic is a burden (unless modifying the tactic so that it doesn't rewrite types to add trivial invariants is easy). If typechecking is not significantly more expensive, I think that always carrying an invariant is reasonable.

It would've been some extra complexity to add a mode where no predicate is inserted, and in a sense there should always be a predicate (even though this is not enforced for the HACL vectorized instructions). So, I left it in there, even though it forces a rewriting of some leaf nodes that would not be rewritten otherwise.
The slowdown is not noticeable on CI so I'm inclined to merge this, pending @R1kM's approval once the ICFP deadline is over.

The PR looks good to me, feel free to merge.
How hard would it be to extend the predicate p to operate on HS.mem instead of being a pure predicate? I don't think any of the existing code requires it, but HPKE would: https://github.com/project-everest/hacl-star/blob/afromher_hpke/code/hpke/Hacl.Impl.HPKE.fsti#L35 . At the moment, we only have one implementation of P256, so adding this precondition on any instantiation with P256 is not restrictive, but we might have similar cases in the future.

if hpke needs it then of course we should make it happen; I think it's pretty trivial to do
I don't understand, however, from your example, where h is needed

On lines 36 and 37, when the curve used is P256, we require bounds on nat_from_bytes_le (as_seq h0 skE)
At the moment, we only have one implementation of P256, so it is not really restrictive. Nevertheless,
in the future, I could imagine a different implementation that does not require this precondition, or that internalizes it.

This is trickier because now you need to receive the heap and possibly the arguments, too -- which makes the precondition really not generic at all and means that you can no longer have a single precondition threaded for all the call-graph.

These preconditions aren't necessary. I removed them (see https://github.com/project-everest/hacl-star/commit/afe04c947f55a81f48af6c4e99d4b0bd5abee488). @R1kM please update HPKE when you can.
I discussed this with @Frosne who had concerns about eliminating the requirement that the scalar is in the [1..p-1] interval. I argued that the scalar is a trusted input and doesn't need to be validated, We discussed the possibility of validating it dynamically. She also suggested adding alternative functions with the pre-conditions, but the C code would be identical. I'm curious to hear your preferences.

@beurdouche @blipp

I discussed this with @Frosne who had concerns about eliminating the requirement that the scalar is in the [1..p-1] interval. I argued that the scalar is a trusted input and doesn't need to be validated, We discussed the possibility of validating it dynamically. She also suggested adding alternative functions with the pre-conditions, but the C code would be identical. I'm curious to hear your preferences.

I think it is important for is to implement the dynamic check for P256 before removing the precondition; we have no idea what will happen if a scalar outside the prime field is provided to the code. Can we add a PR to fix this please? Unilaterally removing the precondition from master seems dangerous :(

We know exactly what happens because it's specified here: https://github.com/project-everest/hacl-star/blob/master/specs/ecdsap256sha2/Spec.DH.fst.
I don't believe there is anything wrong with the specification, but if we keep the pre-conditions in the spec, they also would need to be added in the specification.

We know exactly what happens because it's specified here: https://github.com/project-everest/hacl-star/blob/master/specs/ecdsap256sha2/Spec.DH.fst.
I don't believe there is anything wrong with the specification, but if we keep the pre-conditions in the code, they also would need to be added in the specification.

Ok, makes sense.. : ) We'll discuss more in Paris as we wanted a defensive API for a while and we'll get back...

Unilaterally removing the precondition from master seems dangerous :(
I don't see how this could be dangerous for C clients. The extracted code is identical.

What I'm getting from this discussion is:

threading a restrictive precondition throughout spec & implementation is redundant, because the outermost entry point that is exposed to F* clients already properly specifies what happens if the point is out of range
@beurdouche would like a defensive C API; maybe you would be interested in sketching Hacl.Defensive.ECDSA?
this means that my pull request is good in its current and I can merge it.

Let's chat more about this this week.

I will double check the precondition story with @Frosne, until please don't merge until then.

@beurdouche this is exclusively about the tactic, which changes nothing to the current state of the code -- why block this pull request?

Hacl.Defensive.ECDSA, I think it's more about me.

@beurdouche https://github.com/beurdouche this is exclusively about the tactic, which changes nothing to the current state of the code -- why block this pull request?

I didn’t go back to the top of the discussion, that’s why I didn’t realize, but  - please - I am
not the one that started to fork the discussion on something else within this thread…
This discussion should move to its own issue...

Thanks @s-zanella . I merged from master and I'm currently removing this precondition from the HPKE code on my branch. Changes to add predicates operating on memory do not seem needed at the moment, nor in the near future.
I'd be in favour of merging this PR: It is clean and self-contained to the tactic changes (and propagating it to actual code), and it'd be nice to have this code under CI.


Split generated

What's the status of this PR?

What's the status of this PR?

This one is included and a bit changed here


Deprecate and remove 'eager' task classes task and shared_task

Once we have a way to synchronously block waiting for a task in #27 and when_all in #10 there shouldn't be any need for the eagerly started task types any more and we can limit use to just the lazy task types (ie. lazy_task and shared_lazy_task)
One of the big motivations for getting rid of eagerly-started tasks is that it is difficult to write exception-safe code with use of eagerly-started tasks that aren't immediately co_awaited.
We don't want to leave dangling tasks/computation as this can lead to ignored/uncaught exceptions or unsynchronised access to shared resources.
For example, consider the case where we want to execute two tasks concurrently and wait for them both to finish before continuing, using their results:
task<A> do1();
task<B> do2();

task<> do1and2concurrently_unsafe()
{
  // The calls to do1() and do2() start the tasks eagerly but are potentially executed
  // in any order and either one of them could fail after the other has already started
  // a concurrent computation.
  auto [a, b] = cppcoro::when_all(do1(), do2());

  // use a and b ...
}

To implement this in an exception-safe way we'd need to modify the function as follows:
task<> do1and2concurrently_safe()
{
  // This implicitly starts executing the task and it executes concurrently with the
  // rest of the logic below until we co_await the task.
  task<A> t1 = do1();

  // Now we need to start the do2() task but since this could fail we need
  // to start it inside a try/catch so we can wait for t1 to finish before it
  // goes out of scope (we don't want to leave any dangling computation).
  task<B> t2;
  std::exception_ptr ex;
  try
  {
    // This call might throw, since it may need to allocate a coroutine frame
    // which could fail with std::bad_alloc.
    t2 = do2();
  }
  catch (...)
  {
    // Don't want to leave t1 still executing but we can't co_await inside catch-block
    // So we capture current exception and do co_await outside catch block.
    ex = std::current_exception();
  }

  if (ex)
  {
    // Wait until t1 completes before rethrowing the exception.
    co_await t1.when_ready();
    std::rethrow_exception(ex);
  }

  // Now that we have both t1 and t2 started successfully we can use when_all() to get the results.
  auto [a, b] = when_all(std::move(t1), std::move(t2));

  // use a, b ...
}

Compared with lazy_task version which is both concise and exception-safe:
lazy_task<> do1();
lazy_task<> do2();

lazy_task<> do1and2concurrently()
{
  // The calls to do1() and do2() can still execute in any order but all they
  // do is allocate coroutine frames, they don't start any computation.
  // If the second call fails then the normal stack-unwinding will ensure the
  // first coroutine frame is destroyed. It doesn't need to worry about waiting
  // for the first task to complete. Either they both start or none of them do.
  auto [a, b] = cppcoro::when_all(do1(), do2());

  // use a and b ...
}

With a lazy_task, the task is either being co_awaited by some other coroutine or it is not executing (it has either not yet started, or has completed executing) and so the lazy_task is always safe to destruct and will free the coroutine frame it owns.
A side-benefit of using lazy_task everywhere is that it can be implemented without the need for std::atomic operations to synchronise coroutine completion and awaiter. This can have some potential benefits for performance by avoiding use of atomic operations for basic sequential flow of execution.

Started the process of removing use of task<T> from tests in b78bc3ed7b0c2c13fb92483a6b34e31589e22286.
There are still the where_all_tests.cpp and where_all_ready_tests.cpp to go but I've been running into some crashes when updating these that I need to investigate.

Commit 722fe1b24a3d0cb5fbafa17ee167e2a024f59bf7 removes use of eager tasks from where_all_ready_tests.cpp.
Only where_all_tests.cpp to go.

Commit 421de76f189c1f38a621898eaec77dc3700b9c47 remotes last of eager task usage from tests (except where testing eager tasks themselves).

Commit 64b0a049f9e07eef7082b1d4de31686975c3cdc8 removes the task<T> and shared_task<T> classes and updates the README.
All that remains now is renaming lazy_task -> task and shared_lazy_task -> shared_task.

Commit bf9eb2c58110a650cc4128bef03c8d9d87be62f7 has implemented the rename.
Closing.


libvgm & co: init

Motivation for this change
Packaging:

libvgm, a library for emulating a plethora of sound chips & systems
libvgm-based vgmplay, a player for VGM files
(there exists a more well known, regular vgmplay but it's considered legacy and an absolute licensing mess)
mmlgui, a graphical MML editor that uses libvgm for Sega Mega Drive sound emulation & general audio playback

To test vgmplay:

Download a VGM/VGZ file
i.e. from https://vgmrips.net/packs/
example: https://vgmrips.net/packs/vgm/Computers/Sharp/Asuka_120%25_Burning_Fest._(Sharp_X68000)/11 Megumi Stage.vgz)
vgmplay /path/to/file, have working playback

To test mmlgui: I include the examples from the underlying MML compiler in the output. The program doesn't understand command-line arguments or symlinks in its file manager so you'll have to press the L next to the current directory marker and paste the full result path into it to find them.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 147503 run on x86_64-linux 1

2 packages built:

libvgm
vgmplay-libvgm

Result of nixpkgs-review pr 147503 run on x86_64-darwin 1

2 packages built:

libvgm
vgmplay-libvgm

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/647

Bumped libvgm to latest commit.

Added another libvgm-using tool that only needs GPL-licensed emulation cores, so I can get some testing from OfBorg. Will look into the aarch64-darwin failure.

I don't quite understand what could be wrong on aarch64-darwin, the logs between aarch64 & x86_64 look pretty much the same but only one of them links fine. On aarch64 it isn't using fixDarwinDylibNames, but adding that didn't seem to fix anything.
Maybe someone from @NixOS/darwin-maintainers could help me out here? I don't have access to aarch64-darwin hardware to debug this myself.

Something seems to be wrong with pkgconf on aarch64-darwin. When I run the link step manually, it is killed due to an invalid signature.
default	17:57:30.061889-0500	kernel	CODE SIGNING: cs_invalid_page(0x1043e0000): p=42113[pkgconf] final status 0x23000200, denying page sending SIGKILL
default	17:57:30.061911-0500	kernel	CODE SIGNING: process 42113[pkgconf]: rejecting invalid page at address 0x1043e0000 from offset 0x8000 in file "/nix/store/q7j2ygnd3xj0k00dc450y47yn4c4hnzf-pkgconf-1.7.4-lib/lib/libpkgconf.3.dylib" (cs_mtime:1.0 == mtime:1.0) (signed:1 validated:1 tainted:1 nx:0 wpmapped:0 dirty:0 depth:0)
default	17:57:30.063328-0500	kernel	CODE SIGNING: process 42113[pkgconf]: rejecting invalid page at address 0x1043e0000 from offset 0x8000 in file "/nix/store/q7j2ygnd3xj0k00dc450y47yn4c4hnzf-pkgconf-1.7.4-lib/lib/libpkgconf.3.dylib" (cs_mtime:1.0 == mtime:1.0) (signed:1 validated:1 tainted:1 nx:0 wpmapped:0 dirty:0 depth:0)
default	17:57:30.063366-0500	kernel	pkgconf[42113] Corpse allowed 1 of 5
default	17:57:30.091727-0500	ReportCrash	Formulating fatal 309 report for corpse[42113] pkgconf
default	17:57:30.094407-0500	ReportCrash	no MetricKit for process pkgconf type 309 bundleId (null)
default	17:57:30.094608-0500	ReportCrash	Sending event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x00000001043e1aec(\n    50,\n<PHONE_NUMBER>\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x00000001043e1aec","incidentID":"64553884-CE64-4052-B0E4-E0CA2DB5D3FA","logwritten":0,"process":"pkgconf","responsibleApp":"iTerm2","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING"}
default	17:57:30.094930-0500	analyticsd	Received event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x00000001043e1aec(\n    50,\n<PHONE_NUMBER>\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x00000001043e1aec","incidentID":"64553884-CE64-4052-B0E4-E0CA2DB5D3FA","logwritten":0,"process":"pkgconf","responsibleApp":"iTerm2","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING"}
default	17:57:30.095097-0500	analyticsd	Aggregated. Transform: StabilityCrashNumerator3WithBundleVersion Dirty: 1 Event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x00000001043e1aec(\n    50,\n<PHONE_NUMBER>\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x00000001043e1aec","incidentID":"64553884-CE64-4052-B0E4-E0CA2DB5D3FA","logwritten":0,"process":"pkgconf","responsibleApp":"iTerm2","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING","timestamp":1646175450094545}
default	17:57:30.095211-0500	analyticsd	Aggregated. Transform: StabilityCrashNumerator3 Dirty: 1 Event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x00000001043e1aec(\n    50,\n<PHONE_NUMBER>\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x00000001043e1aec","incidentID":"64553884-CE64-4052-B0E4-E0CA2DB5D3FA","logwritten":0,"process":"pkgconf","responsibleApp":"iTerm2","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING","timestamp":1646175450094545}
default	17:57:30.095323-0500	analyticsd	Aggregated. Transform: StabilityCrashNumerator3WithIncidentID Dirty: 1 Event: com.apple.stability.crash {"exceptionCodes":"0x0000000000000032, 0x00000001043e1aec(\n    50,\n<PHONE_NUMBER>\n)EXC_BAD_ACCESSSIGKILL (Code Signature Invalid)UNKNOWN_0x32 at 0x00000001043e1aec","incidentID":"64553884-CE64-4052-B0E4-E0CA2DB5D3FA","logwritten":0,"process":"pkgconf","responsibleApp":"iTerm2","terminationReasonExceptionCode":"0x2","terminationReasonNamespace":"CODESIGNING","timestamp":1646175450094545}

I can confirm mmlgui builds and runs now on aarch64-darwin. nixpkgs-review now doesn’t build anything though.
Result of nixpkgs-review pr 147503 run on aarch64-darwin 1

Thanks for the feedback. It looks like pkgconf is using removeReferencesTo in its postFixup phase to remove some unneeded references in its binaries & libraries. removeReferencesTo has a setup hook on aarch64-darwin which will redo the codesigning in the fixupPhase.

The removeReferencesTo script is meant to correct the signature automatically. The hook configuration is internal to removeReferencesTo, though the name is a bit confusing. There's a bug in the script itself which doesn't apply the fixupHooks to all of the modified files, which let the pkgconf library slip through without being fixed up. I've made https://github.com/NixOS/nixpkgs/pull/162447 to fix it.

Added some collections of related & often used tools for the format.
Result of nixpkgs-review pr 147503 run on x86_64-linux 1

5 packages built:

fmtoy
libvgm
mmlgui
vgmplay-libvgm
vgmtools

Result of nixpkgs-review pr 147503 run on x86_64-darwin 1

5 packages built:

fmtoy
libvgm
mmlgui
vgmplay-libvgm
vgmtools

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/892


Allow user-specified bundle uuid for upload().

@jessebrennan There shouldn't be any problems.  The version is still live so it'll create an updated bundle each time.

Codecov Report

Merging #499 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master     #499   +/-   ##
=======================================
  Coverage   85.67%   85.67%
=======================================
  Files          42       42
  Lines        1906     1906
=======================================
  Hits         1633     1633
  Misses        273      273

Impacted Files
Coverage Δ

hca/dss/__init__.py
90.41% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a7c950...e7f56c1. Read the comment docs.


How to use the docs Search component on custom Navbar or above the Sidebar

I want to use a custom navbar and I need to have the searchbar also.
But it's not available without the default navbar.
Is there any way we can enable the search bar on the sidebar or on the custom navbar?

Did you find a solution yet?

Did you find a solution yet?

No, I don't think it's possible. So, we used the default navbar with the searchbar.


open question - should inset values be calculated from the labels.

You can see the change when the labels are displayed for the leading edge:

vs

(debugging shows the areas identified more easily):

That small gap between the axis and the chart area is inset calculated from the height and width of the labels of the scales, and set automatically so that there will be sufficient space to display the labels without clipping them, for the case when there's no margin.
So maybe rather than insets, these adjustments should be made to the margins of the chart, which would still keep the labels visible and clear, but not "loosen up" the chart from the axis.

reviewing the calculations, I was effectively mixing margin and inset, so I adjusted the code to make the margin adjustment more explicit, and a max(spec vs. needed space) calculation in the renderer


Error: Unable to access jarfile /root/Tools/java2smali/build/libs/java2smali-1.1.jar

any one get this problem can help please ?!

nano java2smali and change the directory in the code to where the jar file is. Sorry it's taken 3 years for someone to even reply to this.


[ko]Translate tasks/configure-pod-container/configure-liveness-readiness-startup-probes into Korean

This is a Feature Request

What would you like to be added
Translate tasks/configure-pod-container/configure-liveness-readiness-startup-probes into Korean
◾ Page: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
◾ File: https://github.com/kubernetes/website/blob/main/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md

Why is this needed
Not translated yet

Comments
/language ko
/assign

/triage accepted

I think @gy-ulbak96 is working on this issue.
/remove-lifecycle stale

/remove-lifecycle stale

/triage accepted


Remove watch on secrets from ManagedSeed controller

How to categorize this PR?
/area control-plane
/kind bug
What this PR does / why we need it:
Removes watching of secrets in the garden cluster from the ManagedSeed controller. During deletion, the controller will requeue with WaitSyncPeriod when waiting for the seed secrets to be deleted instead of relying on events, similarly to the way it does it for resources that are not in the garden cluster.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
In the future, gardenlet may no longer be allowed to list/watch secrets - it will only be allowed to get secrets that somehow belong to its seed. Therefore, the watching of secrets by ManagedSeed controller in the garden cluster should be removed.
Release note:
The ManagedSeed controller no longer watches secrets in the garden cluster.

/cc @rfranzke @timebertt


Make the "Review your answers" link (section) work on mobile

We turned off the feature that shows section headings on mobile

This was fixed by changing the default small screen navigation: dropdown setting.


feat(expressions): Extend Expression.url.upload() to support row-specific URLs

Addresses #3320
Expression.url.upload() can now take in a column of urls to upload each row to a specific url.

@raunakab assigning you for review since you're oncall :)

Perfect! Will take a look through.


Support Laravel 5.7

On attempting to install on laravel 5.7, I get this error

Installation request for martinlindhe/laravel-vue-i18n-generator ^0.1.28 -> satisfiable by martinlindhe/laravel-vue-i18n-generator[0.1.28].
Conclusion: remove laravel/framework 5.7.x-dev
Conclusion: don't install laravel/framework 5.7.x-dev

Patch is welcome, if you are able to verify it working.
May just need to update the require block
https://github.com/martinlindhe/laravel-vue-i18n-generator/blob/master/composer.json#L13