HADOOP-18708: Support S3 Client Side Encryption(CSE) With AWS SDK V2

Description of PR
This commit adds support for S3 client side encryption (CSE). CSE can configured in two modes CSE-KMS where keys are provided by AWS KMS and CSE-CUSTOM where custom keys are provided by implementing custom keyring
CSE is implemented using S3EncryptionClient (V3 client) and additional configurations (mentioned below) were added to make it compatible with the older encryption client V1 and V2 which is turned OFF by default.
Inorder to have compatibility with V1 client the following operations are done.

V1 client pads extra bytes in multiple of 16 i.e if the file size is 12 bytes, 4bytes are padded to make it multiple of 16. Inorder to get the unencrypted file size of such S3 object ranged S3 GET call is made
V1/V2 client supports storing encrypted metadata in instruction file (.instruction) and hence those files are skipped during listing.
Unlike V1/V2 client V3 client does not support reading unencrypted object, Additional s3 client (base client) is introduced to read mix of encrypted and unencrypted s3 objects.

Default Behavior
The configurations to make it backward compatible is turned OFF by default considering the performance implications. The default behavior is as follows

The unencrypted file size is computed by simply subtracting 16 bytes from the file size.
When there is a mix of unencrypted and encrypted s3 objects, The client fails.

This PR is based on the initial work done by @ahmarsuhail as part of https://github.com/apache/hadoop/pull/6164
How was this patch tested?

Tested in us-east-1 with mvn -Dparallel-tests -DtestsThreadCount=16 clean verify.
Added integration test for CSE-KMS and CSE-CUSTOM

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
31m 58s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m  3s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
33m  2s

trunk passed

+1 :green_heart:
compile
17m 11s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
15m 58s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
4m 29s

trunk passed

+1 :green_heart:
mvnsite
1m 39s

trunk passed

+1 :green_heart:
javadoc
1m 22s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
1m 29s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+0 :ok:
spotbugs
0m 45s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 10s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
1m  5s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
0m 46s

the patch passed

+1 :green_heart:
compile
16m 39s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
16m 39s

the patch passed

+1 :green_heart:
compile
16m  3s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
16m  3s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m 28s
/results-checkstyle-root.txt
root: The patch generated 11 new + 19 unchanged - 1 fixed = 30 total (was 20)

+1 :green_heart:
mvnsite
1m 37s

the patch passed

+1 :green_heart:
javadoc
1m 18s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

-1 :x:
javadoc
0m 50s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08.

+0 :ok:
spotbugs
0m 37s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
34m 40s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
3m 21s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  5s

The patch does not generate ASF License warnings.

247m 54s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle markdownlint

uname
Linux 99d156afd598 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 0a4784d31b274cd9f699ae4817d58fe441bbfe1b

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/1/testReport/

Max. process+thread count
623 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran @ahmarsuhail Could you please review the changes ?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 51s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 27s

trunk passed

+1 :green_heart:
compile
17m 22s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
16m  4s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
4m 22s

trunk passed

+1 :green_heart:
mvnsite
1m 35s

trunk passed

+1 :green_heart:
javadoc
1m 17s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
1m 28s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+0 :ok:
spotbugs
0m 47s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 35s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
1m  3s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
0m 45s

the patch passed

+1 :green_heart:
compile
16m 54s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
16m 54s

the patch passed

+1 :green_heart:
compile
16m  3s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
16m  3s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 19s
/results-checkstyle-root.txt
root: The patch generated 1 new + 19 unchanged - 1 fixed = 20 total (was 20)

+1 :green_heart:
mvnsite
1m 35s

the patch passed

+1 :green_heart:
javadoc
1m 16s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

-1 :x:
javadoc
0m 50s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08 with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+0 :ok:
spotbugs
0m 38s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
34m 36s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
3m 22s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

216m  7s

Subsystem
Report/Notes

Docker
ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle markdownlint

uname
Linux b143260b31c9 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 7b52819577b91f03f5e943f0d07912841605c1e4

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/2/testReport/

Max. process+thread count
554 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

FYI, Jira on create performance. I'm worrying about how to remove/delay client configuration, so adding a new client makes things way worse: HADOOP-19205

FYI, Jira on create performance. I'm worrying about how to remove/delay client configuration, so adding a new client makes things way worse: HADOOP-19205

by default we are not initializing new s3 client. It is done under a config to provide backward compatibility only when required.

It's still happening in FileSystem.initialize().
If I get time this week I'll do my PoC of lazy creation of transfer manager and async client, as that's doubled startup time already. all of that will be moved behind S3Store with only the copy() methods exposed.
Ultimately I want the S3Client itself hidden behind that API, so here

new unencryptedS3 operations getLengthOfEncryptedObject(path), ..., would trigger demand creation of the method
accessor to this passed down.

Actually, createFileStatusFromListingEntry() could be part of S3AStore took, somehow. all the config information for the creation, especially cSE flags, would be in the store config, so listing wouldn't need it

#6892 is the draft pr with lazy creation; a new s3 client would go in there, but ideally we'd actually push the listing/transformation code into S3AStore so the extra client would never be visible outside it.

#6892 is the draft pr with lazy creation; a new s3 client would go in there, but ideally we'd actually push the listing/transformation code into S3AStore so the extra client would never be visible outside it.

@steveloughran - I really like the idea of lazy initializing async client and new interface for creating s3 client. I will wait for your changes to get in and refactor my code based on it. By this way i could lazy initialize unencryted s3 client as well when the config is enabled.

exactly. when its not needed: no penalty.

@steveloughran @ahmarsuhail  I have rebased the changes on top of https://github.com/apache/hadoop/pull/6892 .
Please review the changes
Thanks

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 53s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 5 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
44m 19s

trunk passed

+1 :green_heart:
compile
0m 44s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
0m 36s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
0m 33s

trunk passed

+1 :green_heart:
mvnsite
0m 43s

trunk passed

+1 :green_heart:
javadoc
0m 29s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
0m 36s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
spotbugs
1m 10s

trunk passed

+1 :green_heart:
shadedclient
33m 51s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

-1 :x:
mvninstall
0m 13s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
0m 12s
/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt
hadoop-aws in the patch failed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.

-1 :x:
javac
0m 12s
/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt
hadoop-aws in the patch failed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.

-1 :x:
compile
0m 12s
/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08.

-1 :x:
javac
0m 12s
/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 11s
/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt
The patch fails to run checkstyle in hadoop-aws

-1 :x:
mvnsite
0m 12s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
javadoc
0m 13s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt
hadoop-aws in the patch failed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.

-1 :x:
javadoc
0m 12s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08.

-1 :x:
spotbugs
0m 13s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
shadedclient
3m  2s

patch has errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 13s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+0 :ok:
asflicense
0m 14s

ASF License check generated no output?

88m 53s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle markdownlint

uname
Linux 80be02ce71a3 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 6464730deef714895b3c23314033e4a117a4e455

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/3/testReport/

Max. process+thread count
693 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 50s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 5 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 48s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
31m 45s

trunk passed

+1 :green_heart:
compile
17m 29s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
16m  8s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
4m 24s

trunk passed

+1 :green_heart:
mvnsite
1m 39s

trunk passed

+1 :green_heart:
javadoc
1m 22s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
1m 30s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+0 :ok:
spotbugs
0m 46s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 26s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
1m  7s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
0m 45s

the patch passed

+1 :green_heart:
compile
16m 45s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
16m 45s

the patch passed

+1 :green_heart:
compile
16m  4s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
16m  4s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 25s
/results-checkstyle-root.txt
root: The patch generated 3 new + 23 unchanged - 1 fixed = 26 total (was 24)

+1 :green_heart:
mvnsite
1m 37s

the patch passed

+1 :green_heart:
javadoc
1m 19s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

-1 :x:
javadoc
0m 50s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08 with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+0 :ok:
spotbugs
0m 40s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
34m 33s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
2m 56s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  5s

The patch does not generate ASF License warnings.

215m 30s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle markdownlint

uname
Linux 8e49038bce21 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / bc9cfbd09a0823adc06e360c6abafef8d3e53a59

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/5/testReport/

Max. process+thread count
668 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/5/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran I really appreciate your time and effort to review this changes. Indeed it was a detailed review.
I have tried to address all your comments and raised a new revision (I hope i have covered all your comments).
Please take a look
Thanks

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
16m 58s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 10 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 46s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 49s

trunk passed

+1 :green_heart:
compile
17m 53s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
16m  9s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
4m 22s

trunk passed

+1 :green_heart:
mvnsite
3m 22s

trunk passed

+1 :green_heart:
javadoc
2m 32s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 23s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+0 :ok:
spotbugs
0m 44s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 30s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
34m 57s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
1m  5s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 46s

the patch passed

+1 :green_heart:
compile
16m 48s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
16m 48s

the patch passed

+1 :green_heart:
compile
16m 20s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
16m 20s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 30s
/results-checkstyle-root.txt
root: The patch generated 5 new + 26 unchanged - 0 fixed = 31 total (was 26)

+1 :green_heart:
mvnsite
3m 19s

the patch passed

-1 :x:
javadoc
1m 14s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.txt
hadoop-common in the patch failed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2.

-1 :x:
javadoc
0m 50s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_412-8u412-ga-1~20.04.1-b08 with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)

+0 :ok:
spotbugs
0m 39s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
34m 33s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 35s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 57s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  5s

The patch does not generate ASF License warnings.

266m 58s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux e72d48ad2ace 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b2207a4d18d4eca008fe407dd1f89b1d728ca4cc

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/6/testReport/

Max. process+thread count
1252 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/6/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 52s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 10 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 21s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 11s

trunk passed

+1 :green_heart:
compile
17m 24s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
compile
16m 17s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
checkstyle
4m 19s

trunk passed

+1 :green_heart:
mvnsite
3m 22s

trunk passed

+1 :green_heart:
javadoc
2m 38s

trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 23s

trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+0 :ok:
spotbugs
0m 44s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 35s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
35m  2s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
1m  5s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 44s

the patch passed

+1 :green_heart:
compile
16m 49s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javac
16m 49s

the patch passed

+1 :green_heart:
compile
16m 10s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+1 :green_heart:
javac
16m 10s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 22s
/results-checkstyle-root.txt
root: The patch generated 2 new + 26 unchanged - 0 fixed = 28 total (was 26)

+1 :green_heart:
mvnsite
3m 22s

the patch passed

+1 :green_heart:
javadoc
2m 32s

the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2

+1 :green_heart:
javadoc
2m 24s

the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

+0 :ok:
spotbugs
0m 40s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
35m  3s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 39s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 55s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

250m  8s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux f618fc5af4b6 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3b31b5a8df9d567e9dc0caac3afa5ae7bbb3620e

Default Java
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/7/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/7/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@shameersss1 unless you have no choice, please try not to rebase and force push prs once reviews have started. makes it a lot harder to see what has changed

@shameersss1 unless you have no choice, please try not to rebase and force push prs once reviews have started. makes it a lot harder to see what has changed

Yeah, I had to rebase for merge conflicts in hadoop-common module. This commit (https://github.com/apache/hadoop/pull/6884/commits/838dc24c88cb1d20503c19f99c2886bcbb2c1e25) contains the addressing of review comments.

@steveloughran - Gentle reminder for the review
Thanks

@shameersss1 what do you think of @raphaelazzolini 's comment

@shameersss1 what do you think of @raphaelazzolini 's comment

I understand  @raphaelazzolini 's concern of having many isCSEEnabled conditions especially in the listing methods. If i understand it correctly the ask is to introduce to handler interface pattern as follows
`interface ListingHandler {
Listing getListing(Path path, boolean includeSelf);
}
class CSEEnabledListingHandler implements ListingHandler {
@Override
public Listing getListing(Path path, boolean includeSelf) {
return includeSelf
? new Listing.AcceptAllButS3nDirsAndCSEInstructionFile()
: new Listing.AcceptAllButSelfAndS3nDirsAndCSEInstructionFile(path);
}
}
class CSEDisabledListingHandler implements ListingHandler {
@Override
public Listing getListing(Path path, boolean includeSelf) {
return includeSelf
? new Listing.AcceptAllButS3nDirs()
: new Listing.AcceptAllButSelfAndS3nDirs(path);
}
}
@Retries.RetryTranslated
public RemoteIterator listFilesAndDirectoryMarkers(
final Path path,
final S3AFileStatus status,
final boolean includeSelf) throws IOException {
auditSpan.activate();
ListingHandler handler = isCSEEnabled ? new CSEEnabledListingHandler() : new CSEDisabledListingHandler();
return innerListFiles(path, true, handler.getListing(path, includeSelf), status);
}
`
I agree handler interface strategy will be more elegant. I will make the required code changes. I see there are conflicting changes which requires rebase and force push

@steveloughran  - I have raised a new revision. Please do review the same.
Thanks

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
17m  3s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 10 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 21s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 35s

trunk passed

+1 :green_heart:
compile
17m 32s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m  1s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 30s

trunk passed

+1 :green_heart:
mvnsite
3m 23s

trunk passed

+1 :green_heart:
javadoc
2m 35s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 22s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 44s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 46s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
35m 12s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
1m  7s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 45s

the patch passed

+1 :green_heart:
compile
16m 48s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 48s

the patch passed

+1 :green_heart:
compile
16m 21s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m 21s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 21s
/results-checkstyle-root.txt
root: The patch generated 2 new + 26 unchanged - 0 fixed = 28 total (was 26)

+1 :green_heart:
mvnsite
3m 23s

the patch passed

+1 :green_heart:
javadoc
2m 31s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 21s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 38s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
35m  4s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 37s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 58s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  6s

The patch does not generate ASF License warnings.

267m  6s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/8/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux c24e8baf0f29 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 069ec479c1aaa76f99acc99eeca572990d1da3ba

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/8/testReport/

Max. process+thread count
2742 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/8/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran  and @ahmarsuhail  - I have addressed your comments. Could you please re-review ?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 18 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
0m 21s

Maven dependency ordering for branch

-1 :x:
mvninstall
0m 23s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
0m 23s
/branch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
compile
0m 23s
/branch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in trunk failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-0 :warning:
checkstyle
0m 21s
/buildtool-branch-checkstyle-root.txt
The patch fails to run checkstyle in root

-1 :x:
mvnsite
0m 23s
/branch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

-1 :x:
mvnsite
0m 15s
/branch-mvnsite-hadoop-project.txt
hadoop-project in trunk failed.

-1 :x:
mvnsite
0m 23s
/branch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in trunk failed.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-common in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-project-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-project in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-aws in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-common in trunk failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-project-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-project in trunk failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javadoc
0m 24s
/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-aws in trunk failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
spotbugs
0m 23s
/branch-spotbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

-1 :x:
spotbugs
0m 23s
/branch-spotbugs-hadoop-project.txt
hadoop-project in trunk failed.

-1 :x:
spotbugs
0m 23s
/branch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in trunk failed.

+1 :green_heart:
shadedclient
5m 41s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
6m  5s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 21s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvninstall
0m 24s
/patch-mvninstall-hadoop-project.txt
hadoop-project in the patch failed.

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
0m 23s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 23s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
compile
0m 23s
/patch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javac
0m 23s
/patch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
0m 20s
/buildtool-patch-checkstyle-root.txt
The patch fails to run checkstyle in root

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-project.txt
hadoop-project in the patch failed.

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
javadoc
0m 24s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-common in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-project-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-project in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-aws in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-common in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javadoc
0m 47s
/patch-javadoc-hadoop-project-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-project in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javadoc
0m 24s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
spotbugs
0m 23s
/patch-spotbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
spotbugs
0m 24s
/patch-spotbugs-hadoop-project.txt
hadoop-project in the patch failed.

-1 :x:
spotbugs
0m 24s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
shadedclient
8m  2s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 24s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
unit
0m 23s
/patch-unit-hadoop-project.txt
hadoop-project in the patch failed.

-1 :x:
unit
0m 13s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 37s

The patch does not generate ASF License warnings.

22m 26s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/9/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 128ede2d01d2 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 613bc4da833bbe2cbe17e20bef36286932b3ba4e

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/9/testReport/

Max. process+thread count
47 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-project hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/9/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 29s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 18 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 31s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
36m 11s

trunk passed

+1 :green_heart:
compile
17m 53s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 10s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 16s

trunk passed

+1 :green_heart:
mvnsite
3m 25s

trunk passed

+1 :green_heart:
javadoc
2m 40s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 26s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 46s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 23s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
34m 50s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 38s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 44s

the patch passed

+1 :green_heart:
compile
16m 47s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 47s

the patch passed

+1 :green_heart:
compile
16m  6s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m  6s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m 21s
/results-checkstyle-root.txt
root: The patch generated 5 new + 31 unchanged - 2 fixed = 36 total (was 33)

+1 :green_heart:
mvnsite
3m 20s

the patch passed

+1 :green_heart:
javadoc
2m 32s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

-1 :x:
javadoc
0m 50s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

+0 :ok:
spotbugs
0m 38s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
34m 32s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 35s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 34s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 57s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  5s

The patch does not generate ASF License warnings.

252m 38s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/10/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux bfdc5a0a8e2f 5.15.0-113-generic #123-Ubuntu SMP Mon Jun 10 08:16:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4ed70260ed1d5fdde7220d14659ec70fdbb68f96

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/10/testReport/

Max. process+thread count
2406 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/10/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 59s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
15m 10s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
33m 20s

trunk passed

+1 :green_heart:
compile
18m 38s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 34s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 37s

trunk passed

+1 :green_heart:
mvnsite
3m  9s

trunk passed

+1 :green_heart:
javadoc
2m 22s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 10s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 39s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
39m 39s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
40m  6s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 39s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 54s

the patch passed

+1 :green_heart:
compile
19m 59s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
19m 59s

the patch passed

+1 :green_heart:
compile
18m 13s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
18m 13s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 51s

root: The patch generated 0 new + 32 unchanged - 1 fixed = 32 total (was 33)

+1 :green_heart:
mvnsite
3m 20s

the patch passed

+1 :green_heart:
javadoc
2m 23s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

-1 :x:
javadoc
0m 50s
/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05 with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+0 :ok:
spotbugs
0m 35s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
40m  4s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 32s

hadoop-project in the patch passed.

+1 :green_heart:
unit
20m 15s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 55s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  7s

The patch does not generate ASF License warnings.

269m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/11/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 6c080fda69a6 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 737c0716a412a333335843a1c36e542613b6c2cc

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/11/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/11/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 56s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 34s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 38s

trunk passed

+1 :green_heart:
compile
17m 26s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m  9s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 21s

trunk passed

+1 :green_heart:
mvnsite
3m 25s

trunk passed

+1 :green_heart:
javadoc
2m 37s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 24s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 44s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 39s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
35m  6s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 39s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 44s

the patch passed

+1 :green_heart:
compile
16m 57s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 57s

the patch passed

+1 :green_heart:
compile
16m 19s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m 19s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 18s

root: The patch generated 0 new + 32 unchanged - 1 fixed = 32 total (was 33)

+1 :green_heart:
mvnsite
3m 20s

the patch passed

+1 :green_heart:
javadoc
2m 29s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 22s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 40s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
34m 42s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 37s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 40s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 54s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  6s

The patch does not generate ASF License warnings.

250m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/12/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 8f546c237b45 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4316f5372e374e38fc9c3b2a02def553d8ef82b6

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/12/testReport/

Max. process+thread count
1905 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/12/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran  @ahmarsuhail  request for rereview
Thanks

@ahmarsuhail Thanks a lot for your review. I have addressed the additional/minor comments.
@steveloughran  - Can you also please take a look ?

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 19s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 24s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 26s

trunk passed

+1 :green_heart:
compile
17m 39s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 16s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 25s

trunk passed

+1 :green_heart:
mvnsite
3m 23s

trunk passed

+1 :green_heart:
javadoc
2m 36s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 23s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 44s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
34m 50s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
35m 16s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 39s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 44s

the patch passed

+1 :green_heart:
compile
16m 53s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 53s

the patch passed

+1 :green_heart:
compile
16m  0s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m  0s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 23s

root: The patch generated 0 new + 32 unchanged - 1 fixed = 32 total (was 33)

+1 :green_heart:
mvnsite
3m 19s

the patch passed

+1 :green_heart:
javadoc
2m 35s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 23s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 39s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
34m 50s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 55s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 53s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m 10s

The patch does not generate ASF License warnings.

250m 45s

Subsystem
Report/Notes

Docker
ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/13/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 04175bacbfa1 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 93b6f7a8f11fd15b9ba89b7ca209dc5ec1861e74

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/13/testReport/

Max. process+thread count
3066 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/13/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran  - Are we good to take this forward ? Please let me know if there is anything else to address

@steveloughran  - Gentle reminder for review

@steveloughran  - Gentle reminder for review

@shameersss1 you have to fix the conficts first
i am distracted right now by SDK regressions; not looking at anything else

@steveloughran  - I have resolved the merge conflict and force pushed.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m  1s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 40s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
33m  3s

trunk passed

+1 :green_heart:
compile
17m 35s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 20s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 26s

trunk passed

+1 :green_heart:
mvnsite
3m 27s

trunk passed

+1 :green_heart:
javadoc
2m 52s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 25s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 45s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
35m 27s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 38s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 46s

the patch passed

+1 :green_heart:
compile
17m  3s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
17m  3s

the patch passed

+1 :green_heart:
compile
16m 24s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m 24s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m 21s
/results-checkstyle-root.txt
root: The patch generated 1 new + 33 unchanged - 1 fixed = 34 total (was 34)

+1 :green_heart:
mvnsite
3m 21s

the patch passed

+1 :green_heart:
javadoc
2m 49s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 26s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 40s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
35m 42s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 42s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 58s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

254m 23s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/14/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 11f94ee58154 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 1b9c497d5af51e9bb11e350757f5dc1023aa741a

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/14/testReport/

Max. process+thread count
1252 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/14/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 54s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 58s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
33m 39s

trunk passed

+1 :green_heart:
compile
17m 24s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 21s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 27s

trunk passed

+1 :green_heart:
mvnsite
3m 24s

trunk passed

+1 :green_heart:
javadoc
2m 54s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 25s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 45s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
36m 46s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 39s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 45s

the patch passed

+1 :green_heart:
compile
17m 24s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
17m 24s

the patch passed

+1 :green_heart:
compile
16m 25s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 21s

root: The patch generated 0 new + 33 unchanged - 1 fixed = 33 total (was 34)

+1 :green_heart:
mvnsite
3m 23s

the patch passed

+1 :green_heart:
javadoc
2m 15s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 15s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 35s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
35m 36s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 39s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 41s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 59s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  7s

The patch does not generate ASF License warnings.

255m 33s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/15/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux d2ab10820633 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 32169e07b7122907e9b4f78912ddc17161e8ed87

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/15/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/15/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran  - Due to changes spanning accross 50 files, This PR is prone to merge conflicts very often if left opened for a while. Could you please review this ?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
22m 32s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
0m 20s

Maven dependency ordering for branch

-1 :x:
mvninstall
0m 23s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
0m 24s
/branch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
compile
0m 23s
/branch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in trunk failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-0 :warning:
checkstyle
0m 21s
/buildtool-branch-checkstyle-root.txt
The patch fails to run checkstyle in root

-1 :x:
mvnsite
0m 24s
/branch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

-1 :x:
mvnsite
0m 23s
/branch-mvnsite-hadoop-project.txt
hadoop-project in trunk failed.

-1 :x:
mvnsite
0m 24s
/branch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in trunk failed.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-common in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-project-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-project in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 24s
/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-aws in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 24s
/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-common in trunk failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javadoc
0m 23s
/branch-javadoc-hadoop-project-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-project in trunk failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
spotbugs
0m 15s
/branch-spotbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

+0 :ok:
spotbugs
0m 28s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

-1 :x:
shadedclient
1m 57s

branch has errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 21s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-project.txt
hadoop-project in the patch failed.

-1 :x:
mvninstall
0m 23s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
compile
0m 23s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javac
0m 23s
/patch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
compile
0m 22s
/patch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javac
0m 22s
/patch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 21s
/buildtool-patch-checkstyle-root.txt
The patch fails to run checkstyle in root

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvnsite
0m 24s
/patch-mvnsite-hadoop-project.txt
hadoop-project in the patch failed.

-1 :x:
mvnsite
0m 23s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-common in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 24s
/patch-javadoc-hadoop-project-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-project in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
hadoop-aws in the patch failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
javadoc
0m 41s
/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-common in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javadoc
0m 23s
/patch-javadoc-hadoop-project-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-project in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
javadoc
0m 24s
/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
hadoop-aws in the patch failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-1 :x:
spotbugs
0m 24s
/patch-spotbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
spotbugs
0m 23s
/patch-spotbugs-hadoop-project.txt
hadoop-project in the patch failed.

-1 :x:
spotbugs
0m 23s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
shadedclient
7m 55s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
0m 23s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
unit
0m 23s
/patch-unit-hadoop-project.txt
hadoop-project in the patch failed.

-1 :x:
unit
0m 23s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+0 :ok:
asflicense
0m 24s

ASF License check generated no output?

49m 26s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/16/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 3fe10281ab3b 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / ca4e35ec93de36871ef743d141f85d4d36e7ae2e

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/16/testReport/

Max. process+thread count
88 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-project hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/16/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
1m 10s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
0m 20s

Maven dependency ordering for branch

-1 :x:
mvninstall
0m 23s
/branch-mvninstall-root.txt
root in trunk failed.

-1 :x:
compile
0m 23s
/branch-compile-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root in trunk failed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.

-1 :x:
compile
0m 40s
/branch-compile-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root in trunk failed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05.

-0 :warning:
checkstyle
0m 24s
/buildtool-branch-checkstyle-root.txt
The patch fails to run checkstyle in root

-1 :x:
mvnsite
3m 16s
/branch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

+1 :green_heart:
javadoc
1m 53s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
1m 16s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

-1 :x:
spotbugs
0m 13s
/branch-spotbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in trunk failed.

+0 :ok:
spotbugs
0m 24s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
38m 44s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
6m 25s

Maven dependency ordering for patch

-1 :x:
mvninstall
0m 14s
/patch-mvninstall-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvninstall
0m 20s
/patch-mvninstall-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
compile
17m 38s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

-1 :x:
javac
17m 38s
/results-compile-javac-root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04.txt
root-jdkUbuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 generated 35 new + 0 unchanged - 0 fixed = 35 total (was 0)

+1 :green_heart:
compile
16m 29s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

-1 :x:
javac
16m 29s
/results-compile-javac-root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05.txt
root-jdkPrivateBuild-1.8.0_422-8u422-b05-1~20.04-b05 with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 generated 44 new + 0 unchanged - 0 fixed = 44 total (was 0)

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 19s
/results-checkstyle-root.txt
root: The patch generated 34 new + 0 unchanged - 0 fixed = 34 total (was 0)

-1 :x:
mvnsite
0m 43s
/patch-mvnsite-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
mvnsite
0m 47s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
javadoc
2m 44s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 23s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 38s

hadoop-project has no data from spotbugs

-1 :x:
spotbugs
0m 34s
/patch-spotbugs-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
spotbugs
0m 43s
/patch-spotbugs-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
shadedclient
34m 43s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 39s

hadoop-project in the patch passed.

-1 :x:
unit
0m 37s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch failed.

-1 :x:
unit
0m 37s
/patch-unit-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
1m  6s

The patch does not generate ASF License warnings.

150m 26s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/17/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux ade6b4e278fa 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 37eead58f3d4350a560b3ef01c1e629ecd301bb7

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/17/testReport/

Max. process+thread count
551 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/17/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

I will do this and other pending stuff once we've got 3.4.1 out the door -that is the only thing I'm focused on right now. The best thing you could do right now is help qualify that in your environments.

@steveloughran  Sure i would gladly be part of hadoop-3.4.1 release. Is there any specific thing you want me do? Should i run all the S3A integration tests for the 3.4.1 release candidate branch ? If you could you please point me to the same and the place where i should update the results.

@shameersss1 I'll send you the steps on how to do this

@ahmarsuhail  - Thanks Ahmar for the steps, I have signed off on ARM64 EC2 machine.

@steveloughran   - Now that Hadoop 3.4.1 is released, Gentle reminder for review

Thinking about this some more: those .instruction files raise a lot of problems; I don't like the way t
There's already the notion that files beginning with . or _ are hidden, applications scanning directo
So do we really need to hide them at all? Making them visible stops us playing yet more tricks with the
What is actually in the files? Is it just some list of attributes encrypted with the same CSE settings?

Thinking about this some more: those .instruction files raise a lot of problems; I don't like the way t
There's already the notion that files beginning with . or _ are hidden, applications scanning directo
So do we really need to hide them at all? Making them visible stops us playing yet more tricks with the
What is actually in the files? Is it just some list of attributes encrypted with the same CSE settings?

Instruction file contains metadata about the encryption and some headers.
If a file named "abc" is written to s3 with CSE V1 with Instruction mode, It will store two files, "abc"  and "abc.instruction"
So we are ultimately skipping all files ending  .instruction when that flag is enabled.

Force pushed to rebase with the trunk.
@steveloughran  - Thanks a lot for a detailed review. I know it is a pain to review changes which are big (~50 files). I really appreciate the time you put into this.
I have addressed your comments with the new commit
PS: Ignored instruction file handling for simplicilty

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
18m  8s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 41s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
34m 38s

trunk passed

+1 :green_heart:
compile
17m 26s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
15m 59s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 19s

trunk passed

+1 :green_heart:
mvnsite
3m 24s

trunk passed

+1 :green_heart:
javadoc
2m 51s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 24s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 45s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
35m 53s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 39s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 44s

the patch passed

+1 :green_heart:
compile
16m 39s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 39s

the patch passed

+1 :green_heart:
compile
16m  7s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m  7s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 18 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
4m 15s
/results-checkstyle-root.txt
root: The patch generated 2 new + 34 unchanged - 0 fixed = 36 total (was 34)

-1 :x:
mvnsite
0m 55s
/patch-mvnsite-hadoop-tools_hadoop-aws.txt
hadoop-aws in the patch failed.

+1 :green_heart:
javadoc
2m 45s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 26s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 40s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
35m 50s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 39s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m  0s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  5s

The patch does not generate ASF License warnings.

271m 51s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/18/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 93ff913c98eb 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 1deb43a78d7ba5a6b11f2654cfe8092c2bf35faa

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/18/testReport/

Max. process+thread count
1252 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/18/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 52s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 58s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 26s

trunk passed

+1 :green_heart:
compile
17m 19s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 10s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 20s

trunk passed

+1 :green_heart:
mvnsite
3m 27s

trunk passed

+1 :green_heart:
javadoc
2m 49s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 18s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 45s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
35m 13s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 39s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 43s

the patch passed

+1 :green_heart:
compile
16m 40s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 40s

the patch passed

+1 :green_heart:
compile
16m 13s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m 13s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
checkstyle
4m 19s

the patch passed

+1 :green_heart:
mvnsite
3m 21s

the patch passed

+1 :green_heart:
javadoc
2m 48s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 25s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 40s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
35m 41s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 37s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 46s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m  0s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  6s

The patch does not generate ASF License warnings.

252m  7s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/19/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux a49bc6f9a380 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b1969658f4383b62cb0d4ec303580c0620a24fb6

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/19/testReport/

Max. process+thread count
1899 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/19/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 53s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
14m 22s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
31m 23s

trunk passed

+1 :green_heart:
compile
17m 15s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m  2s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 20s

trunk passed

+1 :green_heart:
mvnsite
3m 22s

trunk passed

+1 :green_heart:
javadoc
2m 54s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 25s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 46s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
35m 41s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 38s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 43s

the patch passed

+1 :green_heart:
compile
16m 39s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 39s

the patch passed

+1 :green_heart:
compile
16m  2s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m  2s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
4m 16s

the patch passed

+1 :green_heart:
mvnsite
3m 18s

the patch passed

+1 :green_heart:
javadoc
2m 46s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 26s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 40s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
35m 17s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 40s

hadoop-project in the patch passed.

+1 :green_heart:
unit
19m 43s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m  0s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  4s

The patch does not generate ASF License warnings.

250m 12s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/20/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux c533c8b57a58 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 59a7e05efb8e303084eef8022a1fea9dcfb941c2

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/20/testReport/

Max. process+thread count
1252 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/20/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

let's make this the next big change, with everything else blocked before it goes in. That way, no more merge pain

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
17m 29s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
xmllint
0m  0s

xmllint was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m  5s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
32m 32s

trunk passed

+1 :green_heart:
compile
17m 23s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 19s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 22s

trunk passed

+1 :green_heart:
mvnsite
3m 20s

trunk passed

+1 :green_heart:
javadoc
2m 53s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 26s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 46s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
36m 15s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
36m 42s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 40s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 45s

the patch passed

+1 :green_heart:
compile
16m 40s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 40s

the patch passed

+1 :green_heart:
compile
16m 10s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m 10s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 15s
/results-checkstyle-root.txt
root: The patch generated 1 new + 34 unchanged - 1 fixed = 35 total (was 35)

+1 :green_heart:
mvnsite
3m 19s

the patch passed

+1 :green_heart:
javadoc
2m 48s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 25s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 39s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
37m 18s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
21m 13s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m  7s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  5s

The patch does not generate ASF License warnings.

274m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/21/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 6dca9c8ecbd5 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 2f2790f5d2f3ae18ad32f991a6f549eefd1dda79

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/21/testReport/

Max. process+thread count
1252 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/21/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran  - Thanks a lot for the review. I have addressed your comment.

Yeah, Error translation feature make sense now. I have addressed your comments.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 55s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+0 :ok:
xmllint
0m  1s

xmllint was not available.

+0 :ok:
markdownlint
0m  1s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 17 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
16m 19s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
31m 43s

trunk passed

+1 :green_heart:
compile
17m 22s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
compile
16m 13s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
checkstyle
4m 28s

trunk passed

+1 :green_heart:
mvnsite
3m 21s

trunk passed

+1 :green_heart:
javadoc
2m 53s

trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 25s

trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 44s

branch/hadoop-project no spotbugs output file (spotbugsXml.xml)

+1 :green_heart:
shadedclient
35m 46s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
36m 14s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 41s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 45s

the patch passed

+1 :green_heart:
compile
16m 42s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javac
16m 42s

the patch passed

+1 :green_heart:
compile
16m 16s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+1 :green_heart:
javac
16m 16s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
4m 22s
/results-checkstyle-root.txt
root: The patch generated 1 new + 34 unchanged - 1 fixed = 35 total (was 35)

+1 :green_heart:
mvnsite
3m 20s

the patch passed

+1 :green_heart:
javadoc
2m 44s

the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04

+1 :green_heart:
javadoc
2m 25s

the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05

+0 :ok:
spotbugs
0m 40s

hadoop-project has no data from spotbugs

+1 :green_heart:
shadedclient
36m  4s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 38s

hadoop-project in the patch passed.

+1 :green_heart:
unit
20m 43s

hadoop-common in the patch passed.

+1 :green_heart:
unit
3m  7s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
1m  7s

The patch does not generate ASF License warnings.

255m 11s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/22/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6884

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint markdownlint

uname
Linux 13650f953a63 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / d4397bf7a5ef847602ca39776b5957e7daeb883c

Default Java
Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/22/testReport/

Max. process+thread count
3154 (vs. ulimit of 5500)

modules
C: hadoop-project hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6884/22/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran  - Gentle reminder for review.

@steveloughran  - Gentle reminder for review.

ok, merged. provide a 3.4 version and I'll pull that in too.

Sure, will work on branch-3.4 after HADOOP-19336