8000 HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider. by simbadzina · Pull Request #5142 · apache/hadoop · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider. #5142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Nov 29, 2022
Merged

HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider. #5142

merged 4 commits into from
Nov 29, 2022

Conversation

simbadzina
Copy link
Member
@simbadzina simbadzina commented Nov 17, 2022 •
edited
Loading

HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider.

Description of PR

Description
In order for clients to have routers forward their reads to observers, the clients must use a proxy with an alignment context. This is currently achieved by using the ObserverReadProxyProvider.

Using ObserverReadProxyProvider allows backward compatible for client configurations.
However, the ObserverReadProxyProvider forces an msync on initialization which is not required with routers.

Performing msync calls is more expensive with routers because the router fans out the cal to all namespaces, so we'd like to avoid this when possible since the router will proxy the initial client call without federatedstate to the Active.

How was this patch tested?

New test cases in TestObserverWithRouter.

For code changes:

  • [ x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsys 8000 tem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 15m 56s Maven dependency ordering for branch
+1 💚 mvninstall 27m 19s trunk passed
+1 💚 compile 6m 31s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 compile 6m 5s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 31s trunk passed
+1 💚 mvnsite 3m 59s trunk passed
+1 💚 javadoc 3m 1s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 41s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 7m 54s trunk passed
+1 💚 shadedclient 21m 50s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 33s Maven dependency ordering for patch
+1 💚 mvninstall 2m 51s the patch passed
+1 💚 compile 5m 58s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javac 5m 58s the patch passed
+1 💚 compile 5m 57s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 javac 5m 57s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 15s the patch passed
+1 💚 mvnsite 3m 10s the patch passed
+1 💚 javadoc 2m 22s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 12s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 7m 54s the patch passed
+1 💚 shadedclient 22m 29s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 41s hadoop-hdfs-client in the patch passed.
+1 💚 unit 250m 31s hadoop-hdfs in the patch passed.
+1 💚 unit 23m 5s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 13s The patch does not generate ASF License warnings.
434m 38s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/1/artifact/out/Dockerfile
GITHUB PR #5142
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux 145b5c8046bf 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 668d5d2
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/1/testReport/
Max. process+thread count 3386 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@simbadzina
Copy link
Member Author

@ZanderXu @tomscut could you please take a look whenever you get a chance.

@ZanderXu
Copy link
Contributor

@ZanderXu @tomscut could you please take a look whenever you get a chance.

@simbadzina I did a quick review. The change looks good to me. I am curious about that what scenarios will use this feature without msyc(). Can you share me some cases?

@simbadzina
Copy link
Member Author
simbadzina commented Nov 18, 2022 •
edited
Loading

@ZanderXu @tomscut could you please take a look whenever you get a chance.

@simbadzina I did a quick review. The change looks good to me. I am curious about that what scenarios will use this feature without msyc(). Can you share me some cases?

Thanks for the quick review @ZanderXu . Third party readers issuing multiple reads will use msync if they want consistency, both before and after this path. The msync this flag eliminates is the one at the initialization of the ObserverReadProxyProvider here
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/ObserverReadProxyProvider.java#L341-L348

Taking an example of a client just doing a list status.

If the client is using ObserverReadProxyProvider. They'll be two calls.

  1. msync (during initialization)
  2. getListing (actual call the client wants to do)
    When the router gets the msync, it will fan this out to all namespaces, which is expensive.
    The getLIsting will then go to the observer.

With this patch, they'll only be one call.

  1. getListing
    The router will send this call to the active.

For future reads after this, there is no difference. These will go to the observer.

@mkuchenbecker
Copy link

Mostly mechanical review, with some questions.

mkuchenbecker
mkuchenbecker reviewed Nov 21, 2022
View reviewed changes
Copy link
@mkuchenbecker mkuchenbecker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly mechanical review, some questions.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 42s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 15m 40s Maven dependency ordering for branch
-1 ❌ mvninstall 23m 21s /branch-mvninstall-root.txt root in trunk failed.
-1 ❌ compile 1m 45s /branch-compile-hadoop-hdfs-project-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt hadoop-hdfs-project in trunk failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.
-1 ❌ compile 0m 52s /branch-compile-hadoop-hdfs-project-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt hadoop-hdfs-project in trunk failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.
+1 💚 checkstyle 1m 52s trunk passed
+1 💚 mvnsite 4m 17s trunk passed
+1 💚 javadoc 3m 27s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 4m 8s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
-1 ❌ spotbugs 3m 24s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.txt hadoop-hdfs-client in trunk failed.
-1 ❌ spotbugs 0m 47s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in trunk failed.
-1 ❌ spotbugs 0m 48s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in trunk failed.
+1 💚 shadedclient 20m 37s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for patch
-1 ❌ mvninstall 0m 32s /patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
-1 ❌ mvninstall 0m 30s /patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt hadoop-hdfs-client in the patch failed.
-1 ❌ mvninstall 0m 31s /patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch failed.
-1 ❌ compile 0m 32s /patch-compile-hadoop-hdfs-project-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt hadoop-hdfs-project in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.
-1 ❌ javac 0m 32s /patch-compile-hadoop-hdfs-project-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt hadoop-hdfs-project in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.
-1 ❌ compile 0m 30s /patch-compile-hadoop-hdfs-project-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt hadoop-hdfs-project in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.
-1 ❌ javac 0m 30s /patch-compile-hadoop-hdfs-project-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt hadoop-hdfs-project in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 29s /buildtool-patch-checkstyle-hadoop-hdfs-project.txt The patch fails to run checkstyle in hadoop-hdfs-project
-1 ❌ mvnsite 0m 31s /patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
-1 ❌ mvnsite 0m 32s /patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-client.txt hadoop-hdfs-client in the patch failed.
-1 ❌ mvnsite 0m 32s /patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch failed.
-1 ❌ javadoc 0m 32s /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.
-1 ❌ javadoc 0m 31s /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-client-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt hadoop-hdfs-client in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.
-1 ❌ javadoc 0m 31s /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt hadoop-hdfs-rbf in the patch failed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.
-1 ❌ javadoc 0m 32s /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.
-1 ❌ javadoc 0m 31s /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-client-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt hadoop-hdfs-client in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.
-1 ❌ javadoc 0m 30s /patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt hadoop-hdfs-rbf in the patch failed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.
-1 ❌ spotbugs 0m 31s /patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
-1 ❌ spotbugs 0m 32s /patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.txt hadoop-hdfs-client in the patch failed.
-1 ❌ spotbugs 0m 34s /patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch failed.
+1 💚 shadedclient 11m 46s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 0m 31s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
-1 ❌ unit 0m 31s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt hadoop-hdfs-client in the patch failed.
-1 ❌ unit 0m 31s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch failed.
+0 🆗 asflicense 0m 31s ASF License check generated no output?
81m 5s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/2/artifact/out/Dockerfile
GITHUB PR #5142
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux a9ad688627a4 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 95d23f0
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/2/testReport/
Max. process+thread count 101 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/2/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 57s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 18m 12s Maven dependency ordering for branch
+1 💚 mvninstall 27m 32s trunk passed
+1 💚 compile 6m 48s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 compile 7m 14s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 52s trunk passed
+1 💚 mvnsite 3m 50s trunk passed
+1 💚 javadoc 3m 17s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 58s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 8m 23s trunk passed
+1 💚 shadedclient 21m 29s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 29s Maven dependency ordering for patch
+1 💚 mvninstall 2m 58s the patch passed
+1 💚 compile 6m 49s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javac 6m 49s the patch passed
+1 💚 compile 6m 54s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 javac 6m 54s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 19s /results-checkstyle-hadoop-hdfs-project.txt hadoop-hdfs-project: The patch generated 2 new + 33 unchanged - 0 fixed = 35 total (was 33)
+1 💚 mvnsite 3m 38s the patch passed
+1 💚 javadoc 2m 28s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 16s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 8m 0s the patch passed
+1 💚 shadedclient 21m 7s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 30s hadoop-hdfs-client in the patch passed.
-1 ❌ unit 315m 34s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 unit 23m 33s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 10s The patch does not generate ASF License warnings.
506m 5s
Reason Tests
Failed junit tests hadoop.hdfs.TestLeaseRecovery2
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/3/artifact/out/Dockerfile
GITHUB PR #5142
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux 3503756845d9 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4eda670
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/3/testReport/
Max. process+thread count 3352 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 40s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 20m 10s Maven dependency ordering for branch
+1 💚 mvninstall 27m 32s trunk passed
+1 💚 compile 7m 23s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 compile 7m 6s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 35s trunk passed
+1 💚 mvnsite 4m 8s trunk passed
+1 💚 javadoc 3m 11s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 46s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 8m 15s trunk passed
+1 💚 shadedclient 21m 15s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 32s Maven dependency ordering for patch
+1 💚 mvninstall 3m 11s the patch passed
+1 💚 compile 7m 17s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javac 7m 17s the patch passed
+1 💚 compile 6m 46s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 javac 6m 46s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 17s /results-checkstyle-hadoop-hdfs-project.txt hadoop-hdfs-project: The patch generated 2 new + 33 unchanged - 0 fixed = 35 total (was 33)
+1 💚 mvnsite 3m 17s the patch passed
+1 💚 javadoc 2m 23s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 9s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 7m 52s the patch passed
+1 💚 shadedclient 21m 9s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 41s hadoop-hdfs-client in the patch passed.
-1 ❌ unit 304m 38s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 unit 34m 39s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 13s The patch does not generate ASF License warnings.
507m 54s
Reason Tests
Failed junit tests hadoop.hdfs.TestLeaseRecovery2
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/4/artifact/out/Dockerfile
GITHUB PR #5142
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux 15a18a4df778 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4eda670
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/4/testReport/
Max. process+thread count 3309 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

omalley
omalley approved these changes Nov 22, 2022
View reviewed changes
Copy link
Contributor
@omalley omalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 LGTM

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 38s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 15m 44s Maven dependency ordering for branch
+1 💚 mvninstall 26m 10s trunk passed
+1 💚 compile 6m 14s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 compile 5m 54s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 34s trunk passed
+1 💚 mvnsite 3m 50s trunk passed
+1 💚 javadoc 2m 58s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 4m 1s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 9m 6s trunk passed
+1 💚 shadedclient 24m 27s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 31s Maven dependency ordering for patch
+1 💚 mvninstall 2m 54s the patch passed
+1 💚 compile 6m 5s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javac 6m 5s the patch passed
+1 💚 compile 5m 41s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 javac 5m 41s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 15s /results-checkstyle-hadoop-hdfs-project.txt hadoop-hdfs-project: The patch generated 1 new + 33 unchanged - 0 fixed = 34 total (was 33)
+1 💚 mvnsite 3m 8s the patch passed
+1 💚 javadoc 2m 22s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 10s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 7m 27s the patch passed
+1 💚 shadedclient 20m 50s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 32s hadoop-hdfs-client in the patch passed.
-1 ❌ unit 281m 46s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 ❌ unit 23m 17s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 2s The patch does not generate ASF License warnings.
465m 14s
Reason Tests
Failed junit tests hadoop.hdfs.TestLeaseRecovery2
hadoop.hdfs.server.federation.router.TestRouterRPCMultipleDestinationMountTableResolver
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/5/artifact/out/Dockerfile
GITHUB PR #5142
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux 5f7126fa20bb 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 1a6ee21
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/5/testReport/
Max. process+thread count 3138 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/5/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 39s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 15m 57s Maven dependency ordering for branch
+1 💚 mvninstall 26m 6s trunk passed
+1 💚 compile 6m 15s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 compile 5m 54s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 24s trunk passed
+1 💚 mvnsite 3m 47s trunk passed
+1 💚 javadoc 3m 0s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 41s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 7m 38s trunk passed
+1 💚 shadedclient 21m 1s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 32s Maven dependency ordering for patch
+1 💚 mvninstall 2m 54s the patch passed
+1 💚 compile 6m 0s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javac 6m 0s the patch passed
+1 💚 compile 5m 44s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 javac 5m 44s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 14s /results-checkstyle-hadoop-hdfs-project.txt hadoop-hdfs-project: The patch generated 1 new + 33 unchanged - 0 fixed = 34 total (was 33)
+1 💚 mvnsite 3m 5s the patch passed
+1 💚 javadoc 2m 10s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 5s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 7m 31s the patch passed
+1 💚 shadedclient 20m 51s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 41s hadoop-hdfs-client in the patch passed.
-1 ❌ unit 306m 17s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 unit 23m 5s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 18s The patch does not generate ASF License warnings.
484m 37s
Reason Tests
Failed junit tests hadoop.hdfs.TestLeaseRecovery2
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/6/artifact/out/Dockerfile
GITHUB PR #5142
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux 164b9b4c8623 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 994ab62
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/6/testReport/
Max. process+thread count 2927 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/6/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 42s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 15m 54s Maven dependency ordering for branch
+1 💚 mvninstall 26m 11s trunk passed
+1 💚 compile 6m 21s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 compile 6m 5s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 28s trunk passed
+1 💚 mvnsite 3m 57s trunk passed
+1 💚 javadoc 3m 1s trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 40s trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 7m 39s trunk passed
+1 💚 shadedclient 21m 13s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for patch
+1 💚 mvninstall 2m 54s the patch passed
+1 💚 compile 6m 4s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javac 6m 4s the patch passed
+1 💚 compile 5m 51s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 javac 5m 51s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 12s the patch passed
+1 💚 mvnsite 3m 17s the patch passed
+1 💚 javadoc 2m 11s the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 3m 15s the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 7m 30s the patch passed
+1 💚 shadedclient 21m 9s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 33s hadoop-hdfs-client in the patch passed.
-1 ❌ unit 301m 28s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 ❌ unit 23m 4s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 1m 4s The patch does not generate ASF License warnings.
481m 4s
Reason Tests
Failed junit tests hadoop.hdfs.TestLeaseRecovery2
hadoop.hdfs.server.ba A93C lancer.TestBalancerService
hadoop.hdfs.server.federation.router.TestRouterRPCMultipleDestinationMountTableResolver
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/7/artifact/out/Dockerfile
GITHUB PR #5142
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux a9b06287bb2f 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4d19c51
Default Java Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/7/testReport/
Max. process+thread count 2965 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5142/7/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@omalley omalley merged commit 909aeca into apache:trunk Nov 29, 2022
slfan1989 pushed a commit to slfan1989/hadoop that referenced this pull request Dec 20, 2022
@simbadzina
HDFS-16845: Adds configuration flag to allow clients to use router ob…
e263d73 
…server reads without using the ObserverReadProxyProvider. (apache#5142)
LiuGuH pushed a commit to LiuGuH/hadoop that referenced this pull request Mar 29, 2024
@simbadzina
HDFS-16845: Adds configuration flag to allow clients to use router ob…
0a2e76f 
…server reads without using the ObserverReadProxyProvider. (apache#5142)
NyteKnight pushed a commit to NyteKnight/hadoop that referenced this pull request Jun 25, 2024
@simbadzina
HDFS-16845: Adds configuration flag to allow clients to use router ob…
e402cac 
…server reads without using the ObserverReadProxyProvider. (apache#5142)

(Cherry-picked from 909aeca)
@simbadzina simbadzina deleted the sdzinama/HDFS-16845 branch June 28, 2024 20:22
LiuGuH pushed a commit to LiuGuH/hadoop that referenced this pull request Jul 4, 2024
@simbadzina
HDFS-16845: Adds configuration flag to allow clients to use router ob…
314ba2f 
…server reads without using the ObserverReadProxyProvider. (apache#5142)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mkuchenbecker mkuchenbecker mkuchenbecker left review comments

@omalley omalley omalley approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@simbadzina @hadoop-yetus @ZanderXu @mkuchenbecker @omalley
0