8000 HDFS-17719. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs Part1. by slfan1989 · Pull Request #7371 · apache/hadoop · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

HDFS-17719. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs Part1. #7371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Feb 12, 2025
Merged

HDFS-17719. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs Part1. #7371

merged 4 commits into from
Feb 12, 2025

Conversation

slfan1989
Copy link
Contributor
@slfan1989 slfan1989 commented Feb 8, 2025 •
edited
Loading

Description of PR

JIRA: HDFS-17719. Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs.

How was this patch tested?

Junit Test & mvn clean test.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@slfan1989 slfan1989 changed the title HDFS-17719. Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs. HDFS-17719. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs. Feb 8, 2025
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 47s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 34 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 40m 44s trunk passed
+1 💚 compile 0m 29s trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 compile 0m 28s trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 checkstyle 0m 29s trunk passed
+1 💚 mvnsite 0m 39s trunk passed
+1 💚 javadoc 0m 31s trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 28s trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 spotbugs 0m 51s trunk passed
+1 💚 shadedclient 38m 13s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 20s the patch passed
+1 💚 compile 0m 21s the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javac 0m 21s the patch passed
+1 💚 compile 0m 19s the patch passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 javac 0m 19s the patch passed
-1 ❌ blanks 0m 0s /blanks-eol.txt The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️ checkstyle 0m 18s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 38 new + 168 unchanged - 8 fixed = 206 total (was 176)
+1 💚 mvnsite 0m 28s the patch passed
+1 💚 javadoc 0m 23s the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 20s the patch passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 spotbugs 0m 49s the patch passed
+1 💚 shadedclient 38m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 1m 41s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt hadoop-hdfs-httpfs in the patch passed.
+1 💚 asflicense 0m 36s The patch does not generate ASF License warnings.
131m 6s
Reason Tests
Failed junit tests hadoop.fs.http.server.TestHttpFSServerNoXAttrs
hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem
hadoop.lib.service.scheduler.TestSchedulerService
hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
hadoop.fs.http.client.TestHttpFSFileSystemLocalFileSystem
hadoop.lib.servlet.TestServerWebApp
hadoop.lib.service.hadoop.TestFileSystemAccessService
hadoop.fs.http.server.TestHttpFSAccessControlled
hadoop.lib.server.TestServerConstructor
hadoop.lib.server.TestServer
hadoop.lib.service.security.TestGroupsService
hadoop.test.TestHFSTestCase
hadoop.test.TestHTestCase
hadoop.fs.http.server.TestHttpFSServerNoACLs
hadoop.fs.http.server.TestHttpFSServer
hadoop.lib.service.instrumentation.TestInstrumentationService
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/1/artifact/out/Dockerfile
GITHUB PR #7371
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname Linux 756801c5e6af 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / d252310
Default Java Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/1/testReport/
Max. process+thread count 527 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989 slfan1989 changed the title HDFS-17719. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs. HDFS-17719. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs Part1. Feb 9, 2025
@slfan1989 slfan1989 marked this pull request as ready for review February 9, 2025 12:11
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 17m 43s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 38 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 40m 3s trunk passed
+1 💚 compile 0m 29s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 0m 27s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 0m 30s trunk passed
+1 💚 mvnsite 0m 39s trunk passed
+1 💚 javadoc 0m 30s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 29s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 0m 51s trunk passed
+1 💚 shadedclient 39m 8s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 21s the patch passed
+1 💚 compile 0m 21s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 0m 21s the patch passed
+1 💚 compile 0m 20s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 0m 20s the patch passed
-1 ❌ blanks 0m 0s /blanks-eol.txt The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️ checkstyle 0m 18s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 48 new + 166 unchanged - 17 fixed = 214 total (was 183)
+1 💚 mvnsite 0m 28s the patch passed
+1 💚 javadoc 0m 23s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 20s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 0m 49s the patch passed
+1 💚 shadedclient 38m 15s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 55s hadoop-hdfs-httpfs in the patch passed.
+1 💚 asflicense 0m 38s The patch does not generate ASF License warnings.
152m 10s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/2/artifact/out/Dockerfile
GITHUB PR #7371
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname Linux 12d8a71878b0 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 2cb4855
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/2/testReport/
Max. process+thread count 876 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989 slfan1989 mentioned this pull request Feb 9, 2025
Draft
74 tasks
@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 49s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 38 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 41m 1s trunk passed
+1 💚 compile 0m 29s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubunt 8000 u-1ubuntu120.04
+1 💚 compile 0m 28s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 0m 29s trunk passed
+1 💚 mvnsite 0m 40s trunk passed
+1 💚 javadoc 0m 31s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 28s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 0m 51s trunk passed
+1 💚 shadedclient 38m 41s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 21s the patch passed
+1 💚 compile 0m 22s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 0m 22s the patch passed
+1 💚 compile 0m 20s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 0m 20s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 17s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 6 new + 166 unchanged - 17 fixed = 172 total (was 183)
+1 💚 mvnsite 0m 27s the patch passed
+1 💚 javadoc 0m 23s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 20s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 0m 49s the patch passed
+1 💚 shadedclient 38m 26s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 55s hadoop-hdfs-httpfs in the patch passed.
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
135m 53s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/3/artifact/out/Dockerfile
GITHUB PR #7371
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname Linux 26b0fdd41914 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / e21088c
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/3/testReport/
Max. process+thread count 870 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989
Copy link
Contributor Author

@cnauroth @ayushtkn This is the first unit test upgrade pr for HDFS. Can you help review this PR? Thank you very much!

This PR may provide additional information as follows:

The following JUnit 4 classes have been removed in JUnit 5:

  • org.junit.rules.MethodRule
  • org.junit.runners.model.FrameworkMethod
  • org.junit.runners.model.Statement

We will use JUnit 5's BeforeEachCallback, AfterEachCallback, and TestExecutionExceptionHandler as replacements for the relevant functionality. So far, it appears to be working as expected.

This PR may have some Checkstyle issues in the end, but I don’t plan to fix them for now, as doing so could introduce additional code changes. I plan to address these issues after all JUnit 4 to JUnit 5 upgrades are completed.

The list of Checkstyle issues is as follows:

./hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/HTestCase.java:40:  public TestDirHelper testDir = new TestDirHelper();:24: Variable 'testDir' must be private and have accessor methods. [VisibilityModifier]
./hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/HTestCase.java:43:  public TestJettyHelper jettyTestHelper = new TestJettyHelper();:26: Variable 'jettyTestHelper' must be private and have accessor methods. [VisibilityModifier]
./hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/HTestCase.java:46:  public TestExceptionHelper exceptionHelper = new TestExceptionHelper();:30: Variable 'exceptionHelper' must be private and have accessor methods. [VisibilityModifier]

@hadoop-yetus
Copy link

🎊 +1 overall

< 8000 thead>
Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 48s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 38 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 41m 1s trunk passed
+1 💚 compile 0m 30s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 0m 28s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 0m 29s trunk passed
+1 💚 mvnsite 0m 39s trunk passed
+1 💚 javadoc 0m 31s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 28s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 0m 50s trunk passed
+1 💚 shadedclient 38m 43s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 21s the patch passed
+1 💚 compile 0m 21s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 0m 21s the patch passed
+1 💚 compile 0m 19s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 0m 19s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 17s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 4 new + 166 unchanged - 17 fixed = 170 total (was 183)
+1 💚 mvnsite 0m 28s the patch passed
+1 💚 javadoc 0m 23s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 0m 20s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 0m 50s the patch passed
+1 💚 shadedclient 39m 2s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 5m 56s hadoop-hdfs-httpfs in the patch passed.
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
136m 32s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/4/artifact/out/Dockerfile
GITHUB PR #7371
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname Linux a016d3d25a36 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 2d768f9
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/4/testReport/
Max. process+thread count 868 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7371/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

cnauroth
cnauroth approved these changes Feb 10, 2025
View reviewed changes
Copy link
Contributor
@cnauroth cnauroth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Thank you, @slfan1989 .

szetszwo
szetszwo reviewed Feb 10, 2025
View reviewed changes
Copy link
Contributor
@szetszwo szetszwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry that I am new to jupiter.api.extension. So I am not able to review those changes. Please see if someone else could review them. If not, I suggest separating them to another PR and commit the other changes first.

+1 the other changes looks good.

@slfan1989
Copy link
Contributor Author
slfan1989 commented Feb 11, 2025 •
edited
Loading

Sorry that I am new to jupiter.api.extension. So I am not able to review those changes. Please see if someone else could review them. If not, I suggest separating them to another PR and commit the other changes first.

+1 the other changes looks good.

@szetszwo Thank you for your message! After carefully reviewing the code, I found that we may not be able to split it into two PRs. The main reason is that most of the HttpFS unit tests inherit from HTestCase, and a key detail is that all the tests rely on TestDirHelper, TestJettyHelper, and TestHdfsHelper to set up the foundational test environment before executing the actual methods. These three methods are the core parts that we have rewritten in this case.

Let me briefly describe the purpose of these methods.

BeforeEachCallback

The BeforeEachCallback interface allows us to perform custom actions before each test method is executed. It is part of JUnit 5's extension model and can be used to perform initialization tasks before each test method runs.

AfterEachCallback

The AfterEachCallback interface allows us to perform custom actions after each test method has been executed. It is commonly used for cleaning up resources, resetting states, or performing other tasks.

TestExecutionExceptionHandler

The TestExecutionExceptionHandler interface allows us to handle exceptions thrown during the execution of a test method. You can use it to customize the handling of exceptions, allowing certain exceptions to be suppressed or transformed into other types of exceptions.

hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/TestHdfsHelper.java

Let's take TestHdfsHelper.java as an example.

The core logic of JUnit 4.

@Override
public void evaluate() throws Throwable {
  Configuration conf = HadoopUsersConfTestHelper.getBaseConf();
  if (Boolean.parseBoolean(System.getProperty(HADOOP_MINI_HDFS, "true"))) {
    miniHdfs = startMiniHdfs(conf);
    conf = miniHdfs.getConfiguration(0);
  }
  try {
    // Step1. Before executing the unit test, initialize some configurations (Conf) and directories.
    HDFS_CONF_TL.set(conf);
    HDFS_TEST_DIR_TL.set(resetHdfsTestDir(conf));
    // Step2. Execute the unit test.
    statement.evaluate();
  } finally {
    // Step3. After the unit test finishes, clean up the configurations (Conf) and directories.
    HDFS_CONF_TL.remove();
    HDFS_TEST_DIR_TL.remove();
  }
}

In JUnit 5, we execute Step 1 in BeforeEach and Step 3 in AfterEach. Overall, this makes the code clearer.

Step1. beforeEach

@Override
public void beforeEach(ExtensionContext context) throws Exception {
  super.beforeEach(context);
  Method testMethod = context.getRequiredTestMethod();
  TestHdfs testHdfsAnnotation = testMethod.getAnnotation(TestHdfs.class);
  if (testHdfsAnnotation != null) {
    this.statement = new HdfsStatement(testMethod.getName());
    this.statement.evaluate();
  }
}

public void evaluate() throws Exception {
    Configuration conf = HadoopUsersConfTestHelper.getBaseConf();
    if (Boolean.parseBoolean(System.getProperty(HADOOP_MINI_HDFS, "true"))) {
      miniHdfs = startMiniHdfs(conf);
      conf = miniHdfs.getConfiguration(0);
    }
    HDFS_CONF_TL.set(conf);
    HDFS_TEST_DIR_TL.set(resetHdfsTestDir(conf));
  }

Step3. afterEach

@Override
  public void afterEach(ExtensionContext extensionContext) throws Exception {
    super.afterEach(extensionContext);
    HDFS_CONF_TL.remove();
    HDFS_TEST_DIR_TL.remove();
  }

The purpose of TestExecutionExceptionHandler is to check whether the thrown exception and message after a unit test fails match the expected ones. It is an additional feature that does not affect the execution of the unit test.

Overall, we haven't changed the existing execution framework; instead, we've optimized some details. The new changes do not impact the original unit tests, so I believe the risk is manageable.

cc: @cnauroth

@slfan1989 slfan1989 requested a review from szetszwo February 11, 2025 17:47
@szetszwo
Copy link
Contributor

Since @cnauroth has also reviewed this, let's merge this.

@slfan1989 slfan1989 merged commit b24996c into apache:trunk Feb 12, 2025
4 checks passed
@slfan1989
Copy link
Contributor Author

@cnauroth @szetszwo Thanks for the review! Merged into trunk.

adideshpande pushed a commit to adideshpande/hadoop that referenced this pull request Feb 27, 2025
@slfan1989 @cnauroth @szetszwo
HDFS-17719. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs P…
34939b5 
…art1. (apache#7371)

Co-authored-by: Chris Nauroth <cnauroth@apache.org>
Co-authored-by: Tsz-wo Sze <szetszwo@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
Reviewed-by: Tsz-wo Sze <szetszwo@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
YanivKunda pushed a commit to YanivKunda/hadoop that referenced this pull request Mar 23, 2025
@slfan1989 @cnauroth
@szetszwo @YanivKunda
HDFS-17719. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs-httpfs P…
7b8b06b 
…art1. (apache#7371)

Co-authored-by: Chris Nauroth <cnauroth@apache.org>
Co-authored-by: Tsz-wo Sze <szetszwo@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
Reviewed-by: Tsz-wo Sze <szetszwo@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cnauroth cnauroth cnauroth approved these changes

@szetszwo szetszwo szetszwo approved these changes

Assignees
No one assigned
Labels
build HDFS trunk
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@slfan1989 @hadoop-yetus @szetszwo @cnauroth
0