HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. #3320

Neilxzn · 2021-08-23T07:30:16Z

Description of PR

https://issues.apache.org/jira/browse/HDFS-16182

How was this patch tested?

add TestBlockStoragePolicy.testAddDatanode2ExistingPipelineInSsd

For code changes:

hadoop-yetus · 2021-08-23T15:01:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 18s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 36s		trunk passed
+1 💚	compile	1m 24s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 1s		trunk passed
+1 💚	mvnsite	1m 28s		trunk passed
+1 💚	javadoc	0m 58s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 23s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 18s		trunk passed
+1 💚	shadedclient	19m 10s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 18s		the patch passed
+1 💚	compile	1m 19s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 19s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 12s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 51s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 154 unchanged - 1 fixed = 155 total (was 155)
+1 💚	mvnsite	1m 18s		the patch passed
+1 💚	javadoc	0m 50s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 20s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 22s		the patch passed
+1 💚	shadedclient	19m 3s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	356m 5s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 40s		The patch does not generate ASF License warnings.
		449m 41s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.ha.TestEditLogTailer

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/1/artifact/out/Dockerfile
GITHUB PR	#3320
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 5ca83a692447 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `74f75f9`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/1/testReport/
Max. process+thread count	1983 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

jojochuang · 2021-08-24T03:49:15Z

Don't really understand the code but seems like an ancient regression from HDFS-6686. As a general code advice, we should not update a parameter variable and pass it on.

Neilxzn · 2021-08-24T04:04:40Z

@jojochuang Agree with you. I think we should fix it.

In my cluster, we use BlockPlacementPolicyDefault to choose dn and the number of SSD DN is much less than DISK DN. It may cause to some block that should be placed to SSD DNs fallback to place DISK DNs when SSD DNs are too busy or no enough place. Consider the following scenario.

Create empty file /foo_file
Set its storagepolicy to All_SSD
Put data to /foo_file
/foo_file gets 3 DISK dns for pipeline because SSD dns are too busy at the beginning.
When it transfers data in pipeline, one of 3 DISK dns shut down.
The client need to get one new dn for existing pipeline in DataStreamer$addDatanode2ExistingPipeline.
If SSD dns are available at the moment, namenode will choose the 3 SSD dns and return it to the client. However, the client just need one new dn, namenode returns 3 new SSD dn and the client threw exception in DataStreamer$findNewDatanode.

jojochuang

Just a few nits.

...src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java

...op-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java

2.declare numOfReplicas a final variable at line 438

Hexiaoqiao

Thanks @Neilxzn for your works. Great catch here! LGTM. +1 once leaved nits comments fixed.

...op-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java

...src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java

hadoop-yetus · 2021-08-25T14:12:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 13s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 5s		trunk passed
+1 💚	compile	1m 26s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 26s		trunk passed
+1 💚	javadoc	0m 58s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 26s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 23s		trunk passed
+1 💚	shadedclient	19m 41s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 20s		the patch passed
+1 💚	compile	1m 26s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 26s		the patch passed
+1 💚	compile	1m 20s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 20s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 58s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 153 unchanged - 2 fixed = 155 total (was 155)
+1 💚	mvnsite	1m 26s		the patch passed
+1 💚	javadoc	0m 51s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 26s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 42s		the patch passed
+1 💚	shadedclient	19m 38s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	344m 21s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		440m 16s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.ha.TestEditLogTailer

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/2/artifact/out/Dockerfile
GITHUB PR	#3320
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 90ed77414ee5 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `53e9dcb`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/2/testReport/
Max. process+thread count	2179 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Neilxzn · 2021-08-25T17:03:46Z

Thanks @Neilxzn for your works. Great catch here! LGTM. +1 once leaved nits comments fixed.

I think it is a bug just when fallback storage policy happens.

hadoop-yetus · 2021-08-26T00:25:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 6s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 34s		trunk passed
+1 💚	compile	1m 28s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 16s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 25s		trunk passed
+1 💚	javadoc	0m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 30s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 20s		trunk passed
+1 💚	shadedclient	19m 6s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 15s		the patch passed
+1 💚	compile	1m 18s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 18s		the patch passed
+1 💚	compile	1m 11s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 11s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 54s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 153 unchanged - 2 fixed = 155 total (was 155)
+1 💚	mvnsite	1m 18s		the patch passed
+1 💚	javadoc	0m 49s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 19s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 21s		the patch passed
+1 💚	shadedclient	18m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	353m 51s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		447m 30s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.ha.TestEditLogTailer
	hadoop.hdfs.TestHDFSFileSystemContract
	hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/3/artifact/out/Dockerfile
GITHUB PR	#3320
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux f0c899902df0 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `90fcf70`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/3/testReport/
Max. process+thread count	2005 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Neilxzn · 2021-08-26T02:29:14Z

Failed junit tests seem unrelated. And I try these tests in IDEA locally, these tests run pass.

hadoop.hdfs.server.namenode.ha.TestEditLogTailer
hadoop.hdfs.TestHDFSFileSystemContract
hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes

Hexiaoqiao · 2021-08-26T03:55:11Z

@Neilxzn Please fix checkstyle and check failed unit tests first. Thanks

hadoop-yetus · 2021-08-26T12:00:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 1s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 13s		trunk passed
+1 💚	compile	1m 23s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 18s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	0m 56s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 25s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 16s		trunk passed
+1 💚	shadedclient	18m 44s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 15s		the patch passed
+1 💚	compile	1m 18s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 18s		the patch passed
+1 💚	compile	1m 9s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 53s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 153 unchanged - 2 fixed = 154 total (was 155)
+1 💚	mvnsite	1m 21s		the patch passed
+1 💚	javadoc	0m 48s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 17s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 21s		the patch passed
+1 💚	shadedclient	18m 56s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	360m 45s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		452m 43s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.ha.TestEditLogTailer
	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
	hadoop.hdfs.TestHDFSFileSystemContract

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/4/artifact/out/Dockerfile
GITHUB PR	#3320
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux b22575c707a1 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `9bb5daa`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/4/testReport/
Max. process+thread count	2066 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3320/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Neilxzn · 2021-08-26T14:12:21Z

@Neilxzn Please fix checkstyle and check failed unit tests first. Thanks
@Hexiaoqiao Thank you for your review.

About BlockPlacementPolicyDefault$chooseTarget, ParameterNumber checkstyle warning is not generated by this patch and it is hard to fix. Maybe we ignore it this patch?
I checked these failed tests agian and these tests run pass locally. And these tests seem unrelated. Please review these tests again. Thanks.

hadoop.hdfs.server.namenode.ha.TestEditLogTailer
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
hadoop.hdfs.TestHDFSFileSystemContract

Neilxzn · 2021-09-01T08:36:39Z

cc @jojochuang . Failed junit tests seem unrelated. Can you review this patch again? Thanks.

jojochuang

+1

…licyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. (#3320) (cherry picked from commit 5626734) (cherry picked from commit ce3a7155335416ed4665f90f0ddd7ac3a3edd7e6)

…licyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. (#3320) (cherry picked from commit 5626734)

…licyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. (apache#3320)

…licyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. (apache#3320) (cherry picked from commit 5626734) Change-Id: Ia57b499cb39eb70272591f8e57b2251da470d83c (cherry picked from commit 7f254f4) Signed-off-by: Arpit Agarwal <aagarwal@cloudera.com>

Neilxzn added 2 commits August 20, 2021 22:56

add test testAddDatanode2ExistingPipelineInSsd

33c7e2a

fix numOfReplicas is wrong value

74f75f9

jojochuang requested changes Aug 25, 2021

View reviewed changes

...src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java Show resolved Hide resolved

...op-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java Outdated Show resolved Hide resolved

1.remove unnecessary log message in test

53e9dcb

2.declare numOfReplicas a final variable at line 438

Hexiaoqiao reviewed Aug 25, 2021

View reviewed changes

...op-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java Show resolved Hide resolved

...src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java Show resolved Hide resolved

add javadoc for testAddDatanode2ExistingPipelineInSsd

90fcf70

1.fix checkstyle in testAddDatanode2ExistingPipelineInSsd

9bb5daa

Neilxzn requested a review from jojochuang August 31, 2 8000 021 06:29

jojochuang approved these changes Nov 15, 2021

View reviewed changes

jojochuang merged commit 5626734 into apache:trunk Nov 16, 2021

asfgit pushed a commit that referenced this pull request Nov 16, 2021

HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPo…

c182bf6

…licyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. (#3320) (cherry picked from commit 5626734)

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPo…

1bdca0d

…licyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. (apache#3320)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. #3320

HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. #3320

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. #3320

HDFS-16182.numOfReplicas is given the wrong value in BlockPlacementPolicyDefault$chooseTarget can cause DataStreamer to fail with Heterogeneous Storage. #3320

Uh oh!

Conversation

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!