8000 Duplicated IDs are generated · Issue #12197 · ray-project/ray · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Duplicated IDs are generated #12197
Open
Open
@kfstorm

Description

@kfstorm

What is the problem?

The string generated by GenerateUniqueBytes is not really unique.

ray/src/ray/common/id.cc

Lines 40 to 53 in 0178d63

std::string GenerateUniqueBytes(const JobID &job_id, const TaskID &parent_task_id,
size_t parent_task_counter, size_t length) {
RAY_CHECK(length <= DIGEST_SIZE);
SHA256_CTX ctx;
sha256_init(&ctx);
sha256_update(&ctx, reinterpret_cast<const BYTE *>(job_id.Data()), job_id.Size());
sha256_update(&ctx, reinterpret_cast<const BYTE *>(parent_task_id.Data()),
parent_task_id.Size());
sha256_update(&ctx, (const BYTE *)&parent_task_counter, sizeof(parent_task_counter));
BYTE buff[DIGEST_SIZE];
sha256_final(&ctx, buff);
return std::string(buff, buff + length);
}

Ray version and other system information (Python version, TensorFlow version, OS): 1.0.1

Reproduction (REQUIRED)

Apply below patch and run bazel run //:id_test,

diff --git a/src/ray/common/id_test.cc b/src/ray/common/id_test.cc
index 926e6fbd11..f1274d92a5 100644
--- a/src/ray/common/id_test.cc
+++ b/src/ray/common/id_test.cc
@@ -51,6 +51,15 @@ TEST(ActorIDTest, TestActorID) {
     const ActorID actor_id = ActorID::Of(kDefaultJobId, kDefaultDriverTaskId, 1);
     ASSERT_EQ(kDefaultJobId, actor_id.JobId());
   }
+
+  {
+    // test no duplicated ID
+    std::unordered_set<ActorID> ids;
+    for (size_t i = 0; i < 1000000; i++) {
+        auto id = ActorID::Of(kDefaultJobId, kDefaultDriverTaskId, i);
+        RAY_CHECK(ids.insert(id).second) << "Duplicated ID generated: " << id;
+    }
+  }
 }

 TEST(TaskIDTest, TestTaskID) {

You will see the error:

[2020-11-20 05:37:07,442 C 104416 104416] id_test.cc:60:  Check failed: ids.insert(id).second Duplicated ID generated: 584a7e60c7000000

If we cannot run your script, we cannot fix your issue.

  • I have verified my script runs in a clean environment and reproduces the issue.
  • I have verified the issue also occurs with the latest wheels.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issue, but not time-criticalbugSomething that is supposed to be working; but isn'tpending-cleanupThis issue is pending cleanup. It will be removed in 2 weeks after being assigned.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0