8000 feat(meta): Try introducing an in-memory actor cache by shanicky · Pull Request #21994 · risingwavelabs/risingwave · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(meta): Try introducing an in-memory actor cache #21994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat(meta): Try introducing an in-memory actor cache #21994

wants to merge 4 commits into from

Conversation

shanicky
Copy link
Contributor

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Please note that this PR is still in a very unstable phase and requires extensive testing to ensure stability. The "ready" status is only for testing CI.

This PR is part of the plan to deprecate the Actor table in the metastore. It changes the access to the Actor table (including direct access like Actor::find and indirect access like find_also_related(Actor)) to access the actor cache within the catalog controller instead, and it will compare the results in debug mode.

In future PRs, it will combine the algorithm from PR #21803 to generate a complete list of actors at startup, replacing the need to read from the metastore.

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • I have checked the Release Timeline and Currently Supported Versions to determine which release branches I need to cherry-pick this PR into.

Documentation

  • My PR needs documentation updates.
Release note

shanicky added 4 commits May 28, 2025 14:25
@shanicky
Cache actor and fragment info in-memory; update lookups; add debug ch…
799dd06 
…ecks for cache/DB consistency

Signed-off-by: Peng Chen <peng@risingwave-labs.com>
@shanicky
Add Hash to enums; refactor controllers to use ActorInfo cache for fr…
b6246bd 
…agment/actor mappings, not DB

Signed-off-by: Peng Chen <peng@risingwave-labs.com>
@shanicky
Refactor CatalogController: use in-memory ActorInfo cache for fragmen…
b459259 
…ts; add actor drop/mutate methods and cache sync

Signed-off-by: Peng Chen <peng@risingwave-labs.com>
@shanicky
Refactor fragment–actor–worker mapping and parallelism logic to use o…
698c160 
…nly in-memory cache; clean imports.

Signed-off-by: Shanicky Chen <peng@risingwave-labs.com>
@shanicky shanicky force-pushed the peng/metadata-actor-cache branch from d9b6f17 to 698c160 Compare May 28, 2025 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
ci/run-e2e-single-node-tests ci/run-e2e-test-other-backends type/feature Type: New feature.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@shanicky
0