8000 feat(frontend): support sink_decouple for sink into table by kwannoel · Pull Request #21784 · risingwavelabs/risingwave · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(frontend): support sink_decouple for sink into table #21784

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
May 9, 2025
Merged

feat(frontend): support sink_decouple for sink into table #21784

merged 5 commits into from
May 9, 2025

Conversation

kwannoel
Copy link
Contributor
@kwannoel kwannoel commented May 8, 2025 •
edited
Loading

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

#19285

Use it when sink_decouple is set to true for sink into table. This is supported by placing a logstore before the sink.

upstream -> logstore -> sink -> rw table

We don't enable sink_decouple by default for sink into table. But it's available if the user specifies sink_decouple=true.

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • I have checked the Release Timeline and Currently Supported Versions to determine which release branches I need to cherry-pick this PR into.

Documentation

  • My PR needs documentation updates.
Release note

We support sink_decouple for sink into table. This will isolate the downstream slowness from upstream.

set sink_decouple=true;
create table t(v1 int primary key, v2 int);
create table t2(v1 int primary key, v2 int);
create sink s1 into t from t2;

-- If this downstream MV is slow, it will not block the sink s1,
-- since the upstream and downstreams are decoupled.
create materialized view m1 as select v1, sum(v2) from t group by v1;

@github-actions github-actions bot added the type/feature Type: New feature. label May 8, 2025
@kwannoel kwannoel force-pushed the kwannoel/sink-into-table-log-stire branch from 1d2eb65 to 8678386 Compare May 8, 2025 17:11
@kwannoel kwannoel marked this pull request as ready for review May 8, 2025 17:12
@kwannoel kwannoel requested review from shanicky and wenym1 May 8, 2025 17:12
@kwannoel kwannoel changed the title feat(frontend): support sink into table logstore feat(frontend): support sink_decouple for sink into table May 8, 2025
wenym1
wenym1 approved these changes May 9, 2025
View reviewed changes
Copy link
Contributor
@wenym1 wenym1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

@kwannoel kwannoel added this pull request to the merge queue May 9, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 9, 2025
@kwannoel kwannoel added this pull request to the merge queue May 9, 2025
Merged via the queue into main with commit d0d4340 May 9, 2025
31 of 32 checks passed
@kwannoel kwannoel deleted the kwannoel/sink-into-table-log-stire branch May 9, 2025 08:48
@kwannoel kwannoel added the user-facing-changes Contains changes that are visible to users label May 14, 2025
@github-actions GitHub Actions
Copy link
Contributor

Hi, there.

📝 Telemetry Reminder:
If you're implementing this feature, please consider adding telemetry metrics to track its usage. This helps us understand how the feature is being used and improve it further.
You can find the function report_event of telemetry reporting in the following files. Feel free to ask questions if you need any guidance!

  • src/frontend/src/telemetry.rs
  • src/meta/src/telemetry.rs
  • src/stream/src/telemetry.rs
  • src/storage/compactor/src/telemetry.rs
    Or calling report_event_common (src/common/telemetry_event/src/lib.rs) as if finding it hard to implement.
    ✨ Thank you for your contribution to RisingWave! ✨

This is an automated comment created by the peaceiris/actions-label-commenter. Responding to the bot or mentioning it won't have any effect.

@neverchanje neverchanje mentioned this pull request May 14, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wenym1 wenym1 wenym1 approved these changes

@shanicky shanicky Awaiting requested review from shanicky

Assignees
No one assigned
Labels
type/feature Type: New feature. user-facing-changes Contains changes that are visible to users
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kwannoel @wenym1
0