8000 feat(sink): support redis sink with pubsub format by xxhZs · Pull Request #20991 · risingwavelabs/risingwave · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(sink): support redis sink with pubsub format #20991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Mar 25, 2025
Merged

feat(sink): support redis sink with pubsub format #20991

merged 7 commits into from
Mar 25, 2025

Conversation

xxhZs
Copy link
Contributor
@xxhZs xxhZs commented Mar 19, 2025 •
edited
Loading

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

  • My PR needs documentation updates.
Release note

We can use pubsub like

create table t1(v1 float, v2 float,v3 varchar, v4 varchar);
CREATE SINK s1
FROM
    t1 WITH (
    primary_key = 'v3,v4',
    connector = 'redis',
    redis.url= 'redis://127.0.0.1:6379/',
)FORMAT UPSERT ENCODE TEMPLATE
(redis_value_type ='pubsub', 
channel= 'test123',
value_format = 'abcd3:{v4}'
);

or

CREATE SINK s1
FROM
    t1 WITH (
    primary_key = 'v3,v4',
    connector = 'redis',
    redis.url= 'redis://127.0.0.1:6379/',
)FORMAT UPSERT ENCODE TEMPLATE
(redis_value_type ='pubsub', 
channel_column = 'v3',
value_format = 'abcd3:{v4}'
);
insert into t1 values(1.1, 1.1, 'test','test');

value_format Required: Format of the message passed to pubsub
pubsub_name pubsub_column both must set at least one.
pubsub_name The name of the message queue to send to
pubsub_column Use the values in this column as the name of the message queue to send to (type must be varchar)

@xxhZs xxhZs requested a review from wenym1 March 19, 2025 08:33
@github-actions github-actions bot added the type/feature Type: New feature. label Mar 19, 2025
wenym1
wenym1 approved these changes Mar 24, 2025
View reviewed changes
Copy link
Contributor
@wenym1 wenym1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation LGTM.

From the doc of publish, the semantic of key is the channel. So should we change the option name pubsub_name to channel, and pubsub_column to channel_column.

@xxhZs xxhZs enabled auto-merge March 24, 2025 09:46
@xxhZs xxhZs force-pushed the xxh/redis-pubsub-2 branch from c8a41f4 to 35f0eb3 Compare March 24, 2025 09:48
@xxhZs xxhZs added this pull request to the merge queue Mar 25, 2025
Merged via the queue into main with commit 20cb21f Mar 25, 2025
43 of 44 checks passed
@xxhZs xxhZs deleted the xxh/redis-pubsub-2 branch March 25, 2025 04:10
xxhZs added a commit that referenced this pull request Mar 25, 2025
@xxhZs
feat(sink): support redis sink with pubsub format (#20991)
e6182d3
xxhZs added a commit that referenced this pull request Mar 25, 2025
@xxhZs
feat(sink): support redis sink with pubsub format (#20991)
61407cb
xxhZs added a commit that referenced this pull request Mar 25, 2025
@xxhZs
feat(sink): support redis sink with pubsub format (#20991)
bf2eb5e
@xxhZs xxhZs added the user-facing-changes Contains changes that are visible to users label Mar 26, 2025
@github-actions GitHub Actions
Copy link
Contributor

Hi, there.

📝 Telemetry Reminder:
If you're implementing this feature, please consider adding telemetry metrics to track its usage. This helps us understand how the feature is being used and improve it further.
You can find the function report_event of telemetry reporting in the following files. Feel free to ask questions if you need any guidance!

  • src/frontend/src/telemetry.rs
  • src/meta/src/telemetry.rs
  • src/stream/src/telemetry.rs
  • src/storage/compactor/src/telemetry.rs
    Or calling report_event_common (src/common/telemetry_event/src/lib.rs) as if finding it hard to implement.
    ✨ Thank you for your contribution to RisingWave! ✨

This is an automated comment created by the peaceiris/actions-label-commenter. Responding to the bot or mentioning it won't have any effect.

@neverchanje neverchanje mentioned this pull request Mar 26, 2025
Closed
@neverchanje neverchanje mentioned this pull request Apr 30, 2025
Closed
github-actions bot pushed a commit that referenced this pull request Apr 30, 2025
@xxhZs
feat(sink): support redis sink with pubsub format (#20991)
2f188ec
@github-actions github-actions bot mentioned this pull request Apr 30, 2025
Merged
github-merge-queue bot pushed a commit that referenced this pull request May 7, 2025
@github-actions @xxhZs
feat(sink): support redis sink with pubsub format (#20991) (#21663)
fb44767 
Co-authored-by: Xinhao Xu <84456268+xxhZs@users.noreply.github.com>
@WanYixian WanYixian mentioned this pull request May 13, 2025
Merged
github-merge-queue bot pushed a commit that referenced this pull request May 19, 2025
@github-actions @xxhZs
@lmatz @cyliu0
feat(sink): support redis sink with pubsub format (#20991) (#21663)
4760952 
Co-authored-by: Xinhao Xu <84456268+xxhZs@users.noreply.github.com>
Co-authored-by: lmatz <lmatz823@gmail.com>
Co-authored-by: Chengyou Liu <35356271+cyliu0@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wenym1 wenym1 wenym1 approved these changes

Assignees
No one assigned
Labels
ci/run-e2e-redis-sink-tests need-cherry-pick-since-release-2.3 type/feature Type: New feature. user-facing-changes Contains changes that are visible to users
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@xxhZs @wenym1
0