8000 [BugFix] Fix some bugs in scenarios where file_bundling and alter operations intersect by sevev · Pull Request #60091 · StarRocks/starrocks · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[BugFix] Fix some bugs in scenarios where file_bundling and alter operations intersect #60091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[BugFix] Fix some bugs in scenarios where file_bundling and alter operations intersect #60091

wants to merge 3 commits into from
+218 −94

Conversation

sevev
Copy link
Contributor
@sevev sevev commented Jun 19, 2025 •
edited
Loading

Why I'm doing:

After enabling file_bundling, there are several bugs in alter tasks:

  1. When file_bundling is enabled, an initial metadata file (000000_0000001.meta) is created during table creation. When executing an alter to create a new tablet, the same metadata file is also created, causing overwriting of duplicate-named metadata files.
  2. When creating a Lake rollup with file_bundling enabled, tablets should be aggregated first, followed by a single aggregatePublish execution, rather than splitting into two operations.
  3. In CN, the meta paths of tablets with file_bundling enabled are cached to reduce invalid get API calls. However, the cached paths are virtual paths provided by StarOS bound to the tablet ID, which prevents other tablets under the same partition from hitting the cache.

What I'm doing:

Fixes https://github.com/StarRocks/StarRocksTest/issues/9831, https://github.com/StarRocks/StarRocksTest/issues/9840

  1. Disable create bundle_metadata when doing alter job and create tablet metadata for each tablet.
  2. Aggregate publish version request first and run aggregatePublish once when adding lake rollup.
  3. Using the real partition path as the cache key.

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.5
    • 3.4
    • 3.3

@sevev sevev requested review from a team as code owners June 19, 2025 11:49
@mergify mergify bot assigned sevev Jun 19, 2025
@sevev sevev force-pushed the fix_aggregate_publish_with_alter branch from 6614d1c to 896182c Compare June 20, 2025 11:35
sevev added 2 commits June 20, 2025 20:01
@sevev
fix aggregate publish bug with alter
42ba216 
Signed-off-by: sevev <qiangzh95@gmail.com>
@sevev
update
0978a2b 
Signed-off-by: sevev <qiangzh95@gmail.com>
@sevev sevev force-pushed the fix_aggregate_publish_with_alter branch from 896182c to 0978a2b Compare June 20, 2025 12:02
@sevev sevev changed the title [BugFix][WIP] Fix some bugs in scenarios where file_bundling and alter operations intersect. [BugFix] Fix some bugs in scenarios where file_bundling and alter operations intersect. Jun 20, 2025
@sevev sevev changed the title [BugFix] Fix some bugs in scenarios where file_bundling and alter operations intersect. [BugFix] Fix some bugs in scenarios where file_bundling and alter operations intersect Jun 20, 2025
@sevev
update
a544752 
Signed-off-by: sevev <qiangzh95@gmail.com>
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@github-actions GitHub Actions
Copy link

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[FE Incremental Coverage Report]

✅ pass : 35 / 40 (87.50%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/lake/Utils.java 20 25 80.00% [300, 302, 303, 304, 311]
🔵 com/starrocks/alter/LakeRollupJob.java 13 13 100.00% []
🔵 com/starrocks/alter/LakeTableSchemaChangeJob.java 2 2 100.00% []

@github-actions GitHub Actions
Copy link

[BE Incremental Coverage Report]

✅ pass : 71 / 73 (97.26%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/storage/lake/tablet_manager.cpp 68 70 97.14% [360, 371]
🔵 be/src/storage/lake/vacuum.cpp 3 3 100.00% []

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@sevev sevev

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@sevev
0