8000 db: consider biasing towards rewriting blob files with few references · Issue #4763 · cockroachdb/pebble · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

db: consider biasing towards rewriting blob files with few references #4763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jbowens opened this issue May 22, 2025 · 0 comments
Open

db: consider biasing towards rewriting blob files with few references #4763

jbowens opened this issue May 22, 2025 · 0 comments
Labels
A-storage T-storage

Comments

@jbowens
Copy link
Collaborator
jbowens commented May 22, 2025 •
edited by cockroach-jira-scripts
Loading

Currently the MaxBlobReferenceDepth heuristic drives the decision of whether or not a compaction should write values to new blob files. This heuristic doesn't take into account the possibility that writing new blob files may allow disk space reclamation. It only prioritizes restoring locality.

In the extreme case, imagine a compaction with 1 input sstable referencing 10KiB of a single 128 MiB blob file. If all the other values in the blob file are unreferenced, the blob file only remains because of the extant references that the compaction is rewriting. Even if the MaxBlobReferenceDepth heuristic wouldn't otherwise choose to write new blob files, reclaiming the ~128MiB is worth the small amount of additional write I/O from writing the 10KiB values to a new blob file.

Jira issue: PEBBLE-453
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-storage T-storage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jbowens
0