Permalink
Comparing changes
Choose two branches to see what’s changed or to start a new pull request.
If you need to, you can also or
learn more about diff comparisons.
Open a pull request
Create a new pull request by comparing changes across two branches. If you need to, you can also .
Learn more about diff comparisons here.
...
- 6 commits
- 53 files changed
- 5 contributors
Commits on Jun 17, 2025
-
feat!: auto-remap indexes before scan (#3971)
BREAKING CHANGE: this PR uses the fragment reuse index (FRI) to auto-remap all indexes during their load time. By doing so, an index will automatically cover the fragments that produced after compaction even before a physical index remap has happened, and related scan/FTS/KNN search will be able to leverage it. Note that when we say "load time", it does not always mean when the index is opened. Most indexes are loaded on-demand at search time and then cached, this is when the auto-remap happens for a specific subset of the row IDs in an index. This change introduces the following breaking changes: 1. add `fri: Option<Arc<FragReuseIndex>>` for all related index loading functions 2. simplify the FRI index structure. With this auto-remap functionality, we no longer need to track the old fragment locations to read old fragments. We now only track fragment ID, physical_rows and num_deleted_rows of old and new fragments 3. modify signatures of related fragment reuse index functions to adopt to the new index structure Note that this change does not include auto-remap for HNSW because it is not widely used and it is not remapped today anyway. This work is tracked separately in #3993. Closes #3837
-
feat(rust): add get config interface for dataset (#4022)
I don't know why this table config is not retrievable. I added an interface for this. If this is unnecessary, I expect someone could tell me how to get it by any means Co-authored-by: majin.nathan <majin.nathan@bytedance.com>
Commits on Jun 18, 2025
-
chore: fix broken spark doc link (#4025)
Fixes https://github.com/lancedb/lance/actions/runs/15718440005/job/44293946196. Note that this does not fix all the out of date Spark-Lance documentation. I am working on an update for that in lancedb/lance-spark#24
-
feat: support unenforced primary key concept in schema (#4002)
Allow users to set primary key through Arrow schema metadata with `primary_key` config key. A primary key column must not be nullable. User can configure composite primary key through `,` delimiter, or using a custom delimiter specified by another `primary_key_delim` config key. Closes #4003
-
fix: filter out null row ids (#4020)
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
Loading
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v0.29.1-beta.2...v0.30.0-beta.1