8000 [Nested] Optimize structs in `LIST_VALUE` by maiadegraaf · Pull Request #17169 · duckdb/duckdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[Nested] Optimize structs in LIST_VALUE #17169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 19 commits into
base: main
Choose a base branch
from
Draft

[Nested] Optimize structs in LIST_VALUE #17169

wants to merge 19 commits into from
+650 −129

Conversation

maiadegraaf
Copy link
Contributor

This is a follow-up to

This update focuses specifically on optimizing performance when constructing LIST_VALUE with STRUCT elements.

Speed up examples:

Large Struct Table (Benchmark added in this PR):

CREATE TABLE large_struct_table AS
SELECT {'a': i - 5, 'b': i - 4} AS a, {'a': i - 3, 'b': i - 2} AS b, {'a': i - 1,'b': i} AS c FROM range(10000000) tbl(i);

SELECT LIST_VALUE(a, b, c) FROM large_struct_table;
v1.2.2 New
5.83 s 0.133 s

Nested Unpivot

CREATE TABLE structs AS
SELECT i AS id, {'id': i} AS "2020", {'id': i + 10} AS "2021", {'id': i + 100} AS "2022", {'id': i + 1000} AS "2023", {'id': i + 10000} AS "2024"
FROM range(10000000) t(i)

UNPIVOT structs ON "2020", "2021", "2022", "2023", "2024"
v1.2.2 New
3.24 s 0.39 s

taniabogatsch
taniabogatsch reviewed Apr 18, 2025
View reviewed changes
Copy link
Contributor
@taniabogatsch taniabogatsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @maiadegraaf, looks good! I left a few small comments.

@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 22, 2025 13:11
@Tishj Tishj mentioned this pull request May 7, 2025
Merged
Mytherin added a commit that referenced this pull request May 8, 2025
@Mytherin
[Dev] Flatten any deeper children vectors, when the top level is a FL…
d4816b0 
…AT vector (#17387)

This PR fixes an issue found in
<#17169>

But since it has (likely) effect on much more tests and codepaths, I'm
making this PR separately.
@maiadegraaf maiadegraaf marked this pull request as ready for review May 8, 2025 17:04
@maiadegraaf maiadegraaf marked this pull request as draft May 9, 2025 14:14
@maiadegraaf maiadegraaf marked this pull request as ready for review May 9, 2025 14:14
Mytherin
Mytherin reviewed May 12, 2025
View reviewed changes
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 14, 2025 14:49
Tishj
Tishj reviewed May 14, 2025
View reviewed changes
@maiadegraaf maiadegraaf marked this pull request as ready for review May 23, 2025 14:45
@maiadegraaf
Copy link
Contributor Author

This PR has undergone further changes and now lists of structs are recursively built through calls to PopulateChild. I've also added even more tests

@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 23, 2025 14:53
@maiadegraaf maiadegraaf marked this pull request as ready for review May 23, 2025 14:54
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 27, 2025 12:15
@maiadegraaf maiadegraaf marked this pull request as ready for review May 27, 2025 12:17
@duckdb-draftbot duckdb-draftbot marked this pull request as draft May 27, 2025 15:46
@maiadegraaf maiadegraaf force-pushed the list_value-struct-optimize branch from c8dcc4c to b83e375 Compare May 28, 2025 08:53
@maiadegraaf maiadegraaf force-pushed the list_value-struct-optimize branch from b83e375 to addfe85 Compare May 28, 2025 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Mytherin Mytherin Mytherin left review comments

@Tishj Tishj Tishj left review comments

@taniabogatsch taniabogatsch taniabogatsch left review comments

Assignees
No one assigned
Labels
Ready For Review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@maiadegraaf @Mytherin @Tishj @taniabogatsch
0