8000 Add conn.query_progress() method by nickzoic · Pull Request #16927 · duckdb/duckdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add conn.query_progress() method #16927

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
May 10, 2025
Merged

Conversation

nickzoic
Copy link
Contributor

This is an alternative PR to Tishj#26 which adds a conn.query_progress() method which can be run from a supervisory thread, similar to how you'd use conn.interrupt() which was added in #7895 and this PR follows that one pretty closely.

@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 1, 2025 00:04
@nickzoic nickzoic marked this pull request as ready for review April 1, 2025 00:06
@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 1, 2025 00:36
@nickzoic nickzoic marked this pull request as ready for review April 1, 2025 00:36
@nickzoic nickzoic force-pushed the nickzoic-query-progress branch from 51e645f to 834fe2d Compare April 1, 2025 00:45
@nickzoic
Copy link
Contributor Author
nickzoic commented Apr 1, 2025

(As is probably pretty clear from the history above and below, I have no idea what I'm doing with this CI pipeline. Please bear with me while I step on a series of rakes.)

@nickzoic nickzoic marked this pull request as draft April 1, 2025 01:28
@nickzoic nickzoic marked this pull request as ready for review April 1, 2025 01:28
@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 1, 2025 01:39
@nickzoic nickzoic marked this pull request as ready for review April 1, 2025 02:50
@Mytherin
Copy link
Collaborator
Mytherin commented Apr 1, 2025

Thanks for the PR! I think the only related failure is the format check - we're having some issues with other CI failures due to some external stuff.

@Mytherin Mytherin requested a review from Tishj April 1, 2025 07:08
@nickzoic nickzoic force-pushed the nickzoic-query-progress branch from 834fe2d to 9e9017e Compare April 2, 2025 00:08
@nickzoic
Copy link
Contributor Author
nickzoic commented Apr 2, 2025

Yeah, I've stuffed something up with whitespace in the generated files, I'll try and get it submitted properly.

@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 2, 2025 00:08
@nickzoic nickzoic marked this pull request as ready for review April 2, 2025 00:10
@nickzoic nickzoic force-pushed the nickzoic-query-progress branch from 9e9017e to 689ef28 Compare April 2, 2025 00:14
@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 2, 2025 00:14
@nickzoic nickzoic marked this pull request as ready for review April 2, 2025 00:15
@nickzoic nickzoic force-pushed the nickzoic-query-progress branch from 689ef28 to 5848a06 Compare April 2, 2025 00:34
@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 2, 2025 00:34
@nickzoic nickzoic marked this pull request as ready for review April 2, 2025 00:34
@nickzoic nickzoic force-pushed the nickzoic-query-progress branch from 5848a06 to cb61b32 Compare April 2, 2025 00:41
@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 2, 2025 00:52
@nickzoic nickzoic marked this pull request as ready for review April 2, 2025 01:18
@nickzoic
Copy link
Contributor Author
nickzoic commented Apr 2, 2025

I think I've used more CPU cycles on CI than the entire Apollo program used for the moon landings, sorry about that!

But hopefully maybe it'll work this time ...

@nickzoic
Copy link
Contributor Author
nickzoic commented Apr 4, 2025

I'd also considered whether this should return a tuple of (rows processed, rows total) instead of just a percentage.

A bonus issue to mention is that this implementation only works if progress bars are active for the cursor, which can be a bit confusing, especially given #16964 these settings aren't shared between cursors. There might be a better way to do this, too.

@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 8, 2025 02:04
@nickzoic nickzoic marked this pull request as ready for review April 8, 2025 02:25
@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 8, 2025 22:42
@nickzoic nickzoic marked this pull request as ready for review April 8, 2025 22:42
@duckdb-draftbot duckdb-draftbot marked this pull request as draft April 9, 2025 03:08
@nickzoic nickzoic marked this pull request as ready for review April 9, 2025 03:45
@nickzoic nickzoic marked this pull request as draft April 9, 2025 04:55
@nickzoic nickzoic marked this pull request as ready for review April 9, 2025 04:56
@nickzoic
Copy link
Contributor Author
nickzoic commented Apr 9, 2025

Woohoo! Finally got this thing to pass all its tests, which was an adventure.

@Mytherin & @Tishj this very simple approach works well for my purposes, where there's a supervisor thread and all I want is a visual progress reassurance. Is it acceptable to the project though or should I be pursuing other approaches to try and get something mergeable?

@nickzoic
Copy link
Contributor Author

No? Yes? Maybe? @Tishj this just seemed like the simplest way but if you hate it I can go back and try to get one of the other approaches working instead.

@Mytherin Mytherin merged commit 18e06ea into duckdb:main May 10, 2025
88 of 105 checks passed
@Mytherin
Copy link
Collaborator

Thanks!

krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 18, 2025
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 18, 2025
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 19, 2025
krlmlr added a commit to duckdb/duckdb-r that referenced this pull request May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0