8000 Fix #2743 by removing NotImplementedException in CreateUnionPipeline by kryonix · Pull Request #6789 · duckdb/duckdb · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix #2743 by removing NotImplementedException in CreateUnionPipeline #6789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Mar 23, 2023
Merged

Fix #2743 by removing NotImplementedException in CreateUnionPipeline #6789

merged 5 commits into from
Mar 23, 2023

Conversation

kryonix
Copy link
Contributor
@kryonix kryonix commented Mar 20, 2023

This pr fixes #2743.

Some tests:

WITH RECURSIVE t(x, y) AS
(
  SELECT 1, 1
    UNION ALL
  SELECT 1, 2
    UNION ALL
  (SELECT x+1, 1
  FROM   t
  WHERE  x < 10 AND y = 1

    UNION ALL
  
  SELECT x+1, 2
  FROM   t
  WHERE  x < 10 AND y = 2)
)
SELECT * FROM t;

WITH RECURSIVE t(x, y) AS
(
  SELECT 1, 1
    UNION ALL
  SELECT 1, 2
    UNION ALL
  SELECT t, z
  FROM   t, LATERAL
    (SELECT x+1, 2
     WHERE  t.y = 1
      UNION ALL
     SELECT x+1, 1
     WHERE t.y = 2) AS _(t, z)
  WHERE t.x < 10
)
SELECT * FROM t;


WITH RECURSIVE t(x, y) AS
(
  SELECT 1, 1
    UNION
  SELECT 1, 2
    UNION
  (SELECT x+1, 1
  FROM   t
  WHERE  x < 10 AND y = 1

    UNION ALL
  
  SELECT x+1, 1
  FROM   t
  WHERE  x < 10 AND y = 2)
)
SELECT * FROM t;

These queries return the expected result.
Therefore I think, the exception is no longer required and should be removed.

@Mytherin
Copy link
Collaborator

Thanks for the PR! I think this might have been fixed by #4970 which reworked the way pipelines are constructed. Perhaps we could add some tests with unions with more data as a sanity check to verify that this actually works as expected?

@kryonix
Copy link
Contributor Author
kryonix commented Mar 20, 2023

I've added some more tests I constructed a while back. They include a few more data types, different query shapes etc.

@lnkuiper
Copy link
Contributor

Happy to see this is fixed! I could not test union pipelines within recursive CTE's as the release date was close when this was merged, and I didn't get to it later either. This probably also means that "child pipelines", e.g., the full outer scan of a build side of a join, should also work properly within recursive CTE's already (but are currently disabled). I will look into this (hopefully before the next release)

@Mytherin
Copy link
Collaborator

It looks like there is still some non-determinism tripping up the CI - could you have a look?

@kryonix
Copy link
Contributor Author
kryonix commented Mar 22, 2023

I did use some temporary tables. Maybe CI did not like that. Changed them to persistent ones just now. I think that should fix the issue.

@Mytherin
Copy link
Collaborator

You can also run the force storage/force reload tests locally by passing the arguments --force-storage --force-reload

@kryonix
Copy link
Contributor Author
kryonix commented Mar 22, 2023

Good to know! Thanks. Locally the force storage/force reload tests work fine now.

@Mytherin Mytherin merged commit 57cb4bf into duckdb:master Mar 23, 2023
@Mytherin
Copy link
Collaborator

Thanks!

@kryonix kryonix mentioned this pull request Apr 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Regression: RuntimeError: Not implemented Error: UNIONS are not supported in recursive CTEs yet
3 participants
@kryonix @Mytherin @lnkuiper
0