Open
Description
Feature description
I would like to parallelize all sources in Airflow, including the first one in the list, PR incoming.
Are you a dlt user?
Yes, I run dlt in production.
Use case
The first source in a pipeline can take a long time to complete, right now in both parallel modes the first source is ran alone, this is time that could be spent running multiple sources.
Proposed solution
Using a DummyOperator at the start, instead of splitting the first source and branching after it has run.
Related issues
No response
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
Todo