Closed
Description
cc @daikeshi -- we were discussing how much code duplication there is between our Scala/Scio-based Dataflow Task and his Python/Beam-based Dataflow Task, mostly in the Dataflow argument passing and some input/output URI parsing. I think it would make sense to contribute a generic DataflowTask to the contrib module of Luigi that we can extend for different SDKs.
What do you think? @daikeshi and I could probably work together to contribute this.
(fo 5878 r reference, here's the python/beam task: https://github.com/spotify/spotify-tensorflow/blob/master/spotify_tensorflow/luigi/python_dataflow_task.py)