8000 Reconcile underscore/dash config style handling (#2688) by soxofaan · Pull Request #2691 · spotify/luigi · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Reconcile underscore/dash config style handling (#2688) #2691

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 23, 2019
Merged

Reconcile underscore/dash config style handling (#2688) #2691

merged 1 commit into from
Apr 23, 2019

Conversation

soxofaan
Copy link
Contributor
@soxofaan soxofaan commented Apr 8, 2019

As discussed in #2688:
Underscore config style is recommended Luigi, but luigi.task.Config also helpfully allows dash style in configuration files (with a deprecation warning).

In a lot of places in contrib modules however, configuration.get_config().get() is used, with dash-style e.g. in the spark contrib module:

    return configuration.get_config().get(self.spark_version, "deploy-mode", None)

configuration.get_config().get() does not provide the dash/underscore translation feature, so one is forced to use deploy-mode in their config file. Using the seemingly recommended deploy_mode alias will not work. This leads to inconsistently styled and confusing configuration files.

This PR reconciles the underscore/dash handling of luigi.task.Config and configuration.get_config().get(), so that both use the same logic: both underscores and dashes are allowed, but dash-style triggers a DeprecationWarning.

Have you tested this? If so, how?

Added some tests about this too.

@soxofaan
Reconcile underscore/dash config style handling (spotify#2688)
8cd0ccc 
Reconciles handling of underscore-style (recommended)
and dash-style (deprecated but allowed) in both
`luigi.task.Config` and and `luigi.configuration.get_config().get()`.
@soxofaan soxofaan force-pushed the issue-2688-config-styles branch from be8c7ef to 8cd0ccc Compare April 8, 2019 19:10
dlstadther
dlstadther approved these changes Apr 13, 2019
View reviewed changes
Copy link
Collaborator
@dlstadther dlstadther left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another great improvement!

@Tarrasch Tarrasch merged commit aee2864 into spotify:master Apr 23, 2019
@Tarrasch
Copy link
Contributor

Thanks @soxofaan. Quality and correctness is what luigi needs the most I think. :)

@soxofaan soxofaan deleted the issue-2688-config-styles branch August 7, 2019 07:18
@soxofaan soxofaan mentioned this pull request Aug 7, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Tarrasch Tarrasch Tarrasch approved these changes

@dlstadther dlstadther dlstadther approved these changes

@honnix honnix Awaiting requested review from honnix

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@soxofaan @Tarrasch @dlstadther
0