Add l10n helper script #2607

KJhellico · 2025-06-06T14:33:16Z

Proposed change

Add l10n helper script - tool to simplify the localization process.

Actually, I just wanted to do export/import from .po to .csv for easy spreadsheet editing, but then I went further.

Recommended operating cycle (for a newly created country)

Make sure that default_language and supported_languages are specified in code.
Run make l10n or python scripts/l10n/generate_po_files.py (to create .pot file).
Run python scripts/l10n/l10n_helper.py MM --create-po --overwrite (MM is the country code hereinafter). This will create .po files for all supported languages. If you don't need .csv files, you can stop here and just work with .po files using your usual methods. For en_US locale, translations will be retrieved from msg comments (!).
Run python scripts/l10n/l10n_helper.py MM --export-csv. This will create holidays/locale/csv/MM.csv file. You can edit it in any way you like. If you plan to use MS Excel, it's advisable to run python scripts/l10n/l10n_helper.py MM --export-csv --bom to generate a file in encoding "UTF-8 with BOM" that Excel prefers.
(Optional step) Run python scripts/l10n/l10n_helper.py MM --fill-missing (with --bom, if you created .csv with it). This will try to fill in missing translations with available variants from the corresponding languages (matching is based on msg comment).
Edit .csv file.
Run python scripts/l10n/l10n_helper.py MM --import-csv (with --bom, if you created .csv with it). This will update .po files with your translation from .csv.

I will appreciate any comments and suggestions.

Type of change

New country/market holidays support (thank you!)
Supported country/market holidays update (calendar discrepancy fix, localization)
Existing code/documentation/test/process quality improvement (best practice, cleanup, refactoring, optimization)
Dependency update (version deprecation/pin/upgrade)
Bugfix (non-breaking change which fixes an issue)
Breaking change (a code change causing existing functionality to break)
New feature (new holidays functionality in general)

Checklist

I've followed the contributing guidelines
I've successfully run make check, all checks and tests are green

coderabbitai · 2025-06-06T14:33:34Z

Summary by CodeRabbit

New Features
- Introduced a command-line tool for managing localization files related to holiday data for different countries.
- Added options to create translation files, export and import translations via CSV, and fill missing translations using data from other countries.

Summary by CodeRabbit

New Features
- Introduced a command-line tool for managing localization files related to holiday data.
- Added support for creating translation files, exporting and importing translations via CSV, and auto-filling missing translations from other countries' data.

Walkthrough

A new script, l10n_helper.py, has been introduced to manage localization files for holiday data by country. It provides command-line operations to create .po files, export/import translations via CSV, and fill in missing translations using data from other countries. The script leverages the polib library and standard Python modules.

Changes

File(s)	Change Summary
scripts/l10n/l10n_helper.py	Added a command-line utility for localization management, including `.po` file creation, CSV export/import, and filling missing translations.

Possibly related PRs

Update .po files generator #2470: Updates the existing .po files generator script to improve .po file handling; both PRs involve .po file processing but focus on different tools and functionalities within localization workflows.

Suggested labels

l10n

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 41e15a5 and 8198c5b.

📒 Files selected for processing (1)

scripts/l10n/l10n_helper.py (1 hunks)

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: PPsyrius
PR: vacanza/holidays#2537
File: holidays/countries/finland.py:195-199
Timestamp: 2025-05-09T18:34:33.990Z
Learning: In the holidays library, localization (l10n) is handled through separate .po files for different languages rather than combining multiple translations in a single string. The code should use the default language with tr() function, and translations are provided in language-specific .po files.

Learnt from: KJhellico
PR: vacanza/holidays#2607
File: scripts/l10n/l10n_helper.py:86-88
Timestamp: 2025-06-06T15:23:42.307Z
Learning: The scripts/l10n/l10n_helper.py script is an internal localization helper tool intended for use by developers who understand the codebase, so additional safety checks may not be necessary.

scripts/l10n/l10n_helper.py (2)

Learnt from: KJhellico
PR: vacanza/holidays#2607
File: scripts/l10n/l10n_helper.py:147-150
Timestamp: 2025-06-06T15:22:39.909Z
Learning: In the holidays project localization system, all .po files are generated from the same .pot template for all supported languages, ensuring that all msgids exist for all languages. This makes defensive programming for missing language keys in po_data[msgid] unnecessary in the normal workflow.

Learnt from: KJhellico
PR: vacanza/holidays#2607
File: scripts/l10n/l10n_helper.py:167-167
Timestamp: 2025-06-07T11:09:09.067Z
Learning: In scripts/l10n/l10n_helper.py, CSV files are structured with exactly 3+ columns: msgid, msgcomment, and at least one language column. The structure is controlled by the export process and guarantees minimum column count.

🪛 Pylint (3.3.7)

scripts/l10n/l10n_helper.py

[convention] 1-1: Missing module docstring

(C0114)

[error] 25-25: Unable to import 'polib'

(E0401)

[convention] 30-30: Missing class docstring

(C0115)

[convention] 68-68: Missing function or method docstring

(C0116)

[refactor] 68-68: Too many local variables (21/15)

(R0914)

[error] 70-70: Unable to import 'holidays.registry'

(E0401)

[convention] 70-70: Import outside toplevel (holidays.registry.COUNTRIES)

(C0415)

[warning] 81-81: Using possibly undefined loop variable 'module_name'

(W0631)

[convention] 120-120: Missing function or method docstring

(C0116)

[convention] 148-148: Missing function or method docstring

(C0116)

[convention] 158-158: Missing function or method docstring

(C0116)

[convention] 165-165: Missing function or method docstring

(C0116)

[convention] 187-187: Missing function or method docstring

(C0116)

[refactor] 187-187: Too many local variables (18/15)

(R0914)

[refactor] 187-187: Too many branches (13/12)

(R0912)

[convention] 223-223: Missing function or method docstring

(C0116)

⏰ Context from checks skipped due to timeout of 300000ms (1)

GitHub Check: Test build on windows-latest

🔇 Additional comments (7)

scripts/l10n/l10n_helper.py (7)

18-27: Imports and constants look good.

The imports are well-organized and appropriate for the localization functionality. The WRAP_WIDTH constant is clearly defined.

🧰 Tools

🪛 Pylint (3.3.7)

[error] 25-25: Unable to import 'polib'

(E0401)

31-67: Constructor setup is solid.

The argument parsing and validation logic works well. Good use of sys.exit(1) for clean program termination.

120-147: CSV export logic is well-structured.

The method correctly extracts translations from .po files and organizes them for CSV output. Good separation of default language handling.

🧰 Tools

🪛 Pylint (3.3.7)

[convention] 120-120: Missing function or method docstring

(C0116)

148-164: CSV helper methods are clean and focused.

Both methods handle their responsibilities well with proper encoding support and consistent delimiter usage.

🧰 Tools

🪛 Pylint (3.3.7)

[convention] 148-148: Missing function or method docstring

(C0116)

[convention] 158-158: Missing function or method docstring

(C0116)

165-186: Import logic is efficient and safe.

Good optimization to only save .po files when actual changes are detected. The entry lookup and update logic is solid.

🧰 Tools

🪛 Pylint (3.3.7)

[convention] 165-165: Missing function or method docstring

(C0116)

187-222: Complex but well-structured translation filling.

The method handles a cohesive task of filling missing translations from other countries' data. While complex, the logic flow is clear and the implementation is sound.

🧰 Tools

🪛 Pylint (3.3.7)

[convention] 187-187: Missing function or method docstring

(C0116)

[refactor] 187-187: Too many local variables (18/15)

(R0914)

[refactor] 187-187: Too many branches (13/12)

(R0912)

223-235: Clean execution interface.

The run method provides a clear dispatch mechanism for different actions. Good separation of concerns.

🧰 Tools

🪛 Pylint (3.3.7)

[convention] 223-223: Missing function or method docstring

(C0116)

✨ Finishing Touches

📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Explain this complex logic.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai explain this code block.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and explain its main purpose.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

codecov · 2025-06-06T14:35:41Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (043f748) to head (8198c5b).
Report is 9 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff            @@
##               dev     #2607   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          228       229    +1     
  Lines        14533     14571   +38     
  Branches      2038      2039    +1     
=========================================
+ Hits         14533     14571   +38

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

coderabbitai

Actionable comments posted: 9

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7618b69 and f36be0d.

📒 Files selected for processing (1)

scripts/l10n/l10n_helper.py (1 hunks)

🧰 Additional context used

🧠 Learnings (1)

📓 Comm 10000 on learnings

Learnt from: PPsyrius
PR: vacanza/holidays#2537
File: holidays/countries/finland.py:195-199
Timestamp: 2025-05-09T18:34:33.990Z
Learning: In the holidays library, localization (l10n) is handled through separate .po files for different languages rather than combining multiple translations in a single string. The code should use the default language with tr() function, and translations are provided in language-specific .po files.

🪛 Pylint (3.3.7)

scripts/l10n/l10n_helper.py

[convention] 1-1: Missing module docstring

(C0114)

[error] 26-26: Unable to import 'polib'

(E0401)

[convention] 31-31: Missing class docstring

(C0115)

[refactor] 63-63: Consider using 'sys.exit' instead

(R1722)

[convention] 69-69: Missing function or method docstring

(C0116)

[refactor] 69-69: Too many local variables (25/15)

(R0914)

[error] 71-71: Unable to import 'holidays.holiday_base'

(E0401)

[convention] 71-71: Import outside toplevel (holidays.holiday_base.HolidayBase)

(C0415)

[error] 72-72: Unable to import 'holidays.registry'

(E0401)

[convention] 72-72: Import outside toplevel (holidays.registry.COUNTRIES)

(C0415)

[refactor] 81-81: Consider using 'sys.exit' instead

(R1722)

[warning] 83-83: Using possibly undefined loop variable 'module_name'

(W0631)

[convention] 125-125: Missing function or method docstring

(C0116)

[refactor] 128-128: Consider using 'sys.exit' instead

(R1722)

[convention] 153-153: Missing function or method docstring

(C0116)

[refactor] 156-156: Consider using 'sys.exit' instead

(R1722)

[convention] 164-164: Missing function or method docstring

(C0116)

[convention] 171-171: Missing function or method docstring

(C0116)

[convention] 193-193: Missing function or method docstring

(C0116)

[refactor] 193-193: Too many local variables (18/15)

(R0914)

[refactor] 193-193: Too many branches (13/12)

(R0912)

[convention] 229-229: Missing function or method docstring

(C0116)

🔇 Additional comments (2)

scripts/l10n/l10n_helper.py (2)

171-192: Implementation looks good!

The import logic correctly handles translation updates and only saves when changes are detected.

🧰 Tools

🪛 Pylint (3.3.7)

[convention] 171-171: Missing function or method docstring

(C0116)

229-238: Clean dispatch implementation!

The run method provides a clear entry point for different operations.

🧰 Tools

🪛 Pylint (3.3.7)

[convention] 229-229: Missing function or method docstring

(C0116)

scripts/l10n/l10n_helper.py

coderabbitai

Actionable comments posted: 5

♻️ Duplicate comments (1)

scripts/l10n/l10n_helper.py (1)
70-70: 🧹 Nitpick (assertive)

Use safer path manipulation for sys.path.

Using sys.path.append() with f-string is less clear than using sys.path.insert(0, str(Path.cwd())) for adding the current directory to the Python path.
-sys.path.append(f"{Path.cwd()}")  # Make holidays visible.
+sys.path.insert(0, str(Path.cwd()))  # Make holidays visible.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f36be0d and 1a3546f.

📒 Files selected for processing (1)

scripts/l10n/l10n_helper.py (1 hunks)

🧰 Additional context used

🧠 Learnings (1)

📓 Common learnings

Learnt from: PPsyrius
PR: vacanza/holidays#2537
File: holidays/countries/finland.py:195-199
Timestamp: 2025-05-09T18:34:33.990Z
Learning: In the holidays library, localization (l10n) is handled through separate .po files for different languages rather than combining multiple translations in a single string. The code should use the default language with tr() function, and translations are provided in language-specific .po files.

🪛 Pylint (3.3.7)

scripts/l10n/l10n_helper.py

[convention] 1-1: Missing module docstring

(C0114)

[error] 26-26: Unable to import 'polib'

(E0401)

[convention] 31-31: Missing class docstring

(C0115)

[convention] 69-69: Missing function or method docstring

(C0116)

[refactor] 69-69: Too many local variables (25/15)

(R0914)

[error] 71-71: Unable to import 'holidays.holiday_base'

(E0401)

[convention] 71-71: Import outside toplevel (holidays.holiday_base.HolidayBase)

(C0415)

[error] 72-72: Unable to import 'holidays.registry'

(E0401)

[convention] 72-72: Import outside toplevel (holidays.registry.COUNTRIES)

(C0415)

[warning] 83-83: Using possibly undefined loop variable 'module_name'

(W0631)

[convention] 125-125: Missing function or method docstring

(C0116)

[convention] 153-153: Missing function or method docstring

(C0116)

[convention] 164-164: Missing function or method docstring

(C0116)

[convention] 171-171: Missing function or method docstring

(C0116)

[convention] 193-193: Missing function or method docstring

(C0116)

[refactor] 193-193: Too many local variables (18/15)

(R0914)

[refactor] 193-193: Too many branches (13/12)

(R0912)

[convention] 229-229: Missing function or method docstring

(C0116)

⏰ Context from checks skipped due to timeout of 300000ms (2)

GitHub Check: Test docs build
GitHub Check: Build distribution

scripts/l10n/l10n_helper.py

coderabbitai

Actionable comments posted: 3

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1a3546f and 261f5b9.

📒 Files selected for processing (1)

scripts/l10n/l10n_helper.py (1 hunks)

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: PPsyrius
PR: vacanza/holidays#2537
File: holidays/countries/finland.py:195-199
Timestamp: 2025-05-09T18:34:33.990Z
Learning: In the holidays library, localization (l10n) is handled through separate .po files for different languages rather than combining multiple translations in a single string. The code should use the default language with tr() function, and translations are provided in language-specific .po files.

Learnt from: KJhellico
PR: vacanza/holidays#2607
File: scripts/l10n/l10n_helper.py:86-88
Timestamp: 2025-06-06T15:23:42.307Z
Learning: The scripts/l10n/l10n_helper.py script is an internal localization helper tool intended for use by developers who understand the codebase, so additional safety checks may not be necessary.

scripts/l10n/l10n_helper.py (1)

Learnt from: KJhellico
PR: vacanza/holidays#2607
File: scripts/l10n/l10n_helper.py:147-150
Timestamp: 2025-06-06T15:22:39.909Z
Learning: In the holidays project localization system, all .po files are generated from the same .pot template for all supported languages, ensuring that all msgids exist for all languages. This makes defensive programming for missing language keys in po_data[msgid] unnecessary in the normal workflow.

🪛 Pylint (3.3.7)

scripts/l10n/l10n_helper.py

[convention] 1-1: Missing module docstring

(C0114)

[error] 25-25: Unable to import 'polib'

(E0401)

[convention] 30-30: Missing class docstring

(C0115)

[convention] 68-68: Missing function or method docstring

(C0116)

[refactor] 68-68: Too many local variables (21/15)

(R0914)

[error] 70-70: Unable to import 'holidays.registry'

(E0401)

[convention] 70-70: Import outside toplevel (holidays.registry.COUNTRIES)

(C0415)

[warning] 81-81: Using possibly undefined loop variable 'module_name'

(W0631)

[convention] 118-118: Missing function or method docstring

(C0116)

[convention] 146-146: Missing function or method docstring

(C0116)

[convention] 157-157: Missing function or method docstring

(C0116)

[convention] 164-164: Missing function or method docstring

(C0116)

[convention] 186-186: Missing function or method docstring

(C0116)

[refactor] 186-186: Too many local variables (18/15)

(R0914)

[refactor] 186-186: Too many branches (13/12)

(R0912)

[convention] 222-222: Missing function or method docstring

(C0116)

⏰ Context from checks skipped due to timeout of 300000ms (2)

GitHub Check: Test build on ubuntu-latest
GitHub Check: Test build on macos-latest

scripts/l10n/l10n_helper.py

arkid15r

Huge improvement 👍
I'd like to suggest some improvements and options to check:

scripts/l10n/l10n_helper.py

sonarqubecloud · 2025-06-10T12:09:19Z

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

arkid15r

LGTM 👍

// I have some naming improvement suggestions that can be addressed separately later.

PPsyrius · 2025-06-10T16:53:48Z

@arkid15r I think I misclicked and accidentally disabled auto-merge - please enable them again 🥲

PPsyrius

LGTM, should be quite useful with the eventual l10n file unification in v1 👍

Add l10n helper script

f36be0d

KJhellico requested review from arkid15r and PPsyrius as code owners June 6, 2025 14:33

github-actions bot added the script label Jun 6, 2025

coderabbitai bot reviewed Jun 6, 2025

View reviewed changes

Use sys.exit

1a3546f

coderabbitai bot reviewed Jun 6, 2025

View reviewed changes

KJhellico added 2 commits June 7, 2025 13:49

Refactor

9bd5ff7

Merge branch 'dev' into add-l10n-helper

261f5b9

coderabbitai bot reviewed Jun 7, 2025

View reviewed changes

scripts/l10n/l10n_helper.py Show resolved Hide resolved

scripts/l10n/l10n_helper.py Outdated Show resolved Hide resolved

scripts/l10n/l10n_helper.py Show resolved Hide resolved

Refactor

41e15a5

arkid15r reviewed Jun 10, 2025

View reviewed changes

PR suggestions

8198c5b

arkid15r enabled auto-merge June 10, 2025 16:00

arkid15r reviewed Jun 10, 2025

View reviewed changes

PPsyrius disabled auto-merge June 10, 2025 16:53

PPsyrius approved these changes Jun 10, 2025

View reviewed changes

arkid15r added this pull request to the merge queue Jun 10, 2025

Merged via the queue into vacanza:dev with commit 0287cbc Jun 10, 2025
33 checks passed

KJhellico deleted the add-l10n-helper branch June 15, 2025 08:55

Uh oh!

Add l10n helper script #2607

Add l10n helper script #2607

Uh oh!

Conversation

Proposed change

Recommended operating cycle (for a newly created country)

Type of change

Checklist

Uh oh!

Uh oh!

Summary by CodeRabbit

Summary by CodeRabbit

Walkthrough

Changes

Possibly related PRs

Suggested labels

Chat

Support

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

Documentation and Community

Uh oh!

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Quality Gate passed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!