fix: fall back to English per-key when locale is missing a msgid - #440
Open
jacalata wants to merge 5 commits into
Open
fix: fall back to English per-key when locale is missing a msgid#440jacalata wants to merge 5 commits into
jacalata wants to merge 5 commits into
Conversation
Previously, gettext.translation was called with a single-locale languages list. If the loaded catalog was missing a msgid the raw key was surfaced to the user instead of the English text. The chained locale_options list in set_client_locale only handled catalog *load* failures, not per-key lookups. Chain "en" behind the preferred locale so per-key fallback works. Also: - Add CI workflow to detect drift between committed .mo files and the .properties sources they are compiled from; regenerates and diffs on every PR that touches localization files. Uploads the regenerated .mo files as an artifact on failure so they can be downloaded and committed. - Fix wrong doit task name (combine_property_files -> properties) in bin/i18n/README.md, and update the English-fallback claim to match the new behavior. - Remove duplicate Localization section from contributing.md. - Fix NameError in dodo.py uniquify_file when no lines were discarded. - Gitignore the doit intermediate combined.tmp and codestrings.tmp files.
5 tasks
The check-mo-freshness workflow only installs .[localize], which doesn't pull setuptools_scm; the top-level import made 'doit properties/po/mo' crash with ModuleNotFoundError. setuptools_scm is only used by task_version, so import it lazily inside that task.
Two follow-on issues discovered when the workflow ran in a clean CI env: 1. prop2po.py imports click, which was not in the [localize] extra. Add click as a dep of [localize] so 'pip install .[localize]' is sufficient to run 'doit localize' from scratch. 2. task_properties invoked check_strings.py in build mode, which hard-fails on missing non-English keys. With the English fallback added in this branch, non-en gaps are handled at runtime, so blocking the build on them is counterproductive. Remove enforce_strings_present from the properties task; check-strings.yml already runs check_strings.py in dev mode on every PR as a separate check.
The committed .mo files were out of sync with their .properties sources: ~20 keys per locale were either orphan translations for removed code or missing translations for currently-used code. Regenerated via the check-mo-freshness workflow on Ubuntu and downloaded from the workflow artifact so the bytes match what CI produces.
.po files are gitignored across the repo, so the workflow's `git diff --exit-code -- '**/*.po'` was always a no-op and the `.po` path filter never triggered a run. Trim both. No behavior change; the .mo drift check is what has always been doing the actual work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
localize.py: chainenbehind the preferred locale ingettext.translation(languages=[preferred, "en"])so a msgid missing from the loaded catalog resolves to the English msgstr, instead of surfacing the raw key.check-mo-freshnessCI workflow: regenerates.po/.mofrom committed sources on every PR that touches localization files, fails if the committed.mofiles disagree. Uploads the regenerated.mofiles as a workflow artifact on failure so they can be downloaded and committed straight back.combine_property_files->properties) inbin/i18n/README.md; update the "English fallback" claim to describe what the code actually does now; remove duplicate Localization section fromcontributing.md.dodo.py: fixNameErrorinuniquify_filewhen no lines were discarded (uniques->unique_lines)..gitignore: ignore the doit intermediatecombined.tmpandcodestrings.tmpfiles.Test plan
pytest tests/commands/test_localize.py -v- 8 passed, including two new tests:test_fallback_to_english_when_key_missing_in_locale- fr load, en-only msgid returns the en msgstrtest_key_present_in_locale_uses_locale_translation- fr load, translated msgid returns the fr msgstr (fallback doesn't clobber)check-mo-freshnessand will (deliberately) fail: the committed.mofiles have drifted from their sources. I will download the regenerated files from the workflow artifact and commit them back to this branch, at which point the workflow should go green.Known drift being fixed
A clean
doit localizerun against the current sources produces ~20 key differences from the committed.mofiles across all 11 locales. Roughly:tabcmd.options.addcalculations,tabcmd.status.job_completed, etc.)tabcmd.debug.error_message,tabcmd.error.file_save_failed, etc.)After this PR the committed
.mofiles will be back in sync, and the freshness workflow will keep them there.Related followup: #441 (stop committing generated
.mo/.pofiles entirely; generate at build time).