Tech Meeting Notes 2021 01 14
Mickael: ~3 years in the past, a contributor, Loic, advocated for i18n support. Loic started the SecureDrop Discourse forum to help with coordinating outreach to translators and other interested contributors.
Today, Localization Lab helps to coordinate translation workflows. We use self-hosted Weblate ( https://weblate.securedrop.org ) for managing translations. Weblate displays the strings to be translated. The strings to be translated are looked up by gettext in the application. Translators can discuss changes within Weblate, and ask fro feedback.
John: Recent versions of Weblate provide reporting capability. We can use this tooling to get an overview about how support for different languages is coming along. We can also generate credits for translators for inclusion in our release announcement (we used to compile that information manually). There's more we could be doing, of course. Erik's tool has helped in keeping the screenshots up to date. It's possible to provide more context, such as screenshots, for translators to aid them.
Earlier today, John suggested that we try to coordinate with other groups, particularly Localization Lab, when scheduling our upcoming releases. Right now LL is saturated with translation requests for other projects, such as Signal.
Allie: How does gettext work? How do the strings get into Weblate?
John: We use Python tooling for gettext, to generate a .pot file. The .pot file declares what translators need to work on, the total list of all strings that should be localized. Once the messages are extracted into a .pot file, Weblate can ingest that .pot file and populate the forms for translators for each language.
Weblate can detect whether a string needs to be re-translated between releases. If the reference string in the .pot file has changed, then the strings in Weblate will automatically be marked as in need of translation.
Allie: Where does the i18n tooling to manage the .pot file live?
John: The i18n_tool.py script. The translation information is stored in a fork of the "securedrop" repository, called "securedrop-i18n". We use a separate repo for managing translations because the Weblate server commits changes based on edits by translations in the web interface. We don't want the Weblate server, or the translators, to be able to inject strings directly to the primary SecureDrop git repo, thus the fork. A SecureDrop maintainer will visually view the i18n changes.
I'd also like to talk about "continuous localization". In that setup, Weblate would open PRs for us, and we could review throughout a sprint, rather than all at once during the QA period for a given release. That may be worth setting up.
Erik: How do branches work between the two SD repos?
John: We merge the develop branch from current SD repo into the securedrop-i18n repo at the beginning of the release cycle. That gets us started with the release workflow. Only the "i18n" branch is used in the securedrop-i18n repo.
Erik: Right. That way, the translators get credit, via github, for their translations, which are commited to the "securedrop-i18n" repo.
Mickael: Two suggestions for improvements: 1) keep the translators engaged, by raising coordination; and 2) investigate continuous localization. Any other suggestions for improvement? Or, can anyone identify other shortcomings, even without solutions?
Conor: John, can you explain in detail what you know about coninutous localizatoin?
John: It's in the Weblate docs. We could still use a separate repo, but I don't see why we would. Rather than batching up the translations in a separate repo, then merging them wholesale into the primary repo, the Weblate server would make PRs against the primary repo as translations are made. It's possible that string churn would disrupt translation workflows, if we modify the same string several times in a sprint.
Erik: Not so concerned about string churn. Mostly we debate strings as part of PR review. Once a PR is merged, the strings it changed are unlikely to change again in that same release. It could be quite beneficial to see the localization string changes landing in the demo website (https://demo.securedrop.org) as it happens. That would potentially be much more engaging for translators.
Mickael: Great point. As far as the demo website goes, it's being built from the i18n repo. Since we only update that repo as part of the release period, it's mostly static rather than continuous.
John: That's right, if we used continuous localization, we could build the demo from the SD "develop" branch and translators would see the updates much more frequently. Currently that's on a nightly rebuild schedule. However it's worth pointing out that according to LL folks, the SD release rhythm has been helpful. It's predictable, and we try to give lots of notice so translators can plan to make time.
As an LM, it's easy to feel a little disconnected. I myself sometimes too. It's invaluable to have someone like Erin coordinating the group of translations. It'd be even better if we had increased contact with the translators.
Erik: After 1.7.0 release, let's reach out to LL and chat about how we can do better. They may have concerns to raise with us, e.g. via the release schedules mentioned earlier.
Mickael: Our developer docs explain the i18n workflows, and specific tooling involved: https://docs.securedrop.org/en/stable/development/i18n.html Perhaps a diagram of the high-level process would be useful.
John: Yes, a diagram would be great. We do the i18n steps infrequently, only during release. It'd be nice to brush up on the docs based on our current procedures. Idea: since Allie is asking such great questions, maybe she could be LM for next release? I feel we'd surface more improvements, particularly in the docs.
Allie: Definitely. The docs are a bit confusing. I wasn't sure where the authoritative LM docs were, but I can work from the i18n URL shared above.
Erik: It's confusing, for sure. There's "internationalization" and "localization" terminology in the docs. Some of the docs are for maintainers, others intended for translators.
Mickael: Difference between i18n and l10n?
Erik: Localization is more than just translating a bunch of strings. For example, technical changes are required to support right-to-left languages (cf. English is left-to-right).
John: Also formatting for specific string types, like timestamps, or filesizes. Mozilla has some good resources to clarify. We should update our docs to be more clear.
Erik: Let's clean up the docs. As written they're potentially confusing both to maintainers and contributors.
Conor: Please dump any high quality resources that you find re: i18n vs. l10n
- https://www.w3.org/International/questions/qa-i18n
- https://wiki.localizationlab.org/index.php/Developer_Resources
Erik: How useful are the translation "checks" in weblate?
John: Useful! It can detect situations like translated interpolated variables. We've seen those checks be flaky, in fact the problem went away but it's back now, I'll look into it. Despite the false alarms, they're pretty handy to have, because the brace check will prevent someone from translating the variable name (which has happened).
Erik: Here's an example: HOTP secrets are 40 characters long - you have entered {num}. The variable name "num" should not be translated! Weblate can detect that now, which is handy.
Allie: But aren't there cases where we'd want those variable names to be translated?
John: In those cases, we use "ngettext" rather than "gettext". That can help with pluralizations, for instance (e.g. "1 submission" vs "2 submissions").
Mickael: Time check, nearly at time. Recapping action items:
- Keep translators engaged, let's reach out to LL and have a chat.
- sounds like continuous localization and keeping a demo server up-to-date with the latest is a way that could keep translators more engaged
- Clean up i18n docs -> diagrams!
- Investigate continuous localization (Weblate submits PRs, we ditch securedrop-i18n repo).
We didn't cover this, but I'm curious: are there other or better checks we can run on the strings? - https://docs.weblate.org/en/latest/user/checks.html