Skip to content

Translation (L18N)

Jump to bottom Edit New page
moving-bits edited this page Dec 25, 2022 · 22 revisions

Translation workflow for cgeo/cgeo

Introduction

cgeo/cgeo uses English as source language. Thus all new strings need to be implemented in English language (e.g. in /main/res/values/strings.xml). For localization into other languages we make use of Crowdin as a collaboration platform for contribution of translations. This integration provides the localized strings back into to the cgeo/cgeo repository (e.g. to /main/res/values-<android_language_code>/string*.xml).

Additionally arbitrary other files of the repository could optionally be included (such as changelogs, etc.) for localization. This is currently used for the translation of the Play Store app description (Source: /main/project/playstore/description/_source/*.txt , Translation: /main/project/playstore/description/translation/<language name>/*.txt).

The Crowdin integration is enabled for branches master and release as while release is periodically synced (fast-forwarded) to the head of master it might still be necessary to use different strings on both branches (e.g. in case a string is removed from 'master' but needs change on release). Furthermore the support of both branches allows to include late incoming translations into a new bugfix release (as these bugfix releases are usually done from release without prior merging of master onto release).

Some strings (e. g.: program names, URI, unit values) should not be translated. Those strings are to be stored in /main/res/values/strings_not_translatable.xml and marked with translatable="false".

This wiki article describes the workflow from/to Crowdin as well as the needed related actions on the cgeo/cgeo repository on Github. The last chapter describes the technical setup of the integration between our Github repository and Crowdin for future reference.

Crowdin project details

Crowdin project homepage

The Crowdin translation project can be found here: https://crowdin.com/project/cgeo

Contributors can create an account on Crowdin (free of charge) or authorize with their Github account to get access. Every Crowdin member can contribute translations to c:geo without further need of approval as c:geo is listed as a public project on Crowdin (refer to role model explained below).

Role model on Crowdin

Role Description Assignment
Translator Default role for all contributors. Translators can provide translations to all languages Assigned to all Crowdin users by default. Single users could be blocked in case of abuse.
Proofreader Proofreader have the permission to validate translations. Need to be manually assigned by the Crowdin project owner. Can be assigned per language (preferred mode) or for all languages.
Owner Owners have full control of the Crowdin project. They can administrate all settings, provide translations, set proofreadings for all languages. Can be assigned/invited by other owners.

Important: As only proofread translations will be contained in the sync workflow from Crowdin (project setting) this is the quality gate for incoming translations into the project. Therefore proofreaders should be carefully selected.

Sync Workflow

For c:geo, Crowdin and GitHub are connected using Crowdin's GitHub Integration plugin. On c:geo side the configuration for this is stored in file crowdin.yml in project root.

For all automated actions described below, Crowdin is configured to use the Github user cgeo-ci-bot.

New source files to Crowdin

The following applies for branches master and release.

New strings added into the files /main/res/values/strings.xml and /cgeo-contacts/res/values/string.xml will automatically be fetched by Crowdin on each commit to the relevant branches of the cgeo/cgeo repository (a max. 10 minute delay might apply). After being fetched by Crowdin the source strings will be visible in the project and available for translation without need of any manual action.

The according configuration is stored in crowdin.yml in the root directory of the master branch and can be changed in case of need to include more files and/or directories. Alternatively it is possible to use an GUI based configuration approach in the Crowdin project settings, which will then push a new crowdin.yml to our repository.

Crowdin will automatically handle duplicated strings between the branches (which is the normal case as the strings on our branches are rather similar) such that duplicated strings will get the same translation (only need to be translated once by contributors on Crowdin) applied to both branches.

New translations from Crowdin

Crowdin will push all new proofread translation to the branches l10n_master / l10n_release of the cgeo/cgeo repository once every 24 hours (configurable in the Crowdin project settings). Individual commits will be created by Crowdin for each file and language changed. Additionally pull requests will be automatically created to merge l10n_master / l10n_release into master / release. Subsequent translations coming in at a later point of time will be added to related l10n_* branch and therefore also added to the existing open pull requests for each branch.

The pull requests needs to be reviewed and merged manually by a member of the team. In order to minimize the amount of commits on our branches, the pull requests should be squashed during merge (Github UI function).

Important: After merging the changes of l10n_* these branches needs to be deleted, to allow Crowdin to recreate it on the next push event based on current state of our head branches and base the commits against the most current state (including the already merged translations). Due to our repository settings the deletion of the l10n_* branches should be done automatically after merging, else it can be done easily via the Github UI directly after merging the pull request (a delete button is shown).

Conflict handling between branches

As we regularly merge changes on release back into master due to our branch and release strategy (see xxx) and translated files might be modified on both branches these merges will result in conflicts. In order to avoid this we implemented the following method: As Crowdin delivers all needed translations to each branch, there is no technical need to merge strings translated on release back into master. Therefore we defined a dedicated "dummy" merge strategy (which does nothing) in our .gitconfig and applied this merge strategy to all translated files in our .gitattributes. If merge conflicts apply this will automatically exclude these files from the merge (they will remain unchanged on the target branch). If no merge conflicts apply the files will however be merged normally.

Important: For security reasons git will not automatically evaluate and use the content of the .gitconfig stored in our repository. All contributors, who directly merge branches on the cgeo/cgeo repository need to execute git config --local include.path ../.gitconfig once for their cloned local repository to allow git to evaluate and use the configuration in .gitconfig.

Manual upload of translation from Github to Crowdin

In order to prevent accidental overwriting of translation contributions, the automated workflow (as described above) does not touch translated strings on Crowdin. In special cases however (e.g. a batch change to existing translations has been done) it might be necessary to back-sync translations from Github to Crowdin. This means, that translated strings existing in our repository shall be uploaded to Crowdin.

This action can be triggered on demand on Crowdin by user with manager role as follows:

  • Open project settings
  • Select Integrations / Github
  • Click on the drop down arrow next to "Sync now" and select "Upload translations"
  • In the following window you have two options:
    • Allow target translation to match source = This should be deactivated, as this will skip translations that match the original strings, as well as translations that are identical to the existing ones during the upload. Checking this option will (potentially) overwrite everything on Crowdin with our repository content (not tested yet).
    • Approve added translations = This should be activated, as only approved translations are delivered via the normal translation workflow later on. Additionally translations coming from the repository can be considered approved.
  • Start the process by selecting "Upload" and give some minutes time to sync.
  • To verify the process select "Sync now" afterwards. There should not be any diff/PR from Crowdin to Github (except unrelated user contributions since last sync) if the process was successful.

Important: On the last occasion, where we tried this procedure, it resulted in problems with our app_description*.txt files due to conflicting auto-segmentation (meaning: some wrong segements were generated in the files, causing repeated or missing text blocks in the files). The mess was cleaned up manually. This is most likely a bug in Crowdin. It needs to be evaluated how this could be avoided (e.g. selective upload, remove files from sync config before uploading).

Translation to new languages

Upon e.g. user request translation to new languages can be enabled as follows:

  1. Enable the target language on Crowdin, so that translation work can be started. Note: It should be checked, whether a remapping of the language code is required as Android might require another code than the standard code, which Crowdin uses. A language mapping can be defined in the Crowdin project settings.
  2. As soon as first translations to the new language are proofread, Crowdin will push them to Github.
  3. As the project rule is, that new languages shall only be included if they have a minimum of approx. 50% translations, the languages to be included are defined in resConfigs in build.gradle. This line needs to be adapted to finally include the new language into the compiled app.

How to setup Crowdin integration

This chapter provides a short description of how the integration between Github and Crowdin was configured for this project for future reference. Individual members/contributors do not need to establish such integration individually.

Prerequisites:

  • Access to a suitable Github user which will later be used to read/write to the Github repository. Ideally this should be a dedicated account for such purposes. (here: cgeo-ci-bot)
  • Access to a Crowdin account, which has owner privileges on the Crowin project (here: Lineflyer)

Actions on Github:

  • In order to start the integration the Github user needs to have owner privileges on the repository. Crowdin will only allow setting up an integration for the repository owner. Therefore cgeo-ci-bot needs to be declared as member of the c:geo organization temporarily until the setup is completed.
  • An access token for the user cgeo-ci-bit has to be generated in the Github user settings (settings: admin:repo_hook, repo). This will be used on Crowdin to establish the communication link.

Actions on Crowdin:

  • The integration can be set up in Project Settings - Integrations on Crowdin. Select the option to use an access token and provide the user access token derived for cgeo-ci-bot.
  • Choose the cgeo/cgeo repository and select the branch master for integration
  • Configure source file patterns (the details can be found in file crowdin.yml which is created based on this input)
  • Configure ignore patterns (the details can be found in file crowdin.yml which is created based on this input)
  • Configure destination file patterns (the details can be found in file crowdin.yml which is created based on this input) using the %two-letters-code% placeholder to separate the languages onto the directories and the placeholder %original-file-name%. Remark: For support of certain language codes and dialects (e.g. Portuguese and Brazilian) a language mapping needs to be configured on Crowdin during this step to map the %two-letters-code% of a specific language to a custom code as expected by Github/Android.
  • Configure XML settings to translate content, but not translate attributes and not use content segmentation.
  • Define the desired push period under advanced settings
  • Save and finalize the configuration
  • Crowdin will automatically commit the needed config.yml to the repository and start synchronizing

Final steps:

  • Check that l10n_master branch, the commits and the pull request are created as soon as there are new (proofread) translations on Crowdin
  • After the integration setup has been completed cgeo-ci-bot has to be again converted to an outside collaborator to remove the admin privileges on Github.

Special tasks

How to move a file

If a crowdin resource file (like strings.xml) is moved within the c:geo structure, crowdin does not understand this automatically. The file must be moved manually in crowdin as well. This is done as follows:

  • Perform the move within c:geo GitHub repository. As part of this move, change the path in configuration file crowdin.yml as well. Check in the move (e.g. by merging into master).
  • In Crowdin, synchronize c:geo project with repository here: https://crowdin.com/project/cgeo/apps/system/github
  • -> now in Crowdin content tree you will find the moved resource two times: one time in the "old" place and one time in the "new" place
  • -> it is important to understand that Crowdin keeps all the translations already done still in the "old" file. The "new" file is a fresh resource file to crowdin without any existing translations yet.
  • In crowdin content tree (https://crowdin.com/project/cgeo/content/files) DELETE the newly created file (or move it to another place at first), then MOVE the original file in the "old" place to the "new" place. Moving files can be done via drag&drop in the Crowdin view.
  • -> now the "new" place has the "old" file associated, together with all existing translations for it
  • In crowdin, synchronize c:geo project again to see if everything worked out
  • -> synchronization may create a Pull Request from crowdin within c:geo. Check what this Pull Request wants to change in c:geo in order to see if all translations are still intact
  • -> If translations seem to be not intact (e.g. Pull Request wants to delete things which are already translated), then check "Restoring translations on Crowdin"

Restoring translations

For yet unknown reasons (please add suggestions if anything is known), Crowdin sometimes "looses" already translated things. This section provides a guide on how to restore translated content if this happened

Restore xml resources (e.g. strings.xml)

You can restore xml resources (especially for strings.xml) from the translated files within c:geo GitHub via Mass Upload. Note that this does NOT work for md files like changelog_xyz.md (see next section for this).

The Crowdin documentation of this process can be found here: https://support.crowdin.com/uploading-translations/#mass-translations-upload

According to the Android structure, the "master" resource is stored in a directory called values whereas translations are stored in directories named values-xy where xy is the country code for the translation (e.g. values-de for german translation). If translation gets lost in crowdin, restore them as follows:

  • From your locally checked out code copy, create a ZIP file which contains the translated files including their directory structure. You can do this by copying all the values-xy directories to a new temporary folder location, then zipping it up. For strings.xml the ZIP-internal structure may look something like this:
main
   src
      main
         res
            values-de
               strings.xml
            values-ell
               strings.xml
            ...
  • In crowdin go to the c:geo project (https://crowdin.com/project/cgeo)
  • Select the Upload button and select the just created ZIP file for upload
  • -> Crowdin needs a while to upload and process the file. It will then show you a list for each contained strings.xml file in the zip and wants you to select the crowdin ressource file and the language to apply it to
  • Check for each line whether the correct crowdin ressource file is selected AND whether crowdin has correctly guessed the right language (usually crowdin does not understand that in is Indonesian and that zh is SImplified Chinese, select this manually)
  • Important: click on the "Settings" Zahnrad at the end of the list and check option Approve added translations
  • Click Import
  • --> the import starts. It takes surprisingly long don't worry
  • In crowdin, synchronize c:geo project again to see if everything worked out
  • -> synchronization will create a Pull Request from crowdin within c:geo (or add to an existing one). Check what this Pull Request wants to change in c:geo in order to see if translations are intact

Restore md files (e.g. changelog_xyz.md)

md files like changelog_xyz.md can't be restored with mass upload. Instead, use Pretranslation via TM. This assumes that translations were done before and are still located in crowdin's TM memory (although they may be gone from the concrete resource file.

  • From crowdin c:geo project page (https://crowdin.com/project/cgeo), select Pretranslate, then select via TM in the menu.
  • In the opening window:
    • Check Target languages (this selects all c:geo target languages at once)
    • Under Files, expand the content tree and check the file you want to restore translations for (e.g. release/main/res/raw/changelog_base.md)
    • As minimum match ratio, check 100%
    • Check Apply for untranslated strings only (in order not to accidentally overwrite existing translations)
    • Check Approve added translations and select All in the DropDown
    • Select Pre-translate
  • After this is done, synchronize c:geo project to see if everything worked out
  • -> synchronization will create a Pull Request from crowdin within c:geo (or add to an existing one). Check what this Pull Request wants to change in c:geo in order to see if translations are intact
Add a custom footer

Information

Development

Usage

Technical documentation

Misc

Outdated:

Clone this wiki locally