Allow translations of BTCPay Server Backend by admins

[NicolasDorier]

Abstract

Sourced from #5468

To achieve our goal of widespread adoption of BTCPay Server by local communities, it is crucial to focus efforts on translating BTCPay. As it stands, only the checkout pages are translated, implying that BTCPay merchants must know English. This requirement significantly hinders local adoption.

The proposed solution should possess these characteristics:

• Enable administrators to modify or add translations during runtime.
• Utilize standard types such as IStringLocalizer, IHtmlLocalizer, or annotations for implementation.
• Facilitate the easy sharing of translations.
• @woutersamaey has already contributed to this endeavor, as seen in Basic localization support #972, where he enabled these interfaces to be supported by a JSON file along with dedicated tag helpers.

We can reasonably presume that the backend operates in a specific language, which can be designated in the server's policies.

Translations should be stored in the database and loaded into memory during runtime.

UX

This PR introduces backend translation capability in BTCPay Server for administrators. A new tab in server settings presents a large text field for updating translations, formatted as [English] => [Translation]. The simplicity of this UI facilitates easy modification and sharing of translations. A dedicated GitHub repository will host all translations, linked directly from this page.

Translatable strings are primarily located under Server Settings -> Translations.

Administrators can effortlessly import translations from other people into the text area and share their translations.

I plan to add a link to a github repository later so the community can share translations.
Prior to major release, we should probably show case a few languages translated through ChatGPT in such repository.

Localizable content

Developers have multiple options for making page content translatable:

• <label asp-for="Property"> in views are automatically translatable if the target property has a [Display] or [DisplayName] attribute.
• A text-translate="true" tag helper in views marks tag content as translatable. This content should be text-only, as HTML is not currently supported.
• In Views' code blocks, a ViewLocalizer object is injected for use as ViewLocalizer["My sentence"].
• Controllers can use IStringLocalizer, ViewLocalizer, or IHtmlLocalizer.

Implementation

The system employs standard MVC interfaces like IStringLocalizer, ViewLocalizer, and IHtmlLocalizer, customized to meet the abstract's outlined requirements. Translations differing from default English versions are stored in the database, ensuring immediate reflection of localized string updates for English-speaking administrators (unless overridden).

A translations table has been added with key and value columns.

Default translations

Ensuring discoverability of translatable strings is vital. These strings are predefined in Translations.Default.cs.

To ease maintenance, a UpdateDefaultTranslations script functions similarly to PullTransifexTranslations. This script, which requires periodic execution, updates hardcoded assets by scanning *.cs files for Display and DisplayName usage, and .cshtml files for ViewLocalizer or text-translate tag helper use. Detected translations are then incorporated into Translations.Default.cs.

It is possible for developer to manually edit Translations.Default.cs to add more strings.

Out of scope

• Translations.Default could be modularized into a plugin format. This would enhance the discoverability of translations for plugins.
• The UI lacks a straightforward dropdown menu for language selection. This design choice is based on the assumption that the interface is infrequently used and that administrators possess the technical proficiency to navigate to a GitHub repository link for translations.
• Currently, there is no integration with the Greenfield API. While this could be considered for future implementation, the practical benefits of such integration are yet to be fully ascertained.
• text-translate should be used in other parts of the source code, however, we can do this gradually rather than everything in this PR
• @woutersamaey make a taghelper on Basic localization support #972 which allow the translation of arbitrary attributes of tags (such as tooltip=), we should probably do this later too.

[woutersamaey]

Great work @NicolasDorier Thanks for picking up my (very old) work.

Some things I'd still like to see are:

• More than 1 language, because users can be international and not just from 1 region.
• A locale setting in the user's profile, which controls not only language, but also number format and date/time notation (AM/PM vs 24h time, which is still an issue for me since very long).
• Use the Accept-Language header for the user's preferred language if no locale is set in his profile, like we do with the checkout. The locale setting could default to "auto", which takes this header into consideration.
• Bundled translations that can update along with BTCPay (so you can easily get the latest & greatest translations without having to jump though hoops). We risk a lot of poor or partially translated installs if the admin needs to copy-paste himself.

[NicolasDorier]

The choice of a single language is because the goal of this PR is to improve local adoption in Bitcoin.

The expectation is that a Bitcoin ambassador onboard local businesses or friends on their local instance, and set the proper language for them so their first time experience with BTCPay is less intimidating. We aren't attempting to improve the life of a "international" host.

Secondly, I am also not comfortable to put translations in merchant level, because at some place, we will want to allow HTML injection. (For example if the string to translate contains bold letters in some part)
Thirdly, some parts are also not tied to a user or a store. (Login screen for example)

This mean that there would be a need for different screen to update the "User level strings" and the "Server level strings". This makes things more complicated for everyone. It makes also make things weird, as then the Server Level strings may be in one language, but the user level one in another language.

This will result in screens mixing two languages.

A locale setting in the user's profile, which controls not only language, but also number format and date/time notation (AM/PM vs 24h time, which is still an issue for me since very long).

For the locale, the formatting should be done by the browser already. Apparently chrome take the data from the format set at the OS level? We should probably copy what the other websites do (AFAIK they do like us, just use the browser's settings). But that should be a separate PR.

Bundled translations that can update along with BTCPay

So if you are using English it's already the case. In other language it's more difficult, as we shouldn't override something that the user edited.

If we did this, this would give us some serious UX dilemna in the following case:

1. The admin edit the translation directly to BTCPay (either because he fixed a typo, or because the language wasn't existing in the bundle beforehand)
2. Then one day, we release a the language in a bundle: Should we override the strings from the admin? It isn't clear we should, because the admin took time to translate things and we don't want to lose his work. But on the other hand, it means he isn't using bundle's latest and greatest.

[woutersamaey]

The choice of a single language is because the goal of this PR is to improve local adoption in Bitcoin.

Makes sense. Just consider if we have an easy upgrade path after this, or if it would be major refactoring down the line? If the latter, might make sense to have basic structure already in place. At the bare minimum, I think we need to add a dropdown to indicate which language is entered in the textarea. If we don't know the language, we cannot upgrade later to multi-language because we won't know what was entered.

The expectation is that a Bitcoin ambassador onboard local businesses or friends on their local instance, and set the proper language for them so their first time experience with BTCPay is less intimidating.

If we bundle 1 language (i.e. Spanish) + provide a way to load others, it wouldn't be intimidating at all.

I am also not comfortable to put translations in merchant level, because at some place, we will want to allow HTML injection. (For example if the string to translate contains bold letters in some part).

The way I solve this typically, is by allowing some tags + escape everything else. Example: the translated output is escaped, except for <strong> and <a> tags. This means updating templates + careful consideration which tags you want to allow and which one's not to allow. In practise I use 2 groups: single line text can have <strong> + <a> and multiline text can have more like <h1>, <ul>, <li>, etc, so there's a whitelisting approach.

An alternative solution is to allow markdown in some places. Markdown is highly predictable and can be done safely.

Some parts are also not tied to a user or a store. (Login screen for example) This mean that there would be a need for different screen to update the "User level strings" and the "Server level strings". This makes things more complicated for everyone.

Why can't all translations be in 1 place, only accessible by the admin? The merchant is just a user and has to use the translations set by the admin.

For the locale, the formatting should be done by the browser already. Apparently chrome take the data from the format set at the OS level? We should probably copy what the other websites do. But that should be a separate PR.

Bundled translations that can update along with BTCPay

So if you are using English it's already the case. In other language it's more difficult, as we shouldn't override something that the user edited.

This will trigger weirdness in other language:

1. A language isn't translated in bundled form already, so the admin edit the translation directly to BTCPay.
2. The bundled form is released: Should we override the strings from the admin? It isn't clear we should, because the admin took time to translate things and we don't want to lose his work.

The situation is the same if the admin edit the translation in BTCPay to fix a typo: When BTCPay update the translation in the bundle, maybe the sentence is a bit different, but we can't really override what the admin did, as we don't want to lose his work.

At the end of the day, there are too much edge case to consider for bundling without good solution. As such, I think simple is better and shouldn't spend too much time doing something too elaborate.

This is not an issue at all if we consider the bundled translations as defaults + use the textarea as a means to override specific lines. Most people don't want to bother with translations and will just use whatever is bundled or contibute via Transiflex. If a translation is missing or particulaly troublesome, they can override.

[woutersamaey]

If you feel bundling translations is risky, we can always make it "opt-in", so nobody is forced into this...