Add additional locale support for Linux/Windows OS images

[tahonermann]

I find it useful to use CE to experiment with locale related programming concerns, but the OS images used by https://compiler-explorer.com lack support for many locales. This is a request to add additional locale support for those OS images.

See https://godbolt.org/z/hr17zcYMn for an example; none of the Japanese or Chinese locales that code attempts to exercise are available.

For Linux, I would find the following locales useful:

• ja_JP.eucjp
• ja_JP.sjis
• ja_JP.ujis
• ja_JP.utf8
• zh_CN.gb18030
• zh_CN.gb2312
• zh_CN.gbk
• zh_CN.utf8
• zh_HK.big5hkscs
• zh_HK.utf8
• zh_TW.big5
• zh_TW.euctw
• zh_TW.utf8

Some of the above locales are not supported by some Linux distributions, but they can still be generated.

I guess Windows locale support doesn't really matter until #1879 is resolved.

[partouf]

We are using Ubuntu 22.04, do you have any suggestions on how to install them? We setup our instances with among others this script https://github.com/compiler-explorer/infra/blob/main/setup-node.sh#L15 in which we do some extra software installation

[tahonermann]

Thanks for the quick response!

The relevant package is locales and it will be installed by default (ubuntu-minimal depends on it).

Most of the locales I mentioned are "supported" ones and they should be present in /usr/share/i18n/SUPPORTED. For those locales, it suffices to uncomment the matching lines in /etc/locale.gen and then run locale-gen. This should suffice for the following locales. Since uncommenting matching lines could be annoying to automate, it should be ok to just append these entries to that file.

• ja_JP.UTF-8
• ja_JP.EUC-JP
• zh_CN.UTF-8
• zh_CN.GB18030
• zh_CN.GBK
• zh_CN.GB2312
• zh_HK.UTF-8
• zh_HK.BIG5-HKSCS
• zh_TW.UTF-8
• zh_TW.EUC-TW
• zh_TW.BIG5

The ja_JP.sjis locale is "unsupported", but still available. I think this is the command I used on my local machine to build it.

• localedef -f SHIFT_JIS -i ja_JP ja_JP.sjis

The ja_JP.ujis locale is an alias of ja_JP.eucJP and should already be present in /etc/locale.alias. I don't think anything needs to be done for it.

If everything works as expected, locale -a should list all of the above locales. It might be useful to add that command at the end of the script to capture the available locales in log output.