Look for a bundled Noto CJK font

[tobylane]

Describe what the proposed change does

• Look for a bundled Noto CJK font, or a packager chosen bundled font path or a system font path. The official Noto project doesn't release a suitable single file, but there are some third party builds that do. satbyy has GoNotoKurrent-Bold.ttf, I chose this because the licence on this project is clearer.
  Other font options include KlokanTechNotoSansCJK-Regular.otf from klokantech or GoNotoKurrent-Regular.ttf from satbyy.
• The new priority is user setting > bundled font > system font. Linux users from before this change will have an old user setting by default that is probably worse than Noto for CJK.

Korean preview

Fixes #1901 fixes #823 fixes #1219

[lewri]

For the time being, I don't think we should remove the ability to set font manually. Instead, just let it sit as the default value.

Also these changes probably break the font dialog.

[TheCycoONE]

Prefer not to bundle in the repo without first debating/setting up git lfs. It's sufficient to make it a packaging requirement.

[tobylane]

Users's setting > bundled font > OS font. This means if the user has opened CorsixTH on Linux before, the incomplete OS font will be preferred. Line 138 could only choose the setting if it isn't the path two lines above?

I tried to have it pick up the youngest ttf file in the folder, but FileTreeNode isn't available.

[lewri]

It's worth noting here, ARIALUNI.TTF was a Microsoft Office font (not Windows) and hasn't been included since Office 2016. This would explain why gfx:loadFontFile() never wrote a unicode font to the config.txt since that time on Windows.
https://learn.microsoft.com/en-us/typography/font-list/arial-unicode-ms

Edit: It looks like MacOS now also ships with Noto?

[lewri]

Users's setting > bundled font > OS font. This means if the user has opened CorsixTH on Linux before, the incomplete OS font will be preferred. Line 138 could only choose the setting if it isn't the path two lines above?

I tried to have it pick up the youngest ttf file in the folder, but FileTreeNode isn't available.

Why not make it an explicit setting?

[tobylane]

I think Arial and Noto are the only pan-CJK fonts. Google font search for the first Korean string in the language file, where after the Noto fonts they are Korean or Korean+European only, not including Chinese and Japanese.

Could we have the non-free Arial package recommended or required by linux packages? Ask the packagers to set the other-os line to the location of the font that their distribution installs Arial to? eg https://packages.debian.org/buster/ttf-mscorefonts-installer

Why not make it an explicit setting?

Ideally it isn't a user decision at all, depending on how close we can get to an answer for everyone. Strictly free-only (avoiding debian non-free) users who don't like Noto may be left out.

[lewri]

We can't rely on Arial Unicode any more in Windows. In Mac, it's still okay (it is in the Font Library). I'd support cyco's suggestion to make it a packaging requirement. Worst case is that if a suitable unicode font is missing unicode languages are just hidden until a font is selected.

[Alberth289346]

It should be possible to extract all code points from all strings in a language, and verify that they exist in the font.

When to do that, and what to do if indeed code points are missing are next choices.

[tobylane]

Rebased, and put the two new variables at the top of the file for packagers to edit.

[lewri]

I think part of the idea is to also change our make files to require the font too as an optional dependency. At the least, we'd need to add more directions to the make instructions say we want to use x specific noto bundle.

Remove references/settings about picking a font file. Do we want this option left in?

Did you put this back in now? If we want to encourage Noto you may need to add something perhaps to the config file and/or Wiki we're using Noto as the default, but if your language doesn't show properly for you you can specify another?
Do you know what specific languages we currently support that aren't covered by this bundle?

[tobylane]

Did you put this back in now?

I did, and have updated the title and post.

If we want to encourage Noto you may need to add something perhaps to the config file and/or Wiki we're using Noto as the default, but if your language doesn't show properly for you you can specify another?

I can draft a wiki page. The config file, and perhaps the folder settings tooltip, could say something like leave empty for a font chosen for you? Should the folder settings run Graphics:loadFontFile() when the font setting is cleared to find what it can?

Do you know what specific languages we currently support that aren't covered by this bundle?

So far as I've seen, all are supported by Noto CJK.

[lewri]

Once merged I would suggest adding to the 'How to Compile' wiki page. We can add (yet another :) ) packaging note on release too.

I appreciate this PR is now only a 'look for', but it definitely is going to need its supporting docs where appropriate.

[TheCycoONE]

I feel like we should favour a system noto if that's what we're packaging, over the system arial if available.

This is starting to feel similar to the hospital lookup function.

[lewri]

I feel like we should favour a system noto if that's what we're packaging, over the system arial if available.

This is starting to feel similar to the hospital lookup function.

Arial unicode is heavily deprecated/removed in Windows. Though that isn't (currently) what this PR is about now.

[tobylane]

Debian follows the official project in not combining the font files. We could have the language file give the suffix (JP, KR, SC, TC from the debian page) for the right file for each non-European language select, but I don't like the sound of that.

[TheCycoONE]

I might not hate language specific font files - it would let us cut down on what we package quite a bit.

Does raise the complexity though.

[tobylane]

Strings are loaded after and depend on Graphics. Strings can load the font file suffix and make the switch later, but that means setting the font twice.

NotoSansCJK-VF.otf.ttc (33mb) is another good option, but not packaged.

[lewri]

@tobylane are #1901 and #823 resolved or partially resolved at all by this PR? If so, could you edit the summary so they point respectively?

[tobylane]

Updated, also #1219.

[lewri]

Is this global? The path given is likely only *nix global. It seems little odd why you isolated this path though from the other OS paths.

[tobylane]

It's for *nix packagers to change, I can make that more clear here or in the wiki. It's likely that *nix packagers will put a font at a global path and Windows packagers will put a font at a local path.