Issues with core:text/i18n.parse_mo_from_bytes(): 1. it doesn't know about key contexts

[mgavioli]

Context

Odin: dev-2024-05-nightly:2250eb3e7
OS: Linux Mint 21.3, Linux 5.15.0-106-generic
CPU: Intel(R) Core(TM) i5-6400 CPU @ 2.70GHz
RAM: 15913 MiB
Backend: LLVM 17.0.6

Background

.MO file entries may contain contexts (i.e. disambiguation strings, one for each entry, possibly lacking and typically different from entry to entry). This context is encoded before the first key string (keys may have 1 or 2 strings), separate from it by a 0x0004 byte.

Expected Behaviour

The procedure shall know about these context strings and treat them appropriately. I see 2 possible ways:

1. sub-optimal, but 'quick and dirty' transient solution: remove it and simply ignore it. Currently the programmer has no way to specify the context in the i18n.get() calls (and it is hard to expect he care to chain the context string + a 0x0004 byte + the key string in the i18n.get() call), this would ensure call key and .MO key at least match.
2. better, definitive solution, but requiring to change the API: add a context parameter in the i18n.get() calls and matching context + sting if the call to context + string in the parsed i18n.Translation.

Please suggest about which one to choose and about implementation details!

Note that the concept of disambiguation string is present also in the Qt Linguist .qm file and a similar issue probably exists for them too. However, as I am not familiar with them, advise is welcome here too.

Current Behaviour

The procedure simply treats the context + '0x0004` + key string found in the .MO file as a single string.

Failure Information (for bugs)

If contexts are present, in a call to i18n.get() a key with context is not found and its translation is not returned.

Steps to Reproduce

1. Compile the provided main.odin file in the same directory as the 2 provided .mo files (1 without and 1 with contexts)
2. Run the resulting executable:
3. when looking in the no_context.mo the translation is found, when looking in the context.mo the translation is not found.

Note: the attached ZIP file contains the main.odin source code, two .mo files and the .po files from which they are compiled via PoEdit.
i18n_context_bug.zip

[mgavioli]

@Kelimion : thanks for looking at this. When you deal with this issue, you may look also at this discussion which may also involve API changes in the i18n.get() family of calls.

[Kelimion]

Btw, I implemented the MO parser referencing https://www.gnu.org/software/gettext/manual/html_node/MO-Files.html, without looking at any of their code (to avoid GPL contamination). But I had no example files with a so-called context.

So how should it work? Suppose we have the same key, with and without "Context".

msgctxt "Context"
msgid "Message1"
msgstr "This is message 1 with context"

msgid "Message1"
msgstr "This is message 1 without context"

I imagine that you want get("Message1") to return This is message 1 without context.
And if you supply the context, then it'd return This is message 1 with context?

This could use get("section", "key") mechanism where any key with a context uses that context string as the section name, and entries without a context end up in the "" section.

[mgavioli]

Thanks for implementing this. Sorry to be late: apparently Github didn't notify me of the update.