Proposal: metadata for static localization (we need your input!)

[boutell]

Right now, when it comes to static localization — which is to say, phrases that are built into your templates or our admin UI that you want to be able to translate in JSON files — a particular module can only contribute to one namespace.

This felt good at first but it's not a good fit sometimes, especially for modules that use our "improve" feature to add custom UI to pieces, for instance. It could also be bad for modules used as base classes for other modules.

If we set the "i18n" option like this in a module like @apostrophecms/piece-type-exporter:

module.exports = {
  improve: '@apostrophecms/piece-type',
  i18n: {
    ns: 'apostrophe'
  }
};

And a project developer wants their own phrases in the default namespace, for their project:

module.exports = {
  improve: '@apostrophecms/piece-type',
  i18n: {}
};

... Then we have a collision. 🚗🚗 💥

Both use cases are valid but the project level module's setting for the i18n option wins and the phrases intended for the apostrophe namespace get stuck in the default namespace. Code that tries to find apostrophe:somePhrase for translation will not find them.

We have a solution in mind, and part of it is easy. Here is the easy part:

• Any module folder, including an "improvement," can just put .json files in a subdirectory like i18n/apostrophe and they will automatically get contributed to that namespace.
• There is no need for an i18n option at all, but for bc you can still set it, to determine what namespace any .json files placed directly in the i18n subdirectory of a module should wind up in.
• For convenience, if you don't specify the i18n option at all and put .json files in the i18n subdirectory of your module, they wind up in the default namespace.

That's the easy part! But now here's the part where we need your opinion because there is one more setting that needs a home.

Some i18n namespaces need to be pushed to the browser for the admin UI (Vue), and some do not. We don't push everything to the admin UI because that would hurt performance.

So we need a place to declare "hey, this namespace belongs in the browser!"

But, we don't want to go right back to the problem of fighting over the right to set the i18n option in an "improve" module, or the original module, or a project level piece type module... et cetera.

We have thought of two solutions. Please let us know which you prefer.

Solution one: metadata.js

The first solution is to have an optional metadata.js file in each localization subdirectory, i.e. right in i18n/apostrophe if that folder exists for a particular module.

This looks like:

modules/my-module/i18n/apostrophe/en.json
modules/my-module/i18n/apostrophe/fr.json
modules/my-module/i18n/apostrophe/metadata.js

And metadata.js could contain:

{
  browser: true
}

Since this file is specific to this folder, Apostrophe can understand it is not setting an option that can be overwritten. It is saying what to do with the JSON files in this folder, no matter what subclasses choose to do with their own localizations.

Solution two: i18n module section

So that's one way. Here's the other way: a new section in index.js.

This would not be an option, it would be outside the options section, so Apostrophe understands it should treat it in a special way, allowing each index.js in the inheritance tree to set the metadata for its own localized JSON files:

// node_modules/@apostrophecms/piece-type-exporter/index.js
module.exports = {
  improve: '@apostrophecms/piece-type',
  i18n: {
    apostrophe: {
      browser: true
    }
  }

In this solution, the JSON files are still organized in namespace subdirectories, but rather than introduce metadata.js in those subdirectories, we keep any metadata about the localizations in a new i18n module format section, broken down by namespace.

Which is best?

Honestly, we don't know for sure. Internally we don't have strong opinions on it. But maybe one of these solutions is clearly better and we're missing it.

An advantage of the metadata.js file is that it clearly groups that file with the .json files that honor it. An advantage of using a module format section is that it's more consistent with how we do everything else, and also translators who are asked to localize a folder of JSON files won't see a .js file in there and get confused.

What do you think? Either way we're going to move ahead next iteration, because we do need to start localizing the UI of modules like @apostrophecms/piece-type-exporter that require a solution to this problem.

Just to be clear, either way the solution is 100% backwards compatible and you can ignore the metadata stuff if you're not shipping admin UI phrases.

Thanks for your thoughts!

[myovchev]

It's a lot of information and I need a day or two to chew it. BUT I already have a real life issue related - both partially and directly - with the proposed changes so I'll need to dig a bit deeper into the problem. At the time of writing this I'm absolutely about introducing a dedicated i18n module configuration property rather than introducing file metadata - it feels apostrophe way or in other words - right.

Meanwhile, I need some details in order to be sure I understand exactly the proposed general concept.

Just to be sure I'm on the same page. For the sake of simplicity I'll ask in the context of the second proposed solution - a configuration property.

1. Is it true that i18n.apostrophe represents a localization namespace apostrophe so that apostrophe:somePhrase will be searched through all modules (merged localization object) having i18n/apostrophe/en.js giving the fact current locale is en?
2. Is it true that a module that have options.i18n.ns = mymodule and also a root configuration i18n.mymodule_ui.browser = true would be able to translate mymodule:* without browser export (the localization sources live in mymodule/i18n/*.json and mymodule_ui:* with a browser export (the localization sources live in mymodule/i18n/mymodule_ui/*.json?

[boutell]

1. I think the answer is yes, I'm not sure why you put a . in that, but the rest is exactly right. It will search through all module folders, even those of improve modules, as long as they are present in the inheritance chain of at least one active module.
2. options.i18n will be completely "legacy only, not for new work" after this change, but supported for those who are already placing localizations for a namespace other than default directly in the i18n subdirectory of their module.
3. For simplicity, an entire namespace will either wind up in the browser or it won't. So if even one module folder specifies browser: true for a namespace, that namespace will go to the browser in its entirety. I don't see a situation where this couldn't be resolved by using a separate namespace for strings that are not worth sending to the browser, which receives them for admin UI only (so far at least).

[myovchev]

Thanks for the clarification Tom.
And a quick clarification on my side. I've used dots . in my examples to represent the module configuration objects paths.
The i18n.aposrophe represents the corresponding path from

  i18n: {
    apostrophe: {
      browser: true
    }
  }

So we are on the same page :)

The more I think the more I'm convinced the option 2 (module configuration) - it feels even more comfortable than current option based configuration. If I understand it right, now a module can introduce multiple namespaces and this is very good thing IMO. My only concern is browser: true part. While I completely agree it's easy solvable it creates distraction and yet another thing the developer should be careful about. And also there are two perspective to think of - public modules and project level internal modules.

And while I can take care of my internal modules and smart split the localization so that only needed for admin UI phrases end up with browser: true, there is no guarantee that 3rd party public module will follow/be aware of this. As a developer it's much more convenient for me to export the whole module localization in the browser instead to create a bundle just to be able to split it due to performance reasons. We are lazy :)

About the internal project level modules and my real world example - I've created a smr_i18n module, a central repo for shared project global localization, namespace smr:*. It was my first static localization attempt (in fact I was getting rid of the phrases containing the : according to the bug you are aware of) and things wasn't looking good until I realized I need browser: true. Done and done. After that I've realized I hate the thought of having localization sources spread through all my internal modules. I'm freak when it comes to code single responsibility, business logic splitting yada yada, but localization is another animal. Keeping it all together would be smart for future automation, l10n tooling, etc. It just feels right (on a project level). It's even better with your current proposed change as it would let me have e.g. the field labels tied to a module, but translated in the central smr_i18n repository and everything l10n related will live nice together. However, the story ends when I've read this proposal and realized I've pumped up the central localization repo with all kind of non admin UI stuff and it's all browser: true. And while this isn't a big issue at the moment, I can imagine it becomes one for really l10n heavy projects.

So to summarize, my only additional proposition in case it doesn't bring hell on your heads is demonstrated via this example, using the real world naming used above (details in the comments of the example)

// modules/smr_i18n/index.js
// ...
  i18n: {
    // let the core decide
    apostrophe: { },
    // this module namespace, no browser export
    smr: {
      browser: false
    },
    // this module namespace, browser export
    smrui: {
      browser: true
    },
    // internal or 3rd party modules - let them decide if they should browser export;
    // we are just translating them/fixing typos/changing labels here
    module1: {},
    module2: {},
    module3: {},
  },
// ...

I can see this working well with more complex public bundle modules as well. The logic you've described about having once browser export for a given namespace, it should be exported, no matter what is still in tact here. Now modules are basically only keeping the l10n schema telling me what I'm able to localize, a smart tooling would diff their schema with the smr_i18n and take care of adding new phrases, warn about deprecated or renamed keys, etc.

If I was about to split browser and non-browser localization in two separate modules, e.g. smr_i18n and smr_i18n-ui, I would create unnecessary dependency/knowledge for i18n modules. For example smr_i18n-ui knows module1 exports to the browser so it is adding it to its namespace list. If this is to be changed one day and for some reason module1 doesn't export to the browser anymore, a change has to be made on 3 places - module1 (change export type), smr_i18n (add namespace), smr_i18n-ui (remove namespace).
The only downside of my proposition would be duplicating common phrases, but I think this is more strategic than technical decision.

I'm sorry for the long post, just shared my thoughts as promised :) I'm not aware about the complexity of implementation, which is a primary factor when taking decisions.

[boutell]

Hi Miro,

I think having two namespaces, "myns-ui" and "myns", in order to put one set into the browser and leave the other out is reasonable because you would never use that naming convention unless it was your intention to put those phrases in the browser.

So if five different modules/improvements/etc. sometimes want to inject into the browser and sometimes do not, that's not a problem because they can contribute phrases to both namespaces, or just one, or neither as they see fit.

[pietro-rutzen]

Learning how to get around apos went along with my experience with development in general and I must say I would much prefer the module approach because it is that much easier to grasp what’s going on by just looking at it.

As a junior developer, any concept that deviated from a more module-oriented perspective or that required files that are too specific to approach a situation would have a learning curve that requires that much more cognitive effort.

If there are no technical drawbacks in making it module-oriented, I would go for that.

I believe this also helps developers from outside the apos community to spend less time reading docs and more time taking advantage of the functionality that is offered.

[Miselrkba]

The team and I would like to see solution 2 also as it's more consistent with what is there now.

[egonzalezg9]

I agree with everyone here, solution 2 feels consistent with apostrophe.