Handling I18N

[agentgt]

There are a lot of techniques to handle internationalization (I18N).
JStachio currently does not ship with an opinionated way to handle it.

Perhaps ship with a lambda that does it.

[agentgt]

Mainly a note to myself but I followed the same i18n technique JMustache uses in spring-comparing-template-engines:

jreijn/spring-comparing-template-engines@5d0bbe1

And it worked rather well.

[agentgt]

More notes for myself:

The trickiest part to i18n for JStachio is parameterization of the i18n message.

I do have an example doing mustache.java style: https://github.com/jstachio/jstachio/tree/main/test/examples/src/main/java/io/jstach/examples/i18n

But what I'm looking into is compile time checking of the property keys similar to this library:
https://santhosh-tekuri.github.io/jlibs/i18n/Internationalization.html

However that still has the problem of parameterization and how Mustache lacks a way to call something with more than one parameter. Mustache.java gets around this problem with dynamic templates produced by the lambda where the template is the i18n message from the locale specific bundle.

[agentgt]

@sviperll @dsyer

Sorry guys for the ping but I wanted to run something by you to solve the i18n problem.

I'm thinking of adding a compiler extension point which may not be public at first.

The issue is we can't solve the i18n like other Mustache implementations which have a dynamic runtime context stack (stack being where you are opening sections/lambdas like {{#someSection}}. Furthermore ideally i18n message properties access are compile time checked.

So I'm thinking of adding what I call static lambdas that have their processing done at compile time.

For i18n we would emulate sort of thymeleafs syntax.

{{#i18n}}home.welcome({{session.user.name}}){{/i18n}}

The contents of that section are sort of a DSL equivalent to: https://www.thymeleaf.org/doc/tutorials/3.0/usingthymeleaf.html#messages.

Where home.welcome is the bundle key and parens are used to parameterize the message.

Thoughts?

[dsyer]

So the mustache.java style example uses newlines as a parameter separator? And it doesn't evaluate mustache expressions in the parameter values? Kind of icky with the newlines, and no expressions in the parameters is probably a show stopper, so I can see why you might be thinking of something more along the lines of Thymeleaf.

The "spring-comparing" sample doesn't have any messages with parameters? These are easy to implement I guess and not likely to be hard to use. But that's missing an important feature.

What I'm not sure about is how much of a language extension the static lambda idea is. I suppose lambdas can do anything they like in mustache, but it would be better to use a pattern that someone else invented if there was one. If the parameters had names would it be more "mustache"? (I know that isn't normal for messages, but I just wondered.)

Also, I'm not wild about the name "i18n" for the lambda. I guess that's not part of the language right, so I could define it the same as you and call it "message"?

[sviperll]

Sorry for taking so long to respond. I thought about i18n before in the context of jstachio and I thought that we do not need to introduce any new templating features. What we need to introduce is some way to introduce static context.

So more specifically, my original plan was to have something like:

@Jstash(template = "{{i18n.welcome}}{{name}}", static context="messages.properties")
record User(String name) {}

So i18n.welcome is just a normal template placeholder, not more special than name. What jstashio does is that it construct context not only from the "model" object, but also from the given properties file.

Now, when I look at your example, I think that maybe context manipulation is not enough and

{{#i18n}}home.welcome({{session.user.name}}){{/i18n}}

Should actually be

{{<i18n.home.welcome}}{{$name}}{{session.user.name}}{{/name}}{{/i18n.home.welcome}}

Here i18n.home.welcome is just a simple template-partial. What jstashio needs is not a new syntax, but a new way to resolve partial names. So that partials can be read from some properties-file, like

i18n.home.welcome=Hello, {{$name}}John Doe{{/name}}
i18n.home.bye=Good bye, {{$name}}John Doe{{/name}}

I feel like having new partials-resolution feature is better than having new fully isolated i18n feature, because such partials-resolution can be used not only for i18n and also can be easier to learn and to explain, because it interacts with other, already existing features.

(I've written this on mobile, so sorry for typos)

[agentgt]

Yes Dave and I were talking about a similar idea over here #343 (reply in thread)

My concern is JMustache compat and Locale formatting of locale specific objects like currency and date.

I was able to easily make JMustache work with the mini DSL described in the discussions:

https://github.com/jstachio/jmustache/tree/fragment_name_lookup

Sorry I’m on mobile as well

[dsyer]

I like Victor's samples, and I would be prepared to sacrifice number and date formats (JStachio already has defaults for that doesn't it?). But I do like Adam's suggestion that "normal" property-based bundles (with {0} style placeholders) could simply be parsed into partials. That would cover 90% of what I usually need. Maybe {{$name}} could be converted dynamically to {{$0}} based on the fact that it is the first parameter to be named?

[agentgt]

The issue in my mind is that we are commandeering a syntax not used for formatting but templating.

The difference is subtle but there is difference.

My fear is that will confuse beginners just trying to learn the newer parent partial syntax but the bigger issue is that JStachio at the moment basically expands all code paths and does not have a runtime parsing/executing capability. I think @sviperll might have forgot that.

That is sure we can parse the default message.properties bundle and turn those into templates which gets expanded into a block of Java code (not a method but a block... this is important).

If we have 10 languages we would have to expand 10 times and the message properties cannot be changed at runtime which is not a good idea. Language bundles can become gigantic and while the default aka fallback is okay to be static I don't think all of them should.

Thus if we did provide a Mustache version of property bundles we would need a runtime Mustache implementation like JMustache.... or we could just use the builtin MessageFormat and not use Mustache which is not a language designed for formatting.

But Dave (@dsyer) is right that if we are willing to live with no formatting I could in theory create what appears to be special parent partial but understand is not remotely a parent partial.

Here is why:

{{<@message.someKey}}{{$0}}{{user.name}} some other text{{$/0}}{{/@message.someKey}}

Without a Mustache runtime I have to expand {{user.name}} some other text eagerly as a String which gets passed to some magic i18n method. That is generated code but the resulting translation needs to be dynamic. Thus the contents of someKey absolutely cannot have any Mustache syntax as we cannot expand.

Thus @message.someKey is not a mustache template.

Finally Dave had originally brought up:

but it would be better to use a pattern that someone else invented if there was one.

And that pattern historically has been lambdas and not parent partials albeit the syntax on passing parameters inconsistent.

I looked around and the other patterns that are not lambda but Handlebar-esque. This includes Trimou and obviously Handlebars.java. Those guys work because the allow something like:

{{blah arg1 namedArg2="blahLiteral"}}

aka helpers.

Overwhelmingly I think the most consistent syntax in our ecosystem is Thymeleaf and hence why I like it most.

Here is an example of the usage in JMustache which is fully functioning (albeit needing better edge case and better parsing): https://github.com/jstachio/jmustache/blob/fragment_name_lookup/src/test/java/com/samskivert/mustache/LocaleLambdaTest.java

[sviperll]

I think we have a disagreement about the way the feature should work. My idea which maybe not ideal, but I think it works for typical (in my imagination) web development assumes that there are not many different languages, i. e. 3 or maybe 5, but not 10. And I've assumed that we want the presence of translations to be statically checked, i, e, compiler will check that Spanish-bundle has translations for all messages. With these assumptions I think it's reasonable to have 3-5 per-language different compiled Java-class-files for each template.

From what @agentgt has written, I think he has different assumptions in mind and he assumes more dynamic nature of this feature.

In this situation I think we can discuss

is static-i18n feature as I have described has any merits and should we leave space for it in the future or should the dynamic i18n be the only available mechanism.

Talking of the dynamic i18n, I think @agentgt is right that the correct feature to use for this purpose are lambdas (or probably dynamic parent partials). And the problem is that jstachio doesn't support mustache-lambdas.

So the way to go forward is

either to wait or start implementing dynamic-lambdas (or dynamic parent-partials) or
otherwise emulate some built-in lambda or parent partial without waiting for full support.

And I think that's what @agentgt is proposing, to have i18n that can be a lambda when we have dynamic lambdas, but since we have no at the time, it is a special built-in thing, that can become real lambda in the future.

I think to properly design this, we should think about the namespace issue.

Should the special lambda be called i18n or should there be some special namespace for possible future built-in lambdas. Maybe it should be jstachio.i18n and maybe we will also introduce jstachio.formatDate later.

But I personally like the @dsyer 's suggestion. I think it's better to emulate special-partial of some kind (that is rendered dynamically) to some special-lambda, because we do not need to introduce another sublanguage.

I do not agree that

contents of someKey absolutely cannot have any Mustache syntax as we cannot expand.

Surely for dynamic-i18n we need some run-time support and I don't see home MessageFormat is better runtime than JMustache for our purposes. The argument against mustache is that

not use Mustache which is not a language designed for formatting

I wouldn't say that Mustache should never be used for formatting, but for me even if Mustache is not ideal for formatting, having to use one language for both templating and formatting is better than having two completely different languages.

[agentgt]

From what @agentgt has written, I think he has different assumptions in mind and he assumes more dynamic nature of this feature.

Yes. This is based off experience with my company but not all companies/software are the same.
You see there isn't 10 languages. There is 10 translations. In my company the "bundle" is stored in a database and site/org can configure the messaging. So there are actually 1000s if not more (I should know the stat to that but I don't. We also might support more than 10 languages but I don't think so). Often times the messaging is adjusted not even for locale reasons but just marketing reasons.

So in some ways I can totally see why @dsyer doesn't like the term i18n because it isn't that. It is more about messaging at least for my scenario.

And I think that's what @agentgt is proposing, to have i18n that can be a lambda when we have dynamic lambdas, but since we have no at the time, it is a special built-in thing, that can become real lambda in the future.

I think to properly design this, we should think about the namespace issue.

Surely for dynamic-i18n we need some run-time support and I don't see home MessageFormat is better runtime than JMustache for our purposes. The argument against mustache is that

Ultimately I agree that MessageFormat sucks (it is also very slow) but the reality is that it is already done. The JMustache implementation was a couple of lines of code to make it happen. The JStachio implementation of the simple syntax will probably be more than JMustache but I feel it will largely be less than parent block parsing even if the tokenizing is already there.

What I'm hoping is the work just to get MessageFormat and compiler extension in JStachio opens up the door to figuring out more Mustachey solutions. Furthermore I'm working with the Mustache Spec on what we are calling Filters. Filters should/could open the door to some level of formatting but they are not in the spec yet so I'm hesitant to make a special language one more complicated than the Thymeleaf message function I have to parse even if some of the plumbing of parsing is already present.

So yeah this is largely a stop gap but I do think formatting aka messaging aka sort of i18n is largely runtime even if there is some level of static checking the defaults.

EDIT sorry forgot to address this:

I do not agree that

contents of someKey absolutely cannot have any Mustache syntax as we cannot expand.

Perhaps you can see given my previous explanation of who is editing these translations how it can be dangerous security wise allowing what essentially is full access to the context stack.

That is it is not trusted developers writing or translating these "mini templates". And even the baked in default supported languages I don't think most are going to proof read. That is what I meant about formatting being different is that it doesn't have looping, including (eww that is another scary one), and branching. MessageFormat while flawed is actually safer.

However I agree we could in the future provide perhaps some subset but the necessary mechanics of providing that are similar to support MessageFormat.