Text shaping & Sitelen Pona

[pontaoski]

(background context)

I and many others have been working on a proposal for an ideographic script called Sitelen Pona in Unicode. Part of this work has been defining an encoding for two of Sitelen Pona’s most distinguishing features from other logographies: extended glyphs and cartouches. Extended glyphs are glyphs that are extended to “reach below” other glyphs, usually helping to disambiguate grammatical groupings of words.

The following is an example of long pi, the most common word to write extended in Sitelen Pona. (It is the L-shaped glyph going below the others.)

jan pi toki pona li jan pi pona mute
(“a toki pona person is a very good person”)

Cartouches surround names transcribed in Sitelen Pona.

ma [ kasi alasa nasin awen telo a ] li suli

Cartouches and other extended glyphs can be nested within extended glyphs. (The proposed encoding technically allows for nesting cartouches, but this isn't attested in usage.)

jan pi ma [ o suli uta ]
(“person of the land of Osu”)

jan pi ma pona pi toki pona
(“person of the good place of toki pona”)

This is pretty straightforward from an encoding perspective, effectively being nothing more than nestable start/end markers.

(the problem)

Problems arise from actually implementing these features in fonts. Most current fonts use a simple approach: hardcoded permutations for different scenarios. For example, the glyph “pona” may have the following variations in a font: base, base with pi underline, base with cartouche, base with reverse ala underline, base with forward ala underline, etc. The main technical issue with permutations is a lack of flexibility to accommodate all of the attested usages of Sitelen Pona, as writers combine extended glyphs and cartouches in uncountably many ways, and even a finite subset can push up against OpenType’s glyph limit, if not completely exceed it.

(the requirements of a solution)

From what I understand, what’s necessary for “100% accurate” Sitelen Pona is:

• the ability to reposition glyphs with respect to other glyphs (we have multiple GPOS tables to potentially use for this, but I understand that application compatibility regarding whether or not to apply these is iffy)

  This would be used to build extended glyphs/cartouches by e.g. having a “pi.line” glyph that glyphs are positioned on top of, and that can be stacked to build glyphs as deeply nested as desired:

  

• something to deal with variable heights that depend on what exactly is enclosed by an extended glyph or cartouche
  

My assessment is that whilst implementing positioning of glyphs within nested extensions/cartouches is doable with current OpenType capabilities, implementing variable-height glyphs that are sized with respect to adjacent glyphs is not currently possible and would need something like Beyond Emulation’s Wasm table or Graphite in order to implement correctly.

(the inquiry)

So, my inquiry is less of a question (because I'm not even sure what questions to ask) and more of a request for a sanity check from people that know this stuff way better than any of us Sitelen Pona users. Does this assessment of what is and isn’t possible and how make sense, or am I missing something/reading something incorrectly?

[plokmijnuhby]

For the first problem, it seems like GPOS tables are supported, if you use the right feature. Certainly harfbuzz will always support them, and other applications seem to as well.
Variable height glyphs are trickier. One option is to split the cartouche edges into seperate glyphs for the top, bottom, and middle of the edge, and repeat the middle section as needed - you'd also need complicated GSUB and GPOS setups to glue everything together.

[behdad]

You don't necessarily need two version of each glyph (one mark, one nonmark). You can simply add back the advance width to whichever glyph should carry it.

[tiroj]

Or you could always have an invisible spacing background glyph and treat everything else as an anchored mark, but I suspect you might still want separate glyph sets for nesting levels, e.g. smaller forms of letters to fit within nested situations within cartouches, shorter vertical height cartouche segments for nested cartouches, etc.. I have a feeling that for this kind of layout I might just want the convenience of working with separate glyph sets within each nesting level, rather than juggling GPOS everywhere.

[plokmijnuhby]

Ideally, these come from the plain text encoding itself, rather than needing to be contextually arrived at in the glyph processing.

That's one idea, although that wouldn't be consistent with how sitelen pona is currently used. The height of each glyph could be determined by something in the encoding.

[pontaoski]

the current UCSUR encoding for extended glyphs is glyph extend-start … extend-end, and the encoding for cartouches is cartouche-start … cartouche-end and this is the mechanism currently being proposed to unicode ATM

[tiroj]

The height of each glyph could be determined by something in the encoding.

That’s probably not needed, or useful. Determining the relative size and position of things is where glyph level processing is appropriate, and trying to manage aspects of that in the encoding—by which I mean character-level—may introduce restrictions that later get in the way of flexible solutions and specific typeface designs, as well as being unlikely to get past Unicode’s technical reviewers.

Where a cartouche begins and ends is semantic content, and I think something similar can be said of the extension of the long pi. So it makes sense for that to be captured in the plain text of the character encoding, and is then useful to glyph processing by providing delimiters to some specific kinds of substition and positioning operations in the font.