[subset] Support for variable fonts with Required Ligatures (rlig)

[yisibl]

The need to subsidize material-design-icons has been very much brought up, and I think it would be better addressed by hb-subset in a unified way.

• Subset icon font google/material-design-icons#1201
• Woff webfont has enormous size google/material-design-icons#1542

Currently, if we want to subset the variable fonts of material-design-icons, the current steps are as follows:

1. Identify what glyphs - not codepoints
   Use hb-shape to get glyph-ids
2. Subset specifying glyph-ids
   instead of specifying unicodes or text, specify --gids

Take the star icon, for example, the gids are:4261,4995,5012,5013,5014

hb-subset input.ttf -o hb.ttf --gids='4261,4995,5012,5013,5014' --no-layout-closure

When using the --no-layout-closure flag, everything is fine, no extra glyphs are generated, but the star.fill glyphs are lost. This causes gaps in the rendering when the FILL axis is 99%.

This can be fixed by adding star.fill, @tphinney explains how this works here.

So here's my request:

1. Improve -no-layout-closure so that it can contain star.fill
2. Do we have an easier way to subset this font? No need to get gids via hb-shape, like adding a new --gids-name flag.

hb-subset input.ttf -o hb.ttf --gids-name="star,close"

With this, maybe the Google fonts API can be used directly as well.

https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:wght,FILL&gids-name=star,close

cc @rsheeter @anthrotype

[garretrieger]

We have a python script that handles subsetting these fonts here: https://github.com/rsheeter/subset-gf-icons/blob/main/src/subset_gf_icons/subset_gf_icons.py

Your description of what you need to do is pretty much the same, but the missing step is that you need to get the glyph closure from only the icon glyphs and not the a-z glyphs (seen here: https://github.com/rsheeter/subset-gf-icons/blob/main/src/subset_gf_icons/subset_gf_icons.py#L106). This can be done in harfbuzz by generating a subset plan and then extracting the set of included glyphs from it (via: hb_subset_plan_create_or_fail and hb_subset_plan_old_to_new_glyph_mapping)

The result of that closure would then be used to form the final subset in combination with the a-z gids and no layout closure.

[garretrieger]

2. Do we have an easier way to subset this font? No need to get gids via hb-shape, like adding a new --gids-name flag.

There's an existing flag that does this: "--glyphs"

[yisibl]

Thanks @garretrieger

I looked at subset_gf_icons, so can we integrate this behavior into hb-subset's command line? This would work out of the box.

For 2, shouldn't it be coupled with a --text flag to achieve something like subset_gf_icons?

Is this right?

hb-subset input.ttf -o hb.ttf --glyphs="star" --text="star" --no-layout-closure

[rsheeter]

Is this right?

No, you need some layout closure because you want the glyphs reachable from the icon glyph, e.g. the star.filled glyph. I don't think the cli will do this out of the box, you would need to write a C++ equivalent of what subset gf icons does in python.

[garretrieger]

Yeah currently the cli can't run the closure for you, that's currently only available via the c api. It's possible we might want to add a new flag to the cli that specifies a set of gids/glyph names on which the closure should run even if no layout closure is set.

That should give all the tools needed to do this from hb-subset cli.

[yisibl]

That's great, I'm going to try to complete a web version of the demo with harfbuzzjs.

I'm really looking forward to eventually having the hb-subset CLI do this with the new flag.

[rsheeter]

imho we would want a new cli as we don't know typically know the glyph names, just the icon names.

I expect eventually the hb-subset CLI will be able to do this with the new flag.

I wonder if it belongs in harfbuzz proper or as a new thing; it's not general purpose which might lead me to think a new thing (that depends on hb) makes sense?

[garretrieger]

imho we would want a new cli as we don't know typically know the glyph names, just the icon names.

I expect eventually the hb-subset CLI will be able to do this with the new flag.

I wonder if it belongs in harfbuzz proper or as a new thing; it's not general purpose which might lead me to think a new thing (that depends on hb) makes sense?

A new cli specific to icon font subsetting also seems reasonable to me. So far this is the only case I've seen where you want a partial closure.

[yisibl]

A new cli specific to icon font subsetting also seems reasonable to me. So far this is the only case I've seen where you want a partial closure.

If tools like nanoemoji support the generation of variable fonts for icons in the future, I think there will be more demand for them.

[behdad]

Another way to think about it is that we want a subset mode that only specific strings are needed to be rendered. Ie. a true --text. I'll think about how feasible that is.

[papandreou]

Another way to think about it is that we want a subset mode that only specific strings are needed to be rendered. Ie. a true --text. I'll think about how feasible that is.

That would be amazing for the subfont use case where we already know all the text that's being rendered, so we can tell up front which ligatures definitely aren't needed.

[behdad]

Another way to think about it is that we want a subset mode that only specific strings are needed to be rendered. Ie. a true --text. I'll think about how feasible that is.

That would be amazing for the subfont use case where we already know all the text that's being rendered, so we can tell up front which ligatures definitely aren't needed.

Indeed. But it's much harder to implement. We are thinking about it though.

[khaledhosny]

Wild idea, shape and only keep glyphs in the final output (instead of GSUB closure). OK, this does not capture intermediate glyphs. A more wild idea, use tracing infrastructure to capture also intermediate glyphs (optionally blank the outlines of these glyphs to reduce file size).

[behdad]

Wild idea, shape and only keep glyphs in the final output (instead of GSUB closure). OK, this does not capture intermediate glyphs. A more wild idea, use tracing infrastructure to capture also intermediate glyphs (optionally blank the outlines of these glyphs to reduce file size).

If we don't keep the intermediate glyphs, the ligating rules won't be retained and this usecase requires those. The tracing infrastructure is too heavy-handed I think. We need to think about a new solution, partly based on that.

[khaledhosny]

Using tracing seemed like something one can test quickly in Python. So I gave it a try (needs harfbuzz/uharfbuzz#177):

import argparse
import uharfbuzz as hb


def main():
    parser = argparse.ArgumentParser(description="Subset a font.")
    parser.add_argument("font", metavar="fontpath", type=str, help="font path")
    parser.add_argument("text", metavar="text", type=str, help="text to subset")
    parser.add_argument("--shape", action="store_true", help="shape the text first")

    args = parser.parse_args()

    blob = hb.Blob.from_file_path(args.font)
    face = hb.Face(blob)

    input = hb.SubsetInput()
    input.flags = hb.SubsetFlags.GLYPH_NAMES
    if args.shape:
        print("Shaping text first")

        font = hb.Font(face)
        buf = hb.Buffer()
        buf.add_str(args.text)
        buf.guess_segment_properties()

        glyph_set = input.glyph_set
        buf.set_message_func(
            lambda msg, buf: glyph_set.update(i.codepoint for i in buf.glyph_infos)
        )
        hb.shape(font, buf)

        glyph_set.update(i.codepoint for i in buf.glyph_infos)
        input.flags |= hb.SubsetFlags.NO_LAYOUT_CLOSURE
    else:
        input.unicode_set.update(ord(c) for c in args.text)

    subset = input.subset(face)
    print(f"{subset.glyph_count=}")


if __name__ == "__main__":
    main()

Shaping does not seem to be significantly slower in my limited test:

$ time python s.py ../../aliftype/raqq/fonts/Raqq.ttf بيت --shape
Shaping text first
subset.glyph_count=15

real	0m0.056s
user	0m0.040s
sys	0m0.013s

$ time python s.py ../../aliftype/raqq/fonts/Raqq.ttf بيت
subset.glyph_count=23

real	0m0.055s
user	0m0.039s
sys	0m0.012s

[behdad]

This is nice! Thanks. Lemme think about it a bit more. Can you perform it on the Material icon-font as requested in this issue?

[khaledhosny]

OK, this does not seem to help here since the desired glyph gets used only conditionally based on variation settings:

$ python s.py MaterialSymbolsOutlined\[FILL\,GRAD\,opsz\,wght\].ttf star
subset.glyph_count=12
['.notdef', 'stars', 'stars.fill', 'atr', 'start', 'rtt', 'star', 'star.fill', 'a', 'r', 's', 't']

$ python s.py MaterialSymbolsOutlined\[FILL\,GRAD\,opsz\,wght\].ttf star --shape
Shaping text first
len(intermediate)=4
subset.glyph_count=6
['.notdef', 'star', 'a', 'r', 's', 't']

[behdad]

Right :(.

[yisibl]

I tried to implement this using harfbuzzjs, not sure if there is any problem. See: yisibl/harfbuzzjs#2