Text Outline

[stephencorwin]

I'm trying to figure out how to have a black text outline around white text so that the text can be easily read. I'm not sure, but it sounds like I might need to use a shader to do with with troika-three-text? Can you provide some guidance on how to achieve this effect?

Thanks!

[stephencorwin]

I almost feel like maybe what I need is accomplished by the debugSDF flag, if only I could color it.

Showing the sdf basically gives me an outline that would probably work if I could colorize it, adjust the size, and potentially make it slightly more crisp.

[lojjic]

This shouldn't be too hard to do. I started playing with textOutline as a feature a while ago but never finished it.

In theory, the glyph edges can be expanded simply by changing the value in the signed distance field that corresponds to the edge. That's 0.5 in the current shader code, but that could be made configurable by passing it in as a uniform.

Ideally the shader could handle drawing both the normal glyph plus any outline around it in a single draw call, so you'd need a couple new uniforms: the outline width (not sure what units this would be in?), and the color of the outline area.

[stephencorwin]

I started playing with textOutline as a feature a while ago but never finished it.

I would be VERY happy if it was natively supported :)
However, I'm very new to shaders, so I'm not sure how much help I can be though. I'll be watching this issue closely.

[lojjic]

I got back into this a bit and quickly realized why I hadn't finished it originally. While the SDF can be used for outlining, there are some caveats:

• The maximum potential width of the outline is limited by the extent of the distance field itself (~8 texels in the 64x64 SDF texture).
• That does not correspond to a consistent visual size across all glyphs, since the SDF texture is scaled to each glyph's bounds.

So larger glyphs like "W" can get a decently thick outline, but small ones like periods can only get a very thin outline. And each glyph needs to use a different SDF cutoff point to make their outlines visually consistent.

I don't think this is insurmountable yet, but isn't as simple as I'd thought initially.

[anzorb]

@lojjic, loving this library 🥇, but text outline is a requirement for our project...

I got back into this a bit and quickly realized why I hadn't finished it originally. While the SDF can be used for outlining, there are some caveats:

• The maximum potential width of the outline is limited by the extent of the distance field itself (~8 texels in the 64x64 SDF texture).
• That does not correspond to a consistent visual size across all glyphs, since the SDF texture is scaled to each glyph's bounds.

So larger glyphs like "W" can get a decently thick outline, but small ones like periods can only get a very thin outline. And each glyph needs to use a different SDF cutoff point to make their outlines visually consistent.

I don't think this is insurmountable yet, but isn't as simple as I'd thought initially.

I am curious, can we do some kind of post processing using shaders to achieve an outline and/or shadow effects? Thanks in advance!
Here's the effect I am trying to achieve:

Edit: this is from mapbox gl, I think they're using SDF as well https://blog.mapbox.com/drawing-text-with-signed-distance-fields-in-mapbox-gl-b0933af6f817, maybe we can dissect their code to see what they did. https://github.com/mapbox/mapbox-gl-js

[lojjic]

Thanks for the info about Mapbox, @anzorb. I'll definitely check out their implementation more closely, but from that blog post (they call the outline a "halo") it sounds like they use the SDF alpha threshold approach like I mentioned before. But their SDF generation and atlas packing strategy is different than mine and doesn't have the same issues with non-uniform distance field sizes per glyph. It would probably have the same max outline width limitation though.

I've actually made some progress on screen space outlines using standard derivatives. Screen space meaning the outline would be e.g. 1 screen pixel wide regardless of the text's size/scale or obliqueness. If the goal is just to add contrast with the background, it seems like that might be as good as (or even preferable to) outlines that scale along with the text size. Do you have an opinion on that?

[stephencorwin]

Standardized widths are far more preferred than scaling widths. It would behave more like css in this way too.
Any text-shadow that is applied via css is configured independently from the font-size.
If your proposed solution @lojjic performs the same way, then that would be 🎉 🎉 🎉

[anzorb]

Thanks for your prompt response @lojjic!

The goal is to add contrast, and your proposal looks great. Do you mean the outline will be static? i.e. no way to control it (as @stephencorwin described, using text-shadow-ish approach).

Honestly, I don't see that as a problem, as long as the 1 pixel is visible on super high density displays (because we currently multiply the font-size by device pixel ratio to achieve the "same" proportions, regardless of density), I think your proposal means the outline will be visibly thinner on higher DPI devices, no?

Thanks in advance!

[lojjic]

I think your proposal means the outline will be visibly thinner on higher DPI devices, no?

Yes, that's one downside of the screen-space approach. That may be fine for many scenarios but it would bother me as a designer to have a different look across devices.

The other big downside is that (I think) increasing the halo thickness would have performance impact due to requiring more texture reads. I need to keep researching this though.

[FunMiles]

I too need this feature. I have implemented such a feature in an Android app I developed ages ago and have used just for myself and friends. I've started re-implemented this app with React and came to find your project.
Like others, I want to thank you for this library. I am amazed by it.

On your comment on more texture reads, I do not think you should worry about that much. The additional texels you need will be the texels neighboring fragments need and thus I think they are going to be in the cache of the GPU anyways and they will cost close to nothing.

[lojjic]

Thanks for the input @FunMiles. (Colorado represent! 😄)

I think you're probably right for outlines of 1, maybe 2 fragments thick. My concern is that as the thickness increases, it's both exploding the number of texture reads (thickness of 4 = 56 reads?) as well as the likelihood those reads will be more expensive. But I'm just speculating based on reading some things, I'm no expert on GPUs by any means. I'll probably just have to try it out.

[FunMiles]

@lojjic (North of you here 😄 ) I tried to find the code for the fragment shader. I admit to have a hard time figuring things out as it seems (and I think I read it somewhere) that you modify the shader instead of the writing it entirely.
However, though I can't figure out what the full shader is like and how your really plug into it, I think I get some of the core routine...

Right now you read a scalar distance for the pixel with texture2D(uTroikaSDFTexture, vTroikaSDFTextureUV).r;. If I am understanding what you wrote before, one issue has to do with the space around the glyphs. Can you explain in more details?
Going back to your example of 'W' and ',' is the vTroikaSDFTextureUV span in the texture tightly fitting exactly around the glyph, or is there some added space? Does the fragment shader know the bounds of the UV for the current glyph? If it does, for a fragment that would fall outside, you can project the current UV to the bounds and go read the data right inside.

I think it is possible to figure out what is needed with at most 5 texel reads or with the use of derivatives (I see you do check if they are available).

[lojjic]

Going back to your example of 'W' and ',' is the vTroikaSDFTextureUV span in the texture tightly fitting exactly around the glyph, or is there some added space?

There is a small bit of extra space around each glyph's true path bounds, just enough to accommodate the outer parts of the distance field. That's 8 texels in the SDF, but since each SDF is a uniform 64x64 scaled onto a glyph quad of varying size, that visible margin varies glyph to glyph.

Does the fragment shader know the bounds of the UV for the current glyph? If it does, for a fragment that would fall outside, you can project the current UV to the bounds and go read the data right inside.

That info should be available. I'm not following you on "project the current UV to the bounds and go read the data right inside" though.

I'd be grateful if you know a way to get thicker outlines with fewer texel reads! I've been conceptualizing it as: for a fragment not inside the glyph path, do a radial search to see if any fragments within r= would fall within the glyph path. That may just be the wrong way to think about it.

[FunMiles]

One issue is what happens when glyphs are put together side-by-side. Putting that aside for now, let's take the case of a single character being drawn with a fairly thick outline t. The character is being drawn with a rectangle around it that is large enough to accommodate the outline.
The fragment shader has two main informations: vTroikaSDFTextureUV and vTroikaGlyphUV.

My understanding is that vTroikaGlyphUV has values clamped between 0 and 1 when inside the glyph. What I meant when I talked about projection was that if vTroikaGlyphUV is outsides of those bounds, you can retrieve the distance of the point "projected" onto the glyph boundary. That is, if uv = (1.05, 0.7), then the projected point is (1.0, 0.7). By using values at (1.0, 0.7+epsilon) and (1.0, 0.7-epsilon), you can retrieve a gradient to the distance and using that, compute the approximated distance at (1.05, 0.7).
When I did my own work for this, I remember reading an article of a someone doing SDF with a gradient in the texture as well. This approach avoids having to read three texels, but it makes the texture a 3 component texture and adds complexity to building the texture.

Does this answer your question?

Now when you have several characters side-by-side, then each has to be aware of the neighbors, making this a bit more complex. You need to carry the GlyphUV and TextureUV not only for the current character, but also for the neighbor. That may require to cut each character into two rectangle, so that the neighbor can be the one before for the first rectangle and the one after for the second rectangle. I am not sure how to deal with multi-line.... I did not have to deal with that case in my application. My application was a map and each label was single line.

[FunMiles]

PS: I may have misunderstood your "extra space around each glyph's true path bounds" . Are you saying that for an X, for example, it is encased into a rectangle and that there are four triangles that have no valid values?

[lojjic]

@FunMiles I think maybe I see where you're going. I'll need some time to think it through, but have other deadline work I need to focus on at the moment.

[FunMiles]

PS: A little bit of fun math note:
Just in case somebody wonders how the gradient can be obtained from two or three aligned points, one has to remember that the gradient of the distance has a fixed norm (depending on the chosen scale). And the gradient definitely points towards the outside of the glyph box. So for example if all 3 points of the example (1.0, 0.7), (1.0, 0.7+epsilon) and (1.0, 0.7-epsilon) are at the same distance, then the gradient is (scale, 0).

[stephencorwin]

Although it is definitely not performant, it is possible to use offsets. I'm using react-three-fiber, which leverages troika-text with the drei package, but figured I would share in case others are looking for a solution as a stopgap until the library supports stroke natively:

import React from 'react';
import {Text} from 'drei';

const StrokedText: React.FC<
  {
    strokeWidth?: number;
    strokeColor?: string;
    strokeResolution?: number;
    bold?: boolean;
  } & any
> = ({
  strokeWidth = 1,
  strokeColor: color = '#000000',
  strokeResolution = 100,
  position,
  bold,
  ...props
}) => {
  const font = bold ? FONTS.BOLD : undefined;
  const sharedProps = {
    ...props,
    font,
    color,
    sdfGlyphSize: 12,
    debugSDF: true,
  };

  let zOffset = 0;
  const offset = () => (zOffset += 0.001);

  return (
    <group name="Stroked Text" position={position}>
      {Array(strokeWidth)
        .fill({})
        .map((_, i) => {
          const s= i / strokeResolution;
          return (
            <React.Fragment key={i}>
              {/* <Text {...sharedProps} position={[-s, 0, offset()]} />*/}
              {/* <Text {...sharedProps} position={[s, 0, offset()]} />*/}
              <Text {...sharedProps} position={[0, -s, offset()]} />
              <Text {...sharedProps} position={[0, s, offset()]} />
              <Text {...sharedProps} position={[-s, -s, offset()]} />
              <Text {...sharedProps} position={[s, s, offset()]} />
              {/* <Text {...sharedProps} position={[-s, s, offset()]} /> */}
              {/* <Text {...sharedProps} position={[s, -s, offset()]} /> */}
            </React.Fragment>
          );
        })}
      <Text name="Text" {...props} position={[0, 0, strokeWidth ? offset() : 0]} />
    </group>
  );
};

export default StrokedText;

Then elsewhere, I can call it with <StrokedText /> instead of <Text /> normally:

<StrokedText
  color="#ffffff"
  fontSize={fontSize}
  clipRect={[-0.5, -0.15, 0.5, 0.15]}
  textAlign="center"
  position={[0, 0, 0.01]}
>
  {text}
</StrokedText>

[lojjic]

@FunMiles I've finally had some time to parse your suggestions. I think that would make sense assuming the entirety of the glyph's SDF rectangle were to contain useful (> 0.0) distance values. That's not currently the case; I think you were realizing it in your followup about "x", where the SDF falls off to zero well within the quad's bounds, so there are significant areas where there is no useful distance gradient from which to extrapolate:

Perhaps I can look into changing the SDF generator to ensure a nonzero gradient across the entirety of the quad...? Thinking out loud, that could have connotations for distance precision, and could introduce artifacts between characters within the atlas texture... 🤔

[lojjic]

Perhaps I can look into changing the SDF generator to ensure a nonzero gradient across the entirety of the quad

I've tried that, and yes it allows the distance field to be extrapolated beyond the quad bounds, but as I feared it lowers the quality of the glyph itself significantly. That's to be expected with only an 8-bit gradient; spreading it out over a larger distance results in a lower precision at each texel. :(

Perhaps I could generate a separate SDF for just the outlines -- a higher-spread but lower-precision field -- encoded into a second texture channel. It would double the texture size but shouldn't be significantly slower. 🤔

[stephencorwin]

encoded into a second texture channel.

@lojjic That sounds very reasonable and basically mimics my workaround right now. Plus, it can be opt-in, so there is no perf hit if the user doesn't ask for outlines.

[FunMiles]

@lojjic The opt-in approach suggested by @stephencorwin would make sure that nobody pays more than they're ready to pay.

Back to technical aspect. Let's take the X image in your response. Am I understanding correctly that for all the glyphs, it is 64x64 pixel? With 8 bits, if you have to encode the whole distance space, that means you have an accuracy of 2 fractional bits. I guess you are stating that this accuracy is not enough.

There's perhaps a trick to be used to gain in accuracy at very little cost. Instead of doing a grid of 64x64 of 8 bits, compress the data of 2x2 blocks into a 32-bit data. One clear feature of the signed distance between two pixels is that it is always within [-1,1] right and left, up and down and [-sqrt(2), sqrt(2)] in diagonal. So imagine you use 12 bits for the SD of the center of the 2x2 block, you would have 5 bits for each center of the pixels to encode the difference between their center and the center of the 2x2. These 5 bits represent at most sqrt(2)/2 distance. Effectively you have a 5.5 bit accuracy.
The center point 12 bits encode at least 6 bits of accuracy over a distance of maximum 64. Technically if the box had only one point at a corner the 12 bit would need to be able to encode up to sqrt(2)*64, but I don't think that's actually possible since the box is presumable reasonably centered around every glyph. In fact, the further from any glyph edge and the less information the deltas have to encode, (when you are far away from a glyph edge, the gradient field becomes almost uniform in a neighborhood) so from a information theory perspective, it is even possible to improve the encoding
to have more bits of actual information.... But I would not start with such an added optimization.

One interesting consequence of this approach is that one would not let the texturing hardware do any interpolation. But on the flip side, one would get gradients for free.

If you need help, I could implement the encoding/decoding of all this.

PS: I have a memory of seeing a discussion in one of the README.md about a more sophisticated SDF. Did I dream it? 😛 Can someone point me back to it to see if that other system could help here?

[lojjic]

@FunMiles Very clever compression idea. I'll keep that in mind if other options fail. Losing linear interpolation by the hardware is a tradeoff I'd rather not have to make. 😉

It occurred to me that my problem with not enough bits is exacerbated by using 0.5 as the "zero" distance, so only half the alpha values are available for encoding the distance outside the glyph. I could potentially shift that to use more values for outside distances and fewer for inside distances, gaining some precision on the periphery.

Re. a "more sophisticated SDF", you may mean "MSDF" where multiple color channels are used?

[FunMiles]

@FunMiles Very clever compression idea. I'll keep that in mind if other options fail. Losing linear interpolation by the hardware is a tradeoff I'd rather not have to make. 😉

I don't think you should worry about losing that interpolation. The cost is very low but the benefits of always having the gradient (and even a bit of curvature) when no in hardware support is there is of greater importance in my view. In effect, you round the sample point and get a new offset value to the rounded sample point. Computing the partial covering of the fragment for anti aliasing proceeds as it would normally do with the gradient being available.

It occurred to me that my problem with not enough bits is exacerbated by using 0.5 as the "zero" distance, so only half the alpha values are available for encoding the distance outside the glyph. I could potentially shift that to use more values for outside distances and fewer for inside distances, gaining some precision on the periphery.

That will gain you at most one bit of accuracy. Not to be sneezed at but not that significant still.

Re. a "more sophisticated SDF", you may mean "MSDF" where multiple color channels are used?
That's what I meant. I did find a GitHub project about it in C++. Did you have something to use that or did I just get confused, mashing up various things I looked up two weeks ago?

I don't think the MSDF would help, by the way.
Can I ask if you could have a small .md file explaining how you build the SDF in JavaScript? With that, I think I could create code to implement my compression idea.

[lojjic]

The SDF is built here: https://github.com/protectwise/troika/blob/master/packages/troika-three-text/src/worker/SDFGenerator.js#L134 -- not really much to explain about it, mostly mapping texels to font units and writing the measured distances into the texel values. We'd have to add some extra distance measurements in there for the centerpoint of each 2x2 block.

Trying to wrap my brain fully around this... Replicating bilinear interpolation in the GLSL looks simple/cheap enough, once you have the 4 nearest values. To get those values when they're encoded with your compression scheme, I think it will involve:

• If exactly on the center of a texel: either 2 or 3 texture samples
• If between the 4 texels of a 2x2 block: 4 texture samples
• If between two neighboring blocks: 9 texture samples either 7 or 8 texture samples
• If between four neighboring blocks: 11 texture samples 13 texture samples

Am I grokking that correctly?

[lojjic]

Oh hold up, I was still thinking of a single-channel texture. Using four rgba channels, each data block is only a single read. So at most 4 texture samples.

[FunMiles]

I had started reading SDFGenerator.js. I will look at it more.

I don't think you ever need more than reading a single texel per fragment, unless you want to specifically improve the alpha blending for corner cases where you might be at a point surrounded by glyph edges on all sides. You just need the closest center of the 2x2 block where the center of the fragment is.
However, I realize that there may be a difficulty I had not foreseen... WebGL 1.0 is very limited in what it provides that would be of use here: No unsigned integer type uint not even 32 bit integers! 🤯 and no bitwise operation... So the compression I am suggesting is a bit more tricky to write using 16 bit integers.

All these are available in WebGL 2.0 but I presume we want to target WebGL 1.0 here?

[stephencorwin]

All these are available in WebGL 2.0 but I presume we want to target WebGL 1.0 here?

I think it might be reasonable to restrict text outline to webgl 2 and just detect if the user has that browser compatibility. Although, I can understand possibly doing both a webgl 2 optimal version with a webgl 1 fallback. Imo we could start with webgl 2 and let that soak with the community before immediately trying to support both.

[lojjic]

I think I have an alternate approach to getting around the precision issue, so @FunMiles don't worry about fighting with the compression stuff for now.

[stephencorwin]

hey @lojjic, just checking in on this. Any progress or things that we can help with?

Side note: Safari is finally coming around to supporting WebGL2, so it might be possible to not support WebGL1 for this feature.

Our WebGL2 implementation is in good enough shape that we should enable it by default for broader testing.
https://trac.webkit.org/changeset/267027/webkit

[FunMiles]

I have an iPhone 6 on which there will never be WebGL 2.0 😒

Interestingly, the WebGL 1.0 specifies even worse requirements than I thought. Integers do not really exist:

4.1.3 Integers
Integers are mainly supported as a programming aid. At the hardware level, real integers would aid
efficient implementation of loops and array indices, and referencing texture units. However, there is no
requirement that integers in the language map to an integer type in hardware. It is not expected that
underlying hardware has full support for a wide range of integer operations. An OpenGL ES Shading
Language implementation may convert integers to floats to operate on them. Hence, there is no portable
wrapping behavior.

However I have not lost hope. I just have to rethink a bit...
Meanwhile, @lojjic, do you mind telling us what alternate approach you've thought of?

[lojjic]

@FunMiles Sure. I'm able to extend the distance field to the edges of the texture, while still maintaining sufficient precision for the glyph's shape, by encoding the distance values using a non-linear scale. So distances very near the glyph's path (within 1-2 texels) have many values to work with, while those further away have fewer. The shader just has to know how to convert back to the original linear distance. The glyph's proper shape looks great, and the lower precision for the extruded outlines is hardly noticeable, as they are rounded off anyway.

I've proved this out using both a two-tier linear scale, and an exponential scale. Both have pros and cons.

I'm now in the middle of implementing your (very smart) approach of using neighboring edge values to approximate the gradient outside the quad. It's making sense so far, though I'm not sure what to do about the areas outside the corners. It may become obvious once I get further into it, but if you've got an easy answer for me I'd be thankful :)

[FunMiles]

I'm now in the middle of implementing your (very smart) approach of using neighboring edge values to approximate the gradient outside the quad. It's making sense so far, though I'm not sure what to do about the areas outside the corners. It may become obvious once I get further into it, but if you've got an easy answer for me I'd be thankful :)

I am not sure what you mean by outside the corners. Do you mean the quarter planes rooted at each corner, where the closest projection is the corner itself? I think there you can use the rule on an edge on both the vertical and horizontal edge and do a weighted average based on the distance to each.

PS: The idea of reducing the accuracy far away had occurred to me earlier today after reading the WebGL integer nonsense... 😛 I still have hope that the compression technique can be implemented in WebGL 1.0 to obtain 11 bits of accuracy cheaply and a few more bits with more tests. i.e. using an rgba, the a could hold 8 bits of the average then for each of rgb, they would be signed and the sign would represent 1 bit each of accuracy for the average, then finally the remainder, which would be in the range 0-127 would have the value 64 subtracted to represent 7 bits for 3 of the 4 values needed. The actual value would this be center (ranging from 0 to 2^11-1) + dist_i. The 4th value would be recovered by doing minus their sum, since the sum must be the average. One would only get 11 bits, but that's probably sufficient. To get more bits would require to test whether the value or r, g, and b are smaller than -64, positive or negative, larger than 64. That would essentially give 14 bits for the center and only 6 bits for the differences. Probably a good compromise? I would need to test that the very lax guarantees that WebGL 1.0 puts on accuracy would not interfere with this thinking....

[lojjic]

Yeah that's what I meant, the areas where both U and V are outside 0-1. Thanks, I'll give that a shot.

[FunMiles]

@lojjic As an aside question, in a previous post, you seemed concerned about having two texture values per SDF texel because of memory. Though I myself prefer to reduce memory, 256 glyphs on a 64x64 grid are only consuming 1/4MB of texture memory. I know some languages may require many more glyphs, but still, the BBC says that to read the newspaper, only 2000/3000 characters are needed. So 4000 characters would make for 4MB with one byte per texel SDF. Worth worrying about?

[lojjic]

@FunMiles Fair point, and I wasn't actually very concerned about that tbh.

[stephencorwin]

Some outstanding work left to do, but b19cd3a fixes this issue. For those using react-three-fiber, this has been released in drei v2.2.0
pmndrs/drei#156

[FunMiles]

I know this is closed but I want to say thanks for the work. I got it to work in my app. Took a bit a figuring out that the properties would not accept just a number for the outline size. However the result is just what I needed.

[lojjic]

@FunMiles Looks nice! You should be able to use a numeric value for outlineWidth, so if that's not working for you for some reason let me know.

And thank you for your input along the way. I still wasn't able to get a smooth extrapolation of distance outside the quad bounds using the sort of technique you mentioned, so a thick outline currently has lumpy bits at the corners in particular, but it's good enough for most cases (up to ~10% the font size). I'm definitely open to trying to refine that still.

[lojjic]

@amcdnl Would you mind moving your support question to a separate discussion, so we're not spamming all the contributors of this original issue? And if you can include a working codesandbox or similar showing your issue that would be helpful in diagnosing. Thanks.

[amcdnl]

@lojjic - Apologies. Made it here: #213