Support different shortcut icon requirements/guidelines on different platforms

[mgiuca]

Spun off from #768.

@christianliebel :

The platform conventions for shortcut icons seem to differ quite a bit. For instance, iOS shortcuts use icon masks; Windows jump list entries have colored icons, etc.

Is there a plan on how to deal with this? Should a user agent on IOS create an icon mask from a given image? And/or should ImageResource’s platform member be evaluated for that scenario?

I assume this question will arise very quickly, so it would be great if the spec could be a bit clearer on this.

Me:

Yeah, since shortcut uses ImageResource, maskable should cover any platform-specific needs with regards to conforming all icons to a specific shape. We don't currently have a solution for platforms that want icons to be all monochrome. It's possible there's a more pressing need for that in relation to shortcuts than generic app icons (since some platforms may have monochrome shortcut icons).

If we want to go down that path, I would propose a new purpose: "monochrome" for ImageResource (that would be separate to this PR). I want to avoid any ability for developers to supply an "iOS icon" and a "Windows icon", etc, because then we will break away from the Web platform's platform neutrality and start seeing apps with poor or no icons on other platforms. (Devs can already technically do this by user-agent-detecting when they serve their manifest, but I want to avoid giving them an explicit feature to do this.)

I suggested that we start with an analysis of what shortcut icon guidelines there are on each major platform so we can see whether "monochrome" is necessary and sufficient (which should be done here perhaps).

@beverloo :

Would the badge purpose be sufficient? It specifically leaves such flexibility as we have the same constraints in other UI surfaces:

A user agent can present this icon where space constraints and/or color requirements differ from those of the application icon.

Let's continue the discussion here.

[mgiuca]

Would the badge purpose be sufficient? It specifically leaves such flexibility as we have the same constraints in other UI surfaces:

A user agent can present this icon where space constraints and/or color requirements differ from those of the application icon.

It looks like purpose: badge was added in #480. I'm not sure if any user agent is actually using purpose: badge, but it doesn't seem very useful to me in its currently specified state. The text of the spec is meaningless: what does "space constraints and/or color requirements differ" really mean? Without further guidance from us, a user agent could not make any assumptions about what it means to developers.

As an example, let's say there's a hypothetical platform where shortcut icons are supposed to be monochrome. Let's say that a user agent on that platform interprets "space constraints and/or color requirements differ" as "the color requirements are monochrome", so decides to prioritize purpose: badge icons for shortcuts on this platform, intending to receive monochrome images from the site. Now two scenarios can emerge:

1. Sites interpret "space constraints and/or color requirements differ" in another way, perhaps they use purpose: badge for much simpler icon designs, but still full-color. Thus, our hypothetical user agent isn't getting monochrome images from these sites.
2. The hypothetical user agent becomes so popular that sites realise if you put a monochrome icon in purpose: badge, it looks perfect on this platform. Any non-monochrome icon in purpose: badge looks very bad, so everybody stops doing it. Thus a de facto standard is created, governed by a single user agent's interpretation of vague spec language.

Either way, there is a mismatch between the spec text and the user agent interpretation. If we're going to define alternative purposes for icons, I think we should define very precisely the graphical expectations of that purpose, establishing a contract between sites and user agents, so that sites know how user agents may interpret the image data. purpose: maskable is a good example of this done right, very clearly establishing spatial rules about which pixels may be cropped by the user agent. I think we could define a purpose: monochrome which says that either the image must be monochrome, or the user agent is allowed to use the brightness of each pixel, ignoring the chromaticity of the pixels. The current purpose: badge I would like to remove if possible.

We might also need a purpose: monochrome_maskable (a combination thereof), or a way to define multiple purposes on an icon.

Also, @christianliebel :

And/or should ImageResource’s platform member be evaluated for that scenario?

Whoa, I didn't actually know this existed, contrary to what I was saying above ("I want to avoid any ability for developers to supply an "iOS icon" and a "Windows icon", etc"). Ideally this usage would be discouraged, for the reason I gave above.

[beverloo]

The idea behind the badge purpose is that it's an icon that can be used in space constraint locations like the address bar and device status bar, where the exact appearance (display size, overlay colour, maybe on top of a circular surface) may not always be well defined, or known at all. In other words, a clear, generic icon that is mendable in regards to colour. I agree that the prose can use refinement.

While I appreciate the goal of being able to precise how everything will be used, because some of these graphics will be handed off to the underlying operating system that isn't always an option across OS versions and flavours (e.g. Android OEMs).

The constraints for jump lists sound very similar to me. Icons might be displayed in greyscale, coloured according to a theme color, or in full colour, but the exact appearance might not always be known. As a concrete example, on Android users are free to install a launcher of their choice that may decide to do something different. Browsers cannot hardcode the configuration of each possible launcher.

[mgiuca]

The idea behind the badge purpose is that it's an icon that can be used in space constraint locations like the address bar and device status bar, where the exact appearance (display size, overlay colour, maybe on top of a circular surface) may not always be well defined, or known at all. In other words, a clear, generic icon that is mendable in regards to colour. I agree that the prose can use refinement.

OK, that description does match what I had in mind for this. I think we could use purpose: badge.

But there needs to be some guidelines as to how it will be interpreted by the user agent. Otherwise you don't know what you're designing against. Ideally, the designer doesn't need to take any platform-specific design criteria into account (if they do, then we will find sites that are biased towards one particular platform to the detriment of others).

For example, you say "maybe on top of a circular surface"; well, we have a precise definition of "maybe on top of a circular surface" in Icon masks and safe zone. So if "badge" referred to that, designers would have a clear idea of which parts of the icon may be cropped out.

Similarly, for colour: it seems that some "badge" contexts (e.g., Android app icon shortcuts) are expected, but not required, to have all non-transparent pixels as the same colour, matching the app's theme colour, and this colour should contrast well with light grey. Other contexts, such as the Android system bar, have the icon be all white on a transparent background. If you look at other platforms, there will be different guidelines. Maybe some platform will say the colour must contrast with black; then the foreground colour you choose will be quite different.

Now, we don't want to ask web developers to develop against the Android guidelines, or the Windows guidelines, etc. We want to clearly specify our own very specific guidelines, then have user agents map that onto the individual platform. For example, we say "it must be greyscale with no transparency, where the background is black and the foreground is white, and also the maskable safe zone applies". Then on Android, to map it to the status bar, we convert luma into alpha, setting all non-transparent pixels to white. To map it to a shortcut icon, we do the same but set all non-transparent pixels to the app's theme colour, possibly darkening so it contrasts well with light grey. On another platform, we can map it onto a different colour scheme to fit the platform's guidelines and contrast requirements. This way, nobody is thinking about Android or Windows, they're just targeting our platform.

The worst case scenario I can imagine is we just say "use purpose: badge" to developers, and then user agents just take the purpose: badge icon and put it in the shortcut menu without modification. Then developers targeting Android realise that they need to conform to the Android shortcut icon guidelines in order to look good, and now those icons can't be used properly on other platforms.

While I appreciate the goal of being able to precise how everything will be used, because some of these graphics will be handed off to the underlying operating system that isn't always an option across OS versions and flavours (e.g. Android OEMs). ... As a concrete example, on Android users are free to install a launcher of their choice that may decide to do something different. Browsers cannot hardcode the configuration of each possible launcher.

I disagree. While the browser doesn't know exactly what launcher the user has installed, or other OEM-specific details, the browser does know exactly as much as any other native app providing icons (which is to say: a lot more than what the web developer knows, since they don't even know what OS they're on). The web developer can't follow the OS icon guidelines. The browser can manipulate a generic "web icon guideline" icon into an image that follows the OS icon guidelines (e.g., the Android icon guidelines). Thus, our web apps will be presenting the same quality of icons as any native app. From there, the individual OEM changes to the OS need to uphold their end of the OS-specific icon contracts (e.g., if an Android OEM places icons that follow the Android icon guidelines onto a black background, then our browser-generated icons and all other native app icons will look bad).

Again, purpose: maskable is a good example of this in action. Our Android implementation of Maskable doesn't take the icon from the manifest and cut it into a circle, or a squircle, or a teardrop, etc. The browser doesn't know what the OEM or user-specific icon mask shape is. Instead, we take the icon that matches the Manifest standard, and add padding so that it matches the Android icon sizing requirements, and then we give it to the OS, and from there, some unknown masking happens to the icon. We don't care what that masking is, because we did our job, which is to produce something conforming to the Android OS guidelines.

[beverloo]

I'm all for being specific, but it should be clear that we're going to run into situations where (a) the appropriate (modifyable) icon hasn't been provided by the developer, or (b) the user's device expects something different from what the browser has selected on the developer's behalf, unbeknown to the browser.

As Christian wrote, iOS wants maskable icons. Windows wants coloured icons. On Android, for most default launchers, it depends on what the icon describes: photos and logos should be full colour, whereas icons should be masked in the app's theme colour. Considering further variables such as colour scheme (e.g. dark mode), differences between OS versions, and so on, the number of permutations of icons we'd ask developers to provide is unreasonable.

Having a purpose that comes down to "do whatever needs doing" can be the starting point for a simple implementation. Being able to specify multiple icons with more specific purposes is a good second step. Perhaps more controversial, but I also think we should consider enabling developers to provide specific resources if they feel the specialization is justified.

[mgiuca]

(a) the appropriate (modifyable) icon hasn't been provided by the developer

That's OK, in that case we do whatever needs to be done as a fallback. If the developer doesn't supply a purpose: badge icon, there's not much we can do. But if the developer does supply purpose: badge, we need to make sure the expectations of the developer and user agent are aligned.

(b) the user's device expects something different from what the browser has selected on the developer's behalf, unbeknown to the browser.

This sounds like the same argument about the browser not knowing about OEM-specific requirements on the device. Is that what you mean? As I said above, if that's a problem, it's a problem for ALL native apps. Can you give a concrete example of an Android launcher that has a different expectation for icons than the Android icon guidelines? And if so, can you explain what other native apps are expected to do about it? (i.e. how is my native app supposed to know to supply a special icon for this launcher?)

On Android, for most default launchers, it depends on what the icon describes: photos and logos should be full colour, whereas icons should be masked in the app's theme colour.

Let's say we specified purpose: badge the way I outlined above.

If you have a simple "action" icon, you might supply a purpose: any, a purpose: maskable and a purpose: badge version of it. The user agent would be free to pick the most specific, e.g., on Android we choose the monochrome purpose: badge icon and convert it to use the app's theme colour. Other platforms might simply take the purpose: maskable version which can be in full colour.

If you have a photo as a shortcut icon (e.g., a contact picture), you would NOT supply a purpose: badge icon, perhaps only supplying a purpose: any and a purpose: maskable. On Android, because purpose: badge is not available, we'd choose the purpose: maskable icon which would show it in full colour.

Thus, the web app isn't aware of platform-specific requirements. It just supplies various permutations (at this time, up to 3) of the icon, and the user agent selects the most appropriate.

Considering further variables such as colour scheme (e.g. dark mode), differences between OS versions, and so on, the number of permutations of icons we'd ask developers to provide is unreasonable.

A purpose: maskable icon doesn't need to vary based on colour scheme, it's just a full colour logo. A purpose: badge icon (per my proposed definition) is monochrome and can be adapted by the user agent to suit the colour scheme. With my scheme, you don't need to supply many permutations (at most 3). With your scheme (the current status quo), it is up to the web site to supply icons specific to each platform, which means you end up needing to do user agent detection if you want to handle all of those different cases.

Having a purpose that comes down to "do whatever needs doing" can be the starting point for a simple implementation.

"do whatever needs doing" doesn't mean anything unless the source image parameters are well defined. Otherwise the user agent doesn't know what needs doing.

[mgiuca]

See #808 for a great analysis of how these should look on different platforms, thanks @NotWoods .