Reintroduce MediaMetadata artwork (a URL)

[foolip]

This spec text is mostly written by @richtr, reverted back from
commit 551bf65.

Fixes #58

[doomdavve]

I suppose this prerequisite should be part of #100 eventually?

[foolip]

Possibly. In an implementation one probably wouldn't have a virtual "do you have some place to display artwork", but perhaps one could say that Media Session has an async algorithm that the platform can invoke to get the artwork. So then maybe the actual invocation of the algorithm would be in the "Virtual platform" section.

[mounirlamouri]

Should we use a format similar to the Manifest icons format instead of going down the path of one image fits all?

[foolip]

Is that http://w3c.github.io/manifest/ and is it ready to integrate into Notifications and Media Session? Would it be something like new MediaMetadata({artwork: [{ src: 'artwork.jpg' }, { src: 'artwork@2x.jpg', density: 2 }] })?

[annevk]

There's a discussion about it in whatwg/notifications#28. I don't see why you wouldn't have a simple artwork option first though. In the long tail of things folks are not going to have the resources to provide all those different images.

[mounirlamouri]

@foolip that's correct. Though, I would say that sizes is more important than density. We just killed density from the spec because it was misunderstood and vastly misused.

Something like:

new MediaMetadata({artwork: [{ src: 'artwork_ld.jgp', sizes: '128x128', type: 'image/jpg'},
                             { src: 'artwork_hd.jpg', sizes: '256x256', type: 'image/jpg'}]);

@annevk I think it is important to offer multiple resolutions. I'm not sure if it is a great idea to make the wrong thing the easiest to do. It sounds like the incentives would be backward. I will not die on that hill but I would prefer for us to spec the full and correct solution first and maybe offer a shorthand if it becomes a major request. Note that no one ever asked for an non-array based icon field in the Manifest.

[annevk]

@mounirlamouri incentives? Why is it wrong to only provide a single image?

[mounirlamouri]

When there is a single image, developers will be incentivised to use a large one, however it, comes with the draw backs of being a larger bandwidth usage. In addition, in term of quality, when scaled down to the required size, it could lose some precision.

If the developer is conscious of bandwidth consumption, they might try to use a smaller image but in that case the quality problems become even worse for devices that require something bigger.

It gives no right solution to the developer apart from doing some device sniffing to check the pixel density of the device and set an icon that would be best. Though, that requires to know the size of the icon the UA will end up using. Also, in some cases, the icon will be used in remote devices. For example, Chrome will be using it on Android Wear devices for which the ideal size might not be the same as your mobile.

To give some numbers, assuming Android uses a 128dp icon, on various devices, the icon in px that would be required would be 128px, 256px, 320px and 448px. That would only be with the common pixel densities :) It's a large difference between the smallest possible icon and the largest.

[foolip]

Here's a situation where a manifest makes good sense: High-resolution artwork is available, but the in-page artwork is a small thumbnail. A Nexus 6P phone could make use of 2560×1440 artwork for the lock screen background, while an iPhone 4s could use at most a 960×960 (ignoring padding) for use on its lock screen UI. The file size difference would be around 4 times.

However, here's a situation I'm not sure how it works with a manifest. If the in-page artwork is already the biggest available artwork, but the manifest also provides scaled-down versions that are a closer match for the thumbnails in notifications, is there any way to express that it's better to just scale down the largest resource already fetched?

[mounirlamouri]

If I understand correctly, your example is the following:

• website is using hd_image.png in in-page content;
• website specifies hd_image.png and sd_image.png to the Media Session metadata.

In this case, I don't think it is obvious that we want to use hd_image.png even if it is already downloaded. If the bandwidth is critical, the UA should obviously try to use the high quality image but using the higher quality image can reduce the quality if scaled down improperly. For example, I wouldn't be surprised that some wearable devices will favour speed over quality when scaling up or down images to reduce CPU usage.

However, if the website thinks saving bandwidth is important, they can simply not pass sd_image.png to the Media Session metadata, right? I don't think we need something for the website to be able to specify something -- though, I'm assuming they have control over their cache using things like Cache API.

[xxyzzzq]

I think this pull request can be closed now. MediaMetadata artwork is already in the spec.