[Impeller] Support rendering compressed texture formats / add capability to detect support.

[jonahwilliams]

I think that we could fairly easily add support for compressed image formats to Impeller, at least on the Vulkan and Metal backends.

Vulkan: https://docs.vulkan.org/spec/latest/appendices/compressedtex.html
iOS: https://developer.apple.com/documentation/metal/mtlpixelformat/mtlpixelformatastc_6x6_hdr?language=objc

I'm not sure what the actual level of device support is on Vulkan and would need to do some research. For iOS, there is a handy feature set table here: https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf . We could look at the intersection of minimum supported formats as a starting point.

These formats specify how pixel data is compressed, but don't include header information on the format/size like commonly used network image formats like PNG/JPEG. This metadata would need to be provided by the application. I'm not sure if there are other standards here like ASTC in PNG or if the expectation is that the application level logic will handle transport compression/bundling.

For example, a low-level dart:ui API for decompressing.

import 'dart:ui' as uil
[
void main() {
  Uint8List data = getBytes(...);
  if (!ui.Codec.checkFormatSupport(PixelFormatASTC_4x4_sRGB)) { //Probably a different name
    return;
  }
  var ui.image = await ui.decodeCompressedTexture(data, width: 100, height: 100, format: VK_FORMAT_BC1_RGB_UNORM_BLOCK);
 

Then some higher level Image provider API:

Image.network(
  ...
  compressedFormat: CompressedFormat(PixelFormatASTC_4x4_sRGB, width: 100, height: 100)
 )

For testing/evaluating, there is an open source ASTC compressor developed by ARM available at https://github.com/ARM-software/astc-encoder.

Some caveats:

• We can't support these in the Skia backend as the formats are unsupported as far as I know, unless we wrote separate import code and treated them as external textures (which I'd rather not do).
• We currently resize images using Skia's software codecs which don't support these formats, so resizing would fail until we switch to GPU based resizing.

[chinmaygarde]

Exposing support for compressed texture formats as described is necessary, but I don't believe is sufficient in making a usable system. Especially given the myriad of formats that GPUs can natively support and the offline tooling necessary.

I am in favor of one of the supercompressed formats in a standardized container like ktx2. In Dart code, the user can specify a file in a KTX2 container and we can do some minor conversion to a known supported format at runtime.

[chinmaygarde]

For reference, this is the guidance on Android. Also, Unity also does the transcoding like I described in an earlier comment. I am not sure if they use the open standard though. But from the perspective of an easy to use system, some sort of transcoding would be necessary.

[chinmaygarde]

More guidance on how other engines reason about a usable way to work with compressed textures. This also uses KTX2 for the container.

[jonahwilliams]

We don't need to build an end to end usable system though, that can be done as a package/community project. The only thing the community can't do is add engine support