gltf-transform optimize --texture-compress webp yields Invalid JPG, marker table corrupted

[jo-chemla]

Hi there,
First-of-all, thanks a lot for this great CLI utility! Not sure if I should post it as an issue (potential bug?) or discussion.
I've got a 3d-scan stored as a 500k triangles glb + external 1x16k jpeg texture (~90MB of input data) that I'd like to compress (geometry compression, texture compression and eventually geometry simplification).

Since the texture file is a real 16384x16384 16k image, I just downscaled it to 16383x16383 to avoid this issue with webp images - see below note ref for other formats size limits if useful.

I've tried multiple combinations of gtlf-transform optimize --compress xyz --texture-compress abc vs a mix of gtlf-transform webp && gtlf-transform draco and think I've encountered a small bug I cannot explain. I'd like the texture to be stored either as webp or ktx2 rather than jpg (which has a way larger gpu size).

• When using gtlf-transform optimize input.glb output_optimized_auto.glb --texture-size 16384, this yields a correct glb I can inspect, but this one has a jpeg texture (probably because the texture-compress arg is set to default auto)
• When using gtlf-transform optimize input.glb output_optimized_draco_webp.glb --compress draco --texture-compress webp, then the gltf-transform inspect command results in the below error message: error: Invalid JPG, marker table corrupted. This glb is hte most efficient in terms of size (4MB) can be imported to blender 4.0 with the glTF 2.0 importer plugin, but this is a strange behavior, seem like a reference is still thinking the texture is jpeg although it has been converted to webp - see below.

OVERVIEW
 ────────────────────────────────────────────
┌────────────────────┬────────────────────────────┐
│ key                │ value                      │
├────────────────────┼────────────────────────────┤
│ version            │ 2.0                        │
├────────────────────┼────────────────────────────┤
│ generator          │ glTF-Transform v3.10.0     │
├────────────────────┼────────────────────────────┤
│ extensionsUsed     │ KHR_draco_mesh_compression │
├────────────────────┼────────────────────────────┤
│ extensionsRequired │ KHR_draco_mesh_compression │
└────────────────────┴────────────────────────────┘

error: Invalid JPG, marker table corrupted

• gltf-transform draco input.glb temp_draco.glb && gltf-transform webp temp_draco.glb output_classic_draco_webp.glb errors out, and results in the following webpsave: unable to encode. Using a lower --texture-size 12000 do avoids this error, but not with --texture-size above ~14000

warn: Decoded KHR_draco_mesh_compression. Further compression will be lossy.

error: webpsave: unable to encode
webpsave: mux error

• Using etc1s or uastc instead of webp texture compression seem to yield correct glb with texture upon inspection but larger size. Using gtlf-transform optimize input.glb output_optimized_draco_ktx2.glb --compress draco --texture-compress ktx2 takes longer (probably because of ktx2 compression) which is okay, but the resulting mesh geometry data seem to have a higher size on disk - looks like simplify outputs the same number of vertices on blender imports, not sure what takes more space as I cannot inspect the other output_optimized_draco_webp.glb

gltf-transform optimize input_500k_1x16k.glb optimized_500k_1x16k_optimized_draco_ktx2_.glb --compress draco --texture-compress ktx2
✔ dedup                2ms
✔ instance             1ms
✔ palette              1ms
✔ flatten              1ms
✔ join                 1ms
✔ weld                 1,597ms
✔ simplify             2,444ms
✔ resample             0ms
✔ prune                1,215ms
✔ sparse               105ms
warn: toktx: No textures were found, or none were selected for compression.
✔ uastc                111ms
warn: toktx: Texture dimensions 16383x16383 are NPOT, and may fail in older APIs (including WebGL 1.0) on certain devices.
warn: toktx: Resizing to nearest power of two, 16384x16384px. Texture dimensions greater than 4096px may not render on some mobile devices. Resize to a lower resolution before compressing, if needed.
✔ etc1s                332,033ms
✔ draco                1ms

If I understand things correctly, the best way to do this compression would probably be to use the optimize in out --compress draco --texture-compress webp/ktx2 CLI utility, and have it output a glb which would not have that error on inspection Invalid JPG, marker table corrupted. Or find a way around the webpsave: unable to encode error on non-optimize workflow.
Thanks a lot, any pointer would be appreciated.

Regarding max images resolutions, see here for other file formats size limits

• webp cannot be used to store 16384 images WebP: 16383 x 16383 (14-bit unsigned)
• JPEG/TIFF/GIF: 65535 x 65535 (16-bit unsigned)
• PNG: 2147483647 x 2147483647 (32-bit signed)

[donmccurdy]

Hi @jo-chemla!

On the choice of KTX2 or WebP, note that WebP will have the same GPU size as JPEG or PNG. WebP will generally be smaller on disk, but no different on the GPU. See Choosing texture formats... for more on that.

I'm not sure what to make of the Invalid JPG, marker table corrupted error — I haven't seen that before, but it sounds like an invalid glTF file is being generated. Would it be possible to share the original file, before the optimization? I probably cannot investigate that error without a way to reproduce it. It might be relevant to know which version of glTF Transform (gltf-transform --version) you're using, as well.

It is possible that --texture-compress ktx2 may increase file size on disk, this depends heavily on the image. Size on the GPU will always decrease, compared to traditional image formats. If you'd like to try KTX2 compression in isolation, without the rest of the "optimize" processing, the following commands (run them both) should have a similar effect:

gltf-transform uastc input.glb output1.glb \
    --slots "{normalTexture,occlusionTexture,metallicRoughnessTexture}" \
    --level 4 --rdo 4 --zstd 18
gltf-transform etc1s output1.glb output2.glb --quality 255

Small adjustments to the flags above would be needed if you're using the v4 alpha release.

[jo-chemla]

Hi @donmccurdy, thanks for getting back!
Thanks for the KTX2 vs WebP clarification regarding gpu-size. We will then probably base our decision for webp vs ktx2 on storage and bandwidth vs gpu decoding total timings/duration (something equivalent to time to first 3d-load between streaming from the server to the client on a given bandwidth, sending to gpu and decompressing via r3f useLoaders and first render) - mostly because the hardware these models will be rendered on will be pretty controlled (both gpu/vram, ram and cpu).

I did try the uastc and etc1s operations in isolation, but did not chain them the way you describe - will try that to see what ktx2 compression alone does with regards to filesize.

I will share the original file via DM in case you want to investigate - not public as it's a pretty sensitive project - and I'm using v3.10.0. Thanks a lot for your help!
Best,

[donmccurdy]

I've merged #1306 and published an updated version to v4.0.0-alpha.10, fixing the Invalid JPG, marker table corrupted error. With that update, using --texture-compress webp should work correctly, up to the size limits of the format and the Sharp encoder (https://sharp.pixelplumbing.com/). Example:

gltf-transform optimize input.glb output.glb --texture-compress webp --texture-size 8192 --compress draco --verbose

It's a lower resolution texture of course, but does reduce total size from 94 MB to 13.5 MB.

I should note that .glb files written by glTF Transform will embed any associated textures, whereas your source .glb file had a reference to an external texture, so that difference would need to be accounted for in any size comparisons. To keep textures external with glTF Transform, just write to a .gltf file instead.

[jo-chemla]

Thanks a lot for that fix!
Just tested with v4.0.0-alpha.10, the resulting glb (optimize:draco+webp) indeed can be inspected correctly by gltf-transform.

The only weird thing is that indeed the texture resolution cannot take the maximum resolution a webp file can handle, which should be 16383x16383. I can optimize with --texture-size 12000 but not with --texture-size 14000 (or 16383), which yields to the error previously seen: error: webpsave: unable to encode webpsave: mux error. Do you think this is a bug from sharp/libvips itself - and if so, should I try to open an issue on the underlying repo to see if this is indeed a bug? I've seen this issue where a cwebp command would fail as well with a resulting error Error code: 6 (PARTITION0_OVERFLOW: Partition #0 is too big to fit 512k., so I might be out of luck.

By the way, when comparing the inspected "optimize:draco+webp" vs "optimize:draco+ktx2", I can see that indeed the geometry size is the same. But funny thing, the gltf-transform optimize in.glb out.glb --texture-size 12000 --texture-compress ktx2 --compress draco yields a file with the original resolution for the texture size.

Again, thanks a lot for your help.

[donmccurdy]

Currently the --texture-size flag affects all formats except KTX2. That needs to be fixed (#1307), but for the moment you'll want to use the standalone KTX2 compression steps (before or after optimize) if you also want to resize the texture. Resizing must happen before KTX2 compression.

The errors from webpsave are coming from Sharp or one of its dependencies. I couldn't say whether that's a bug or not, but it may be worth trying to do standalone webp encoding with sharp and this texture, if the error occurs there, I would file a bug or question on their repository.

[jo-chemla]

Thanks again for all this, perfectly fine for optimize:ktx2 not handling the --texture-size argument atm - will prepend a resize if that's the route we chose. I'll try out a standard libvips cli command to convert that texture to webp, and open a dedicated issue on libvips - or comment that closed issue in case the same problem occurs.

[jo-chemla]

Note: I did retry to optimize a glb with an exactly 16k texture (16384²) with --texture-compress avif and textureCompress did fail with error Input image exceeds pixel limit.

The AVIF format standard do not seem to specify a max resolution, and using vips (which is what's under the hood in sharp), I can convert an 16384² input image from jpg to avif:

vips-dev-8.15\bin\vips.exe copy input_16384_u0_v0_diffuse.jpeg out.avif

Edit; This error comes directly from sharp itself and its defaults limitInputPixels(0x3FFF * 0x3FFF), not gltf-transform for the sharp encoder. Just saw this comment, would it be possible to expose sharp encoder options.limitInputPixels as a optimize CLI flag?