KHR_texture_video

[dwrodger]

This is a proposal for an extension to standardize the way that video textures can be assigned in gltf. Currently video textures are supported by most standard 3D players on the web, but the configuration of the texture requires custom javascript unique to each player. This extension intends to standardize that.

[aaronfranke]

I am interested in video textures, but the spec is very ambiguous and has errors.

[aaronfranke]

The file should be named README.md

[dwrodger]

Thanks for catching this. I have fixed it in the latest commit.

[aaronfranke]

points to the index of the images node which in turn points to the video.

How does images in turn point to a video? In your example, the entirety of the "images" only contains one object with a reference to a PNG file:

"images": [
    {
        "uri": "image.png"
    }
]

I think what you meant is that "source": 0, inside of "KHR_texture_video": { points to /extensions/KHR_texture_video/videos/0, but this is not at all clear from the document.

[dwrodger]

Thanks for catching this. I have updated this section to be more clear and precise in the latest commit. Please let me know if it is clear and unambiguous now.

[aaronfranke]

This is not valid JSON, the [] symbols specify an array but you are defining a property like a sub-JSON.

[dwrodger]

Thanks for catching this. I believe it is resolved in the latest commit. Please let me know if you see any further problems with this.

[aaronfranke]

Where is KHR_interactivity defined? It's not a part of this repository.

For interactivity, note that OMI's physics extensions already define an interactible trigger. You can specify a trigger shape that a user can point at in the world and interact with. What actually gets triggered is defined by other extensions, so this video player extension could define that a given OMI trigger plays the video. I would highly recommend that we move forward with OMI physics so that we can use these extensions together (we could rename to have the KHR_ prefix if Khronos is in favor).

[dwrodger]

KHR_interactivity is an ongoing work by the 3d format's group's DTSG on behaviors and interactivity. It is still an early work in progress, but a partial draft should be available on a branch soon.

[aaronfranke]

This is still not valid JSON.

[dwrodger]

Indeed. Thanks, I believe it's fixed in the latest commit.

[ibouazizi]

There is an MPEG extension MPEG_texture_video, which serves the same purpose but defines the details of how the video format looks like and how it is fed dynamically. Please consider using that extension.

[dwrodger]

Regarding the vendor extensions MPEG_texture_video and related extensions, it looks like there were two parallel efforts on video textures around the same time. I could see a potential consolidation where we make one extension that defines how video textures are referenced and time-controlled (e.g. linkage from texture and autoplay / connection to KHR_animation_pointer) and potentially another set of extensions that add to that support for specific formats and codecs and provide the rest of the details that this extension provides.

I would like to make sure that the video texture control is designed to work well with KHR_animation_pointer (and the upcoming interactivity work that ties heavily with that). If some example jsons were added to the MPEG_texture_video set of specs it might make it easier to understand whether it can already work well with KHR_animation_pointer or, if not, how it could be adapted to do so.

[gurdep]

@dwrodger the following is an example of MPEG_texture_video which might help for better understanding. The format description, codecs, mime-type, tracks, and other information can be exposed via MPEG_media extension as shown in the example.

[aaronfranke]

@gurdep Those JSON files you linked are just small sections, do you have any links to complete example glTF files?

[gurdep]

@gurdep Those JSON files you linked are just small sections, do you have any links to complete example glTF files?

You may ask the MPEG group for complete glTF files at their official github repository.

[kzhsw]

Since many video container format, like webm in given example, could also include audio tracks, should the behavior of audio tracks in video files be standardized here?

Behaviors might include some of these:

• default audio track
• muted or not
• volume
• should video autoplay be blocked by browser for having audios, or should it fallback to muted autoplay