[USD] Support load .usd/.usda/.usdc/.usdz models via "tinyusdz" project (prototyping branch)

[tellypresence]

Evaluate tinyusdz rendermesh-refactor branch for feasibility

Use PR #5551 for testing

rendermesh-refactor branch is under active development, several features are working reasonably well at least on some test models

• vertices
• faces
• normals
• UVs
• material colors
• material textures
• embedded textures

Partially working

• blend shapes (morph meshes) detection: able to extract blendshapes in some rudimentary models

What's not working yet

• rig/bones support
• skeletal/skinned animations
• blend shape animations
• node-only animations (for testing will need USD format model similar to node-only MotionCaptureROM.ase)

Other problems

• "Composition" (layered/hierarchical models) not supported yet (this means many USD models encountered "in the wild" will fail to load)

P.S. During the early stages of this effort, I raised and commented on several issues on the tinyusdz project e.g.

• 128
• 130
• 133

Unfortunately, while I had meant to be constructive and helpful, I appear to have become something of a nuisance, resulting in my being banned from raising or commenting on issues on the tinyusdz project. My own fault, of course, but as it's not an ideal situation for someone trying to adapt the project in this manner I will park this endeavor for the time being. Latest working changes have been pushed up (I had some code which attempted to parse animations and bones but was unable to verify basic operational correctness, so have not included that to avoid clutter and confusion)

[tellypresence]

Blendshapes/morph meshes

Current blendshape status: in some rudimentary models, blendshapes are detected and appear to be valid. In other models expected to contain blendshapes, no blendshapes are detected during model load.

Also struggling to extract animations for blendshapes in a sensible way.

Models with valid blendshapes

Model	Screenshot(s)	Notes
blendshape.usdc		No animations have been extracted, demo screenshot simply morphs between available blendshapes over a few seconds

Models which would be expected to have valid blendshapes where no valid blendshapes detected

Model	Screenshot(s)	Notes
animatedMorphCube.usda		Has 2 blendshapes, but both have zero point offsets i.e. blendshapes are identical to the main mesh so no visible morphing
MorphStressTest.glb.blender.usdz		Model loads OK but while morph meshes are expected to be present, none detected
Morph_Stress_Test.usdz		Model loads OK but while morph meshes are expected to be present, none detected

Blendshape models which fail to load

Model	Screenshot(s)	Notes
morphedCube.usda		Validation failed: a node of the scene-graph is nullptr

[tellypresence]

Bones/skeletons/skinning

I'm struggling to understand how to convert the data from tinyusdz render scene into skeletons. It appears that vertex weights are a per-mesh, rather than per-bone, attribute, but my understanding is that each bone should provide its own set of vertex weights. Anyway I can't seem to discover where those values would be provided.

[tellypresence]

Models which fail to load

Disney

None of the sample models work

Intel

None of the sample models work

Pixar

None of the sample models work

• kitchen set
• city set
• usdskel examples

usd-wg

With the exception of Vehicles, none of the full assets work

[tellypresence]

Models which used to load but now fail in recent commits

Apple

Model	Notes
toy_drummer_idle.usdz	This model used to work with tinyusdz

Sketchfab

Model	Screenshot(s)	Notes
Medical Assistance Bot		This model used to work but in more recent commits now fails to load due to internal tinyusdz failure to convert to "render-scene"

[tellypresence]

(reserved)

[tellypresence]

Models which load and render with problems

Apple

Model	Screenshot	Notes
fender_stratocaster.usdz		UVs incorrect (may be a problem in the model itself)

[tellypresence]

Models which load and render well

Apple

13 of the 15 demo models load and render OK

Model	Screenshot	Notes
toy_biplane_idle.usdz		(Only started working in recent commit)
robot_walk_idle.usdz
toy_car.usdz
flower_tulip.usdz
gramophone.usdz
sneaker_airforce.usdz
sneaker_pegasustrail.usdz
tv_retro.usdz
chair_swan.usdz
wateringcan.usdz
teapot.usdz
pancakes.usdz
cup_saucer_set.usdz

Polyhaven

Test models with .exr textures

Model	Screenshot(s)	Notes
horse_statue_01.geo.usdc
carved_wooden_elephant.geo.usdc
sofa_03.geo.usdc

Sketchfab

Model	Screenshot(s)	Notes
CREATURE MONSTER 3D
Furriers' guild chest in Myślenice
Netsuke - Shoki capturing an oni
compsognathus.usdz

usd-wg test assets

Model	Screenshot(s)	Notes
BrainStem.usdz
CesiumMan.usdz
DamagedHelmet.usdz