Add Mesh.getVertexPosition

[elalish]

Somewhat related issues: #18334, #14499

Description

I'm implementing dynamic hotspots for model-viewer, which requires me to find the current position of a vertex in a possibly animated mesh. Three.js does this internally for raycasting, but doesn't expose the method. I've refactored slightly to expose it and slightly simplify the raycasting code. I'm pretty sure I've seen forum discussions where this feature was asked for, so hopefully I'm not the only user. I have tested this (and the updated reduceVertices) on my dynamic hotspots branch of model-viewer and it's all working properly, handling morph targets, skinned meshes, etc.

[gkjohnson]

It would be nice to make this generic so it works across any vertex attribute (normal, tangent, etc). For three-mesh-bvh (and ultimately three-gpu-pathtracer) the StaticGeometryGenerator applies the morph target and skinned transforms manually:

https://github.com/gkjohnson/three-mesh-bvh/blob/7941ca674f3aad85fae13d00050ddc69b6675585/src/utils/StaticGeometryGenerator.js#L458-L497

[mrdoob]

Handy! How about getVertexPosition?

[gkjohnson]

Handy! How about getVertexPosition?

Can this be something like getVertexAttribute( attrName, index, target ) so it can be used for any skinned or morphed attribute? Or perhaps getVertexPosition can be added along side getVertexAttribute for convenience?

[elalish]

@gkjohnson That sounds like a good idea to me. @mrdoob, would you prefer getVertexAttribute or getVertexPosition, getVertexNormal, and getVertexTangent? Either way I can update this and merge it for the position part and @gkjohnson can add the others; sound good?

[gkjohnson]

or getVertexPosition, getVertexNormal, and getVertexTangent?

Heh, just position, normal, and tangent are not necessarily enough. All vertex attributes include color, uv, and other custom attributes can all be morphed, no? I think it should be a general function and then implentations like getVertexPosition can look like so if needed:

getVertexPosition( i, target ) {

  return this.getVertexAttribute( 'position', i, target );

}

[elalish]

@mrdoob I've updated this to getVertexPosition; I think you can safely merge this to allow @gkjohnson to add his related methods more easily as a follow-on PR.

[WestLangley]

Linting:

The indentation is incorrect.

Also, it should be ! child.isSkinnedMesh -- with a space.

[elalish]

Oh, good call; I thought the auto-linting would catch that kind of thing now.

[WestLangley]

Unrelated: reduceVertices() does not appear to be documented.

In what coordinate space is the returned position in getVertexPosition()?

[elalish]

Unrelated: reduceVertices() does not appear to be documented.

Sorry about that, I'll add it.

In what coordinate space is the returned position in getVertexPosition()?

It's in local space. Honestly though, I feel like there's some kind of bug, or something I don't understand, because I don't think that if ( ! child.isSkinnedMesh ) should be needed at all, yet it very much breaks when I remove it. The raycasting code doesn't need that, but I can't figure out what the difference is. I wonder if the updateMatrixWorld is somehow not updating the bone matrices?

[WestLangley]

I feel like there's some kind of bug, or something I don't understand

Please, please pursue this until it is understood.

[donmccurdy]

if ( ! child.isSkinnedMesh ) {

    vertex.applyMatrix4( child.matrixWorld );

}

The glTF specification requires that the world transform of the SkinnedMesh have no direct effect on the final position of a vertex during skinning. Only the world transform of the bones should be used. The bones may or may not be descendants of the SkinnedMesh.

The code above seems correct from this standpoint, if that's our intention for three.js as well.

[elalish]

Ah, thank you for that explanation @donmccurdy, that mostly makes sense. I think that means getVertexPosition returns things in a different coordinate space depending on whether the mesh is skinned or not, which is confusing. I'm guessing the reason it works with raycasting is that the ray is also transformed into two different possible coordinate spaces to match (though I have yet to figure out where that happens). @WestLangley do you think a small refactor is in order?

[WestLangley]

@elalish Skinning in three.js is implemented in world space on the GPU. As I have said elsewhere, I think that is unadvisable due to the precision problems it can introduce.

[donmccurdy]

• Related: SkinnedMesh: Implement skinning in a different coordinate space #24479 (comment)

[WestLangley]

Also related: #24479 (comment).

[donmccurdy]

Whatever we decide for skinning computations, the intention is for reduceVertices to produce output in world space, correct? We should clarify in documentation what boneTransform produces, too.