SkinnedMesh: Implement skinning in a different coordinate space #24479

jakezira · 2022-08-09T22:47:30Z

Environment

Code

Added midObj to set the offset.

<!DOCTYPE html>

<html>
	<head>
		<meta charset="utf-8" />
		<title>three-vrm example</title>
		<meta
			name="viewport"
			content="width=device-width, initial-scale=1.0, minimum-scale=1.0, maximum-scale=1.0, user-scalable=no"
		/>
		<style>
			body {
				margin: 0;
			}
			canvas {
				display: block;
			}
		</style>
	</head>

	<body>
		<script src="https://unpkg.com/three@0.143.0/build/three.js"></script>
		<script src="https://unpkg.com/three@0.143.0/examples/js/loaders/GLTFLoader.js"></script>
		<script src="https://unpkg.com/three@0.143.0/examples/js/controls/OrbitControls.js"></script>
		<script>
			// renderer
			const renderer = new THREE.WebGLRenderer();
			renderer.setSize( window.innerWidth, window.innerHeight );
			renderer.setPixelRatio( window.devicePixelRatio );
			document.body.appendChild( renderer.domElement );

			const offset = 100000.0;

			// camera
			const camera = new THREE.PerspectiveCamera( 30.0, window.innerWidth / window.innerHeight, 0.1, 30 );
			camera.position.set( 0.0, offset, 5.0 );

			// camera controls
			const controls = new THREE.OrbitControls( camera, renderer.domElement );
			controls.screenSpacePanning = true;
			controls.target.set( 0.0, offset, 0.0 );
			controls.update();

			// scene
			const scene = new THREE.Scene();

			// light
			const light = new THREE.DirectionalLight( 0xffffff );
			light.position.set( 1.0, 1.0, 1.0 ).normalize();
			scene.add( light );

			// gltf
			const loader = new THREE.GLTFLoader();
			loader.crossOrigin = 'anonymous';

			loader.load(

				// URL of the glb you want to load
				'./models/scilly_drophunter_v31.6_Guilty.glb',

				// called when the resource is loaded
				( gltf ) => {

					const midObj = new THREE.Object3D();
					scene.add( midObj );
					midObj.add( gltf.scene );

					midObj.position.copy(new THREE.Vector3( 0, offset, 0 ));

				},

				// called while loading is progressing
				( progress ) => console.log( 'Loading model...', 100.0 * ( progress.loaded / progress.total ), '%' ),

				// called when loading has errors
				( error ) => console.error( error )

			);

			// helpers
			const gridHelper = new THREE.GridHelper( 10, 10 );
			scene.add( gridHelper );

			const axesHelper = new THREE.AxesHelper( 5 );
			scene.add( axesHelper );

			// update
			function animate() {

				requestAnimationFrame( animate );

				renderer.render( scene, camera );
			}

			animate();
		</script>
	</body>
</html>

Test 1: `offset=0` ==> Correct

Test 2: `offset=100000` ==> Bad rendering

The text was updated successfully, but these errors were encountered:

donmccurdy · 2022-08-09T23:36:59Z

I think the issue remains that three.js implements skinning in world space, which means floating point precision is worse for positions much further from the origin. For now there is no workaround other than to avoid this scenario, e.g. by transforming the scene rather than the character.

WestLangley · 2022-08-09T23:39:59Z

Right. This is the same issue as #13288.

This is a three.js problem caused by skinning in world space.

The change to skinning in world space began in #4812:

Previously, we performed skinning in the local space of the SkinnedMesh. But now, since the objects/bones are not necessarily descendants, it is more natural to perform skinning in world space…

donmccurdy · 2022-08-09T23:54:37Z

Hm. If we believe that skinning should be performed in a local space, then which local space? I'm not sure that the local space of the SkinnedMesh would be the right choice. It's a common use case to transform the root Bone instead of the SkinnedMesh when moving a character, and the Bones may not be descendants of the SkinnedMesh.

I wonder if we could allow the user to define an Object3D as the 'root' (possibly the SkinnedMesh, possibly not) such that skinning would be performed in the local space of that Object3D. We could require that all objects used as bones be descendants of that node, and consider it invalid if this requirement is broken. If no such root is identified, the root would implicitly continue to be the Scene, i.e. skinning in world space.

glTF files have a relevant hint, a skin.skeleton node, that can optionally define a root of the joint hierarchy. I've always assumed that property was not relevant to three.js, but perhaps this is a good use case allowing us to guess the right local frame for a SkinnedMesh.

jakezira · 2022-08-10T00:02:03Z

We really need a workaround ... moving scene instead of avatar doesn't make sense, especially when you're relying on physx etc. We need a consistence game space. Also other objects don't have this problem. It's the implementation limit of skinned mesh of bone chain. Better to provide a solution on skinned mesh.

@donmccurdy

I wonder if we could allow the user to define an Object3D as the 'root' (possibly the SkinnedMesh, possibly not) such that skinning would be performed in the local space of that Object3D. We could require that all objects used as bones be descendants of that node, and consider it invalid if this requirement is broken. If no such root is identified, the root would implicitly continue to be the Scene, i.e. skinning in world space.

This is what I was going to implement. 😁 Looking forward for this fix.

Dadibom · 2022-09-01T19:03:33Z

We really need a workaround ... moving scene instead of avatar doesn't make sense, especially when you're relying on physx etc. We need a consistence game space. Also other objects don't have this problem. It's the implementation limit of skinned mesh of bone chain. Better to provide a solution on skinned mesh.

It is not that uncommon to offset the world space for better floating point precision though. This will not be your only problem when you are trying to run floating point engines at high coordinates.

autra · 2022-09-09T14:19:17Z

It is not that uncommon to offset the world space for better floating point precision though. This will not be your only problem when you are trying to run floating point engines at high coordinates.

We do at giro3d (because it's a geographic engine, dealing with coordinates in the millions sometimes). Yes there are other classes of problems, but so far they have been manageable without shifting the scene. Our use case is typically the same as @jakezira but worst. For instance, this is supposed to be the 3 Soldiers of https://threejs.org/examples/webgl_animation_multiple:

From what I understand the solution proposed by @jakezira should fix it for us as well.

sunag · 2022-09-09T18:45:12Z

Maybe a temporary solution can add an attribute vec3 globalPosition after skinning_vertex?

...
#include <skinning_vertex>
transformed += globalPosition;
...

I haven't tested it, but in my mint it should work...

donmccurdy · 2022-09-10T18:36:19Z

It may be the joints rather than the mesh that have a large displacement. In the case of VRM or glTF I think it's more likely the joints; skinned mesh displacement should be ignored.

jakezira changed the title ~~Render skinned mesh object at position of large values distorted the vertices position~~ Render skinned mesh object at position of large values distorts the vertices position Aug 9, 2022

jakezira mentioned this issue Aug 9, 2022

Logarithmic depth buffer for avatars? webaverse/app#3490

Open

Mugen87 added the Suggestion label Aug 10, 2022

Mugen87 changed the title ~~Render skinned mesh object at position of large values distorts the vertices position~~ Consider to implement skinning in a different coordinate space. Aug 10, 2022

Mugen87 changed the title ~~Consider to implement skinning in a different coordinate space.~~ Implement skinning in a different coordinate space. Aug 10, 2022

donmccurdy mentioned this issue Dec 28, 2022

Add Mesh.getVertexPosition #25049

Merged

donmccurdy changed the title ~~Implement skinning in a different coordinate space.~~ SkinnedMesh: Implement skinning in a different coordinate space Dec 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SkinnedMesh: Implement skinning in a different coordinate space #24479

SkinnedMesh: Implement skinning in a different coordinate space #24479

jakezira commented Aug 9, 2022

donmccurdy commented Aug 9, 2022

WestLangley commented Aug 9, 2022

donmccurdy commented Aug 9, 2022

jakezira commented Aug 10, 2022 •

edited

Dadibom commented Sep 1, 2022

autra commented Sep 9, 2022

sunag commented Sep 9, 2022

donmccurdy commented Sep 10, 2022 •

edited

SkinnedMesh: Implement skinning in a different coordinate space #24479

SkinnedMesh: Implement skinning in a different coordinate space #24479

Comments

jakezira commented Aug 9, 2022

Environment

Code

Test 1: offset=0 ==> Correct

Test 2: offset=100000 ==> Bad rendering

donmccurdy commented Aug 9, 2022

WestLangley commented Aug 9, 2022

donmccurdy commented Aug 9, 2022

jakezira commented Aug 10, 2022 • edited

Dadibom commented Sep 1, 2022

autra commented Sep 9, 2022

sunag commented Sep 9, 2022

donmccurdy commented Sep 10, 2022 • edited

Test 1: `offset=0` ==> Correct

Test 2: `offset=100000` ==> Bad rendering

jakezira commented Aug 10, 2022 •

edited

donmccurdy commented Sep 10, 2022 •

edited