Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WebGLRenderer: Tight morph target packing. #27768

Open
wants to merge 5 commits into
base: dev
Choose a base branch
from
Open

WebGLRenderer: Tight morph target packing. #27768

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
83d7f36
tight morph target packing
wizgrav Feb 18, 2024
9d48b7a
remove redundant assignment
wizgrav Feb 19, 2024
28e55a0
updated example with variable instance count
wizgrav Feb 19, 2024
eef9f15
update screenshot
wizgrav Feb 19, 2024
bda507e
fix width calculation when > maxTextureSize
wizgrav Feb 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Binary file modified examples/screenshots/webgl_instancing_morph.jpg
View file
Open in desktop
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
211 changes: 103 additions & 108 deletions examples/webgl_instancing_morph.html
View file
Open in desktop
@@ -1,12 +1,24 @@

<!DOCTYPE html>
<html lang="en">
<head>
<title>three.js webgl - instancing - Morph Target Animations</title>
<title>three.js webgl - morph targets - instancing</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0">
<link type="text/css" rel="stylesheet" href="main.css">
<style>
body {
background-color: #666666;
}
</style>
</head>
<body>

<div id="info">
<a href="https://threejs.org" target="_blank" rel="noopener">three.js</a> webgl - morph targets - instancing<br/>
model by <a href="https://www.bannaflak.com/face-cap" target="_blank" rel="noopener">Face Cap</a>
</div>

<script type="importmap">
{
"imports": {
Expand All @@ -20,176 +32,159 @@

import * as THREE from 'three';

import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';

import Stats from 'three/addons/libs/stats.module.js';

let camera, scene, renderer, stats, mesh, mixer, dummy;

const offset = 5000;

const timeOffsets = new Float32Array( 1024 );

for ( let i = 0; i < 1024; i ++ ) {

timeOffsets[ i ] = Math.random() * 3;
import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';
import { KTX2Loader } from 'three/addons/loaders/KTX2Loader.js';
import { MeshoptDecoder } from 'three/addons/libs/meshopt_decoder.module.js';

}
import { RoomEnvironment } from 'three/addons/environments/RoomEnvironment.js';

const clock = new THREE.Clock( true );
import { GUI } from 'three/addons/libs/lil-gui.module.min.js';

init();
animate();

function init() {

camera = new THREE.PerspectiveCamera( 60, window.innerWidth / window.innerHeight, 100, 10000 );

scene = new THREE.Scene();
let mixer, mesh, head, duration;
const clock = new THREE.Clock( true );

scene.background = new THREE.Color( 0x99DDFF );
const container = document.createElement( 'div' );
document.body.appendChild( container );

scene.fog = new THREE.Fog( 0x99DDFF, 5000, 10000 );
const camera = new THREE.PerspectiveCamera( 45, window.innerWidth / window.innerHeight, 1, 500 );
camera.position.set( 0, 0, 0 );

const light = new THREE.DirectionalLight( 0xffffff, 1 );
const scene = new THREE.Scene();

light.position.set( 200, 1000, 50 );

light.castShadow = true;
const renderer = new THREE.WebGLRenderer( {
antialias: true,
powerPreference: 'high-performance'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to set this to 'high-performance'?

} );

light.shadow.camera.left = - 5000;
light.shadow.camera.right = 5000;
light.shadow.camera.top = 5000;
light.shadow.camera.bottom = - 5000;
light.shadow.camera.far = 2000;
renderer.setPixelRatio( window.devicePixelRatio );
renderer.setSize( window.innerWidth, window.innerHeight );
renderer.toneMapping = THREE.ACESFilmicToneMapping;

light.shadow.bias = - 0.01;
container.appendChild( renderer.domElement );

light.shadow.camera.updateProjectionMatrix();
const ktx2Loader = new KTX2Loader()
.setTranscoderPath( 'jsm/libs/basis/' )
.detectSupport( renderer );

scene.add( light );
new GLTFLoader()
.setKTX2Loader( ktx2Loader )
.setMeshoptDecoder( MeshoptDecoder )
.load( 'models/gltf/facecap.glb', ( gltf ) => {

const hemi = new THREE.HemisphereLight( 0x99DDFF, 0x669933, 1 / 3 );
const orig = gltf.scene.children[ 0 ];

scene.add( hemi );
mixer = new THREE.AnimationMixer( orig );

const ground = new THREE.Mesh(
new THREE.PlaneGeometry( 1000000, 1000000 ),
new THREE.MeshStandardMaterial( { color: 0x669933, depthWrite: true } )
);
mixer.clipAction( gltf.animations[ 0 ] ).play();

ground.rotation.x = - Math.PI / 2;
duration = gltf.animations[ 0 ].duration;

// GUI

ground.receiveShadow = true;
head = orig.getObjectByName( 'mesh_2' );

scene.add( ground );
mesh = new THREE.InstancedMesh( head.geometry, head.material, 128 );

const loader = new GLTFLoader();
mesh.setMatrixAt( 0, head.matrix );

loader.load( 'models/gltf/Horse.glb', function ( glb ) {
mesh.instanceMatrix.setUsage( THREE.DynamicDrawUsage );

dummy = glb.scene.children[ 0 ];
mesh.setMorphAt( 0, head );

mesh = new THREE.InstancedMesh( dummy.geometry, dummy.material, 1024 );
const gridParams = {
width: 3,
height: 2
};

mesh.castShadow = true;
function updateGrid() {

for ( let x = 0, i = 0; x < 32; x ++ ) {
mesh.count = gridParams.width * gridParams.height;

for ( let y = 0; y < 32; y ++ ) {
const w = gridParams.width;
const h = gridParams.height;

dummy.position.set( offset - 300 * x + 200 * Math.random(), 0, offset - 300 * y );
for ( let i = 0; i < h; i ++ ) {

dummy.updateMatrix();
for ( let j = 0; j < w; j ++ ) {

mesh.setMatrixAt( i, dummy.matrix );
head.position.set( j - w / 2, i - h / 2 + 1, - Math.max( w, h ) - 2 ).multiplyScalar( 25 );

mesh.setColorAt( i, new THREE.Color( `hsl(${Math.random() * 360}, 50%, 66%)` ) );
head.rotation.x = Math.PI / 2;

i ++;

}
head.updateMatrix();


}
mesh.setMatrixAt( j + w * i, head.matrix );

scene.add( mesh );
}

mixer = new THREE.AnimationMixer( glb.scene );
mesh.instanceMatrix.needsUpdate = true;

const action = mixer.clipAction( glb.animations[ 0 ] );
}

action.play();
}

} );

//
scene.add( mesh );

renderer = new THREE.WebGLRenderer( { antialias: true } );
renderer.setPixelRatio( window.devicePixelRatio );
renderer.setSize( window.innerWidth, window.innerHeight );
document.body.appendChild( renderer.domElement );
renderer.shadowMap.enabled = true;
renderer.shadowMap.type = THREE.VSMShadowMap;
//
const gui = new GUI();

stats = new Stats();
document.body.appendChild( stats.dom );
gui.add( gridParams, 'width', 2, 16 ).name( 'Grid width' ).step( 1 ).onChange( updateGrid );
gui.add( gridParams, 'height', 2, 8 ).name( 'Grid height' ).step( 1 ).onChange( updateGrid );

//
updateGrid();

window.addEventListener( 'resize', onWindowResize );

}
} );

function onWindowResize() {
const environment = new RoomEnvironment( renderer );
const pmremGenerator = new THREE.PMREMGenerator( renderer );

camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
scene.background = new THREE.Color( 0x666666 );
scene.environment = pmremGenerator.fromScene( environment ).texture;

renderer.setSize( window.innerWidth, window.innerHeight );

}

//
const stats = new Stats();
container.appendChild( stats.dom );

function animate() {
renderer.setAnimationLoop( () => {

requestAnimationFrame( animate );
const time = clock.getElapsedTime();

render();
if ( mixer ) {

stats.update();

}

function render() {

const time = clock.getElapsedTime();

const r = 3000;
camera.position.set( Math.sin( time / 10 ) * r, 1500 + 1000 * Math.cos( time / 5 ), Math.cos( time / 10 ) * r );
camera.lookAt( 0, 0, 0 );
for ( let i = 0; i < mesh.count; i ++ ) {
mixer.setTime( time + i * duration / mesh.count );
mesh.setMorphAt( i, head );
}
mesh.morphTexture.needsUpdate = true;

}

if ( mesh ) {
renderer.render( scene, camera );

for ( let i = 0; i < 1024; i ++ ) {
stats.update();

mixer.setTime( time + timeOffsets[ i ] );
} );

mesh.setMorphAt( i, dummy );
window.addEventListener( 'resize', () => {

}
camera.aspect = window.innerWidth / window.innerHeight;

mesh.morphTexture.needsUpdate = true;
camera.updateProjectionMatrix();

}
renderer.setSize( window.innerWidth, window.innerHeight );

renderer.render( scene, camera );
} );

}

</script>

</body>
</html>
26 changes: 24 additions & 2 deletions src/renderers/shaders/ShaderChunk/morphtarget_pars_vertex.glsl.js
View file
Open in desktop
Expand Up @@ -20,12 +20,34 @@ export default /* glsl */`

vec4 getMorph( const in int vertexIndex, const in int morphTargetIndex, const in int offset ) {

int texelIndex = vertexIndex * MORPHTARGETS_TEXTURE_STRIDE + offset;
int texelIndex = vertexIndex * MORPHTARGETS_TEXTURE_STRIDE + 3 * offset;
Copy link
Collaborator

@Mugen87 Mugen87 Feb 19, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to multiply the offset with 3 now?

Copy link
Contributor Author

@wizgrav wizgrav Feb 19, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To keep in line with what we previously had on the shader chunks that call getMorph. Now the position is at offset 0, normals at offset 3 and colors at offset 6. Alternatively we can modify the other shader chunks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid any misunderstanding, you want me to change the other shader chunks to eliminate the multiplication with 3? At some point we will probably have to revisit that part anyway if/when we add additional morphing attributes like uv. When that time comes more extensive changes on how we handle morphing will be needed because now it's a bit fixed like if there's a color morph but not normals the latter will be empty space in the texture. This is how it used to be even before this PR but we can make it more flexible when more attributes are introduced

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep the current version of this PR 👍 .


int y = texelIndex / morphTargetsTextureSize.x;
int x = texelIndex - y * morphTargetsTextureSize.x;

ivec3 morphUV = ivec3( x, y, morphTargetIndex );
return texelFetch( morphTargetsTexture, morphUV, 0 );
Copy link
Collaborator

@Mugen87 Mugen87 Feb 19, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are multiple fetches really faster than a single one? I'm afraid this needs to be tested one more than one device. Especially mobile devices could show a performance regression here.

Copy link
Contributor Author

@wizgrav wizgrav Feb 19, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it is currently with rgba format it will always fetch 4 floats so fetching 3 instead should always be faster especially on mobiles where bandwidth is precious. I'll replace the horse example with the faces one and do some more testing but I only have one android phone with an older snapdragon and the quest 3 which all show similar improvements

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, you could also add a slider to webgl_instancing_morph that controls the number of instances.

Copy link
Contributor Author

@wizgrav wizgrav Feb 19, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to have the faces instead because they are using more influences and also morph the normals so its a more complete case than the horses.


vec4 ret = vec4(0.);

ret.x = texelFetch( morphTargetsTexture, morphUV, 0 ).r;

morphUV.x++;

ret.y = texelFetch( morphTargetsTexture, morphUV, 0 ).r;

morphUV.x++;

ret.z = texelFetch( morphTargetsTexture, morphUV, 0 ).r;

#if MORPHTARGETS_TEXTURE_STRIDE == 10

morphUV.x++;

ret.a = offset == 2 ? texelFetch( morphTargetsTexture, morphUV, 0 ).r : 0.;

#endif

return ret;

}

Expand Down