SkinnedMesh bug when far from scene root (0,0,0)

[qornflex]

I think I found a bug about your SkinnedMesh (tested on iPhone 6 & 8).

If this is already reported, I'm sorry, I didn't find it in the issue list :(

It seems the gpu skinning is not working correctly and getting crazy on mobile.
When a SkinnedMesh is moving or moved at high value positions (ex: x:0, y:0, z:1000), the skinning is not accurate anymore and starts spider dance.

The scale of the mesh is affecting the bug. Bigger the scale is, lesser the bug.

It seems the skeleton bones values are not calculated correctly at each frame and the bonesTexture/bonesMatrix on the skinning shader is pushing vertices at wrong place.
This is just my feeling of course.

I ran many tests before posting this... looking for a clue in my animated exports but I found the bug is happening with any kind of formats (GLTF, FBX, Collada, JSON, ...) and models from ThreeJS repo.

That's very annoying because that means we are unable to develop a simple runner game with an avatar running (avatar.position.z increasing then) without having this issue :((
I still don't know how I'll manage it as morphTargets is not an option :(

Hope you guys can help here.

I made clear examples with clean source to expose the problem. It's quite easy to verify it on a smart phone:

Appearing only on mobile (z=10000):
http://cornflex.org/webgl/skinning-bug.html

With floatVertexTextures set to false (z=10000):
http://cornflex.org/webgl/skinning-bug2.html

Getting worse with distance (z increasing):
http://cornflex.org/webgl/skinning-bug3.html

Very very far from center (z=70000000) > bug also appearing on desktop but certainly due to float precision issue:
http://cornflex.org/webgl/skinning-bug4.html

Video Preview in my game environment:
This is a realistic scale world (1.0m = 1.0 threejs unit).
Bug is appearing only after 50-60m from scene root and getting worse with distance:
http://cornflex.org/webgl/skin-bug.mp4

VERY IMPORTANT
The mesh used from the ThreeJS repo is way too big. It's like 20m tall.
That's why the z value has to be bigger to see the bug.
If this mesh is scaled down at realistic size, then the bug starts to appear even at 100m.

Three.js version

• Dev
• r89
• ...

Browser

• All of them
• Chrome
• Firefox
• Internet Explorer
• Safari

OS

• All of them
• Windows
• macOS
• Linux
• Android
• iOS

Hardware Requirements (graphics card, VR Device, ...)

iPhone 6, 8

[Mugen87]

Um, your demo looks good on my Pixel.

[donmccurdy]

Confirmed issues on my iPhone SE —

WebGL precision issues in iOS perhaps

[Mugen87]

But the iOS devices actually use the float texture based code path of the skinning implementation, right? Can you verify this with the following conformance test?

https://www.khronos.org/registry/webgl/conformance-suites/1.0.3/conformance/extensions/oes-texture-float.html

[donmccurdy]

iPhone SE has five failures on that page. 😕

[qornflex]

I made a small screencast to show you in my project:
http://cornflex.org/webgl/skin-bug.mp4

If the avatar goes too far from 0,0,0, the skin starts to get really buggy as you can see.
I made a deep profiling of the skeleton and bones. All seems fine except the geometry is "pulled back" in some way... I though about bad animation root but again, all seems fine in Threejs with that...

The prob is clearly linked to the Skeleton and the bonesTexture of the Skinning shader.
I also though about precision but as Mugen said, iOS device should support that with ease.

... why only on iphone then :/

I'm quite stuck... I'm reorienting to png sequences as backup plan ... :(

[titansoftime]

This is most likely due to the iphone 6 not supporting enough bones.

I had the exact same problem.

Check your bone limit here: https://virtulo.us/2/vr/test

[qornflex]

I don't think so.
If the skinnedmesh stays at 0,0,0, there is no prob at all :/
I'm even thinking about moving all the assets instead of the avatar... (this is my workaround #2) but I'd surely prefer to have the skeleton working as good as desktop of course.

A friend just reported the same bug on iPhone 8

[Mugen87]

What results do you get with the following conformance test?

https://www.khronos.org/registry/webgl/conformance-suites/1.0.1/conformance/misc/shader-precision-format.html

[qornflex]

After looking at the ThreeJS sources for 2 days... I think there is a prob when the bonesTexture is updated.
It looks like this texture doesn't udpate correctly and then push back the vertices to original positions... but I could be wrong of course.

[qornflex]

[qornflex]

Hmmm... here I have some FAILED, look:

[Mugen87]

Maybe the problem is related to these fails. My Pixel for instance passes all tests.

Let's try something out: Go to WebGLCapabilities and set the value of floatVertexTextures to false (this will prevent the usage of the float texture). Make a build and use this three.js version in your app. I'm curious what happens 😊.

three.js/src/renderers/webgl/WebGLCapabilities.js

Line 105 in 099e436

floatVertexTextures: floatVertexTextures

[qornflex]

Already tried floatVertexTextures = false on the renderer.
The skin stops going crazy but there is no animation anymore :/
I'll push a sample.

[qornflex]

Here you are:
http://cornflex.org/webgl/skinning-bug2.html

renderer.capabilities.floatVertexTextures = false;

=> No animation running anymore

[Mugen87]

Yeah, same on my Pixel (Desktop works). I guess we are hitting a uniform limit.

[qornflex]

I made a last sample, with bug appearing progressively:
http://cornflex.org/webgl/skinning-bug3.html

Starts from 0,0,0 and goes forward... wait a few seconds... skin starts getting crazy.

It's quite weird all is OK at 0,0,0. isn't it?

[qornflex]

z: 1255 > still OK
z: 18870 > crazy skin

[qornflex]

As I said, I deep profiled all the skeleton and the bones positions on CPU side. All values seem to be quite OK. The only clue I have is the bonesTextures and bonesMatrixes not working correctly on iOS when the mesh is getting far from 0,0,0.

I checked the bones positions because it's a typical behavior of steady bones of the root node or things like that.... but no...

I checked about float precision on GPU side in your skin shaders but again... nothing seems to have an impact.

Very strange nobody moved a SkinnedMesh inside a scene and never reported this :/

[Mugen87]

One more question: Do you have the same problem with Chrome?

[qornflex]

Same prob yeah.

[mrdoob]

Sounds like a precision issue. Have you tried scaling the scene down? (11195 is 11 kilometers).

Another option would be to move the scene instead of the character. Tends to be a common solution for big scenes.

[mrdoob]

Also, I guess the skinning code is in world space. We could investigate this.

[titansoftime]

I see your bone count is 67.

My (wifes) iphone 6 has a Max Hardware Bones limit of 27.

The deformations I see are identical to those I saw on her phone in my game (and the same problem I had before I had my animator fix the meshes armitures). I'd be surprised if this wasn't at least part of your issue.

Why it gets worse over time? No idea. Someone else mentioned something similar, not sure if it's related: https://discourse.threejs.org/t/updated-with-fiddle-as-animations-go-faster-there-is-increasing-error-in-accuracy/1707/6

Another annoying thing I found on iphones (iphone 6 at least): I MUST use highp shader precision, mediump and lowp don't distort the mesh in that same scary looking fashion (as my bone issue on the phone), but the textures look weird and garbled/pixelated while animating.

What is your hardware bone limit on the device you are testing?

[qornflex]

@mrdoob I already tried to scale down but the skinning goes crazy much faster in fact :/
If you scale down at 0.1, then the bug is appearing 10 times faster.

The demo I used here to explain comes from the threejs examples.
In my game, the scale is not that big but the bug is the same, only appearing faster.

Indeed, The skinning seems to be done in worldspace, or localspace but getting stretch when moved far from center. That's why it's running quite well at 0,0,0

Moving all the world instead of the avatar is my workaroung #2.
You must admit it's a bad choice when you developed all the routine of a runner game in worldspace already with physics and all. And z=11000 is not that far for a runner game :/ ... and it's just a running man, not a ship.

I made plenty of games with ThreeJS but it's the first time I use a skinnedmesh. This iOS bug is really annoying.

Workaroung #1: I'll do 6 png sequences as spritesheet animations. Then I can keep a worldspace logic. Deadline is not far :(

@titansoftime The prob is appearing on the iPhone 8 of my wife too. This is not related to iPhone 6 only.
Also, if you have a max bones problem, then it will be buggy from start. Here, you can see it's not buggy at 0,0,0. Then the bones limit is not the prob. Worldspace calculation of the bones and the bonesMatrix in skinning shaders is, IMO.

[titansoftime]

What is the Max Hardware Bones of the iPhone 8?

[qornflex]

I do not know. And I'm not sure it's device dependant.

But again... IMO, it's not related to maxbones value at all.

The sample is working correctly on iphone 6,8,X ... even 5... only when the SkinnedMesh stays at 0,0,0
https://threejs.org/examples/webgl_loader_fbx.html

If maxbones was a problem, we would not have a good skinning at 0,0,0 neither.

[Mugen87]

Related:

https://www.opengl.org/discussion_boards/archive/index.php/t-159613.html
https://www.opengl.org/discussion_boards/archive/index.php/t-159364.html

[Mugen87]

I'd like to highlight this section of the user zeoverlord:

The jittering thing is only natural for floats, variables will loose precision the higher the number is, so precision is the best the closer to 1.0 it is. Now this is not really a problem but it does mean that if you are doing a lot of matrix calculations with huge numbers you are going to have a lot of precision degradation(aka jittering). So you need to do all of your math around the origin with reasonable numbers and then translate it away

[RemusMar]

I have a feeling that it's not just a bug but apple's "solution" to some other problem

Probably true.
Similar with the issue on Firefox Android: bug reported, verified by dev them, but not fixed in the past 5 months (2 releases) !?!

This is how the software development works these days:
instead of fixes in the right place, there are workarounds in the 3rd parties.
Final result: collections of bugs, wasted resources and bloatware !!!

[denbo-ft]

@jfcampos
I would recommend keeping the bone hierarchy in the root and simply adding a function to grab the final WS position of any bone by multiplying it by the martixWorld of the avatar's actual location matrix. It's super simple and will give you back what you had before the fix.

Alternatively, if you have the chops, you can return to the original method (with the bug), include the local matrices in the bone texture, doubling its texture usage, Then modify the shader fragments to do the extra computation on the gpu during skinning. Keep in mind this solution, although "cleaner" from a code perspective, is doubling (possibly quadrupling) the amount of texture that is uploaded to the gpu every frame, and therefore a lot less efficient than the suggestion above. But for whatever reasons you may have that I'm not aware of, it may be more beneficial to you to do it the latter way.

From what was no solution previously, there are now several options here to be compatible with ALL devices, which is an obvious requirement for most projects.

[jfcampos]

Seems like I'll have to go with bones at 0,0,0 and do a few extra matrix calculations for the other stuff. Thanks!

[WestLangley]

@denbo-ft is correct. This is a three.js problem caused by skinning in world space.

For reference, this is where the change to world-space-skinning was made: #4812.

[Fyrestar]

The issue comes from the matrices getting huge numbers from the world position, on some mobile devices the distortion extremely early (using half float i suppose). But even with 32bit it's not as much as JS side with 64bit. The fix of the PR only tries to get the highest precision for the bone texture.

Making the skeleton bones translation local fixes it. However i'm not sure if it's a solution for all use cases. For the regular case of using the bones as a child of the mesh for a character rig there are 2 fixes needed.

Basically at updateMatrixWorldInverse setting the bindMatrixInverse translation zero and in Skeleton update subtracting the root matrixWorld translation from the bone matrices before writing them to the bone texture.

This also could be implemented optionally, there are no shader adaptions needed, the only difference then is the bone calculations on GPU being constant local taking off the translation from the root.