Material: Feature request to use world coordinates of vertex as UV

[finnbear]

Description of the problem

There is an abundance of textures that can be repeated without showing seams. However, if they are applied to anything but the simplest of shapes (squares in a perfect grid), seams will re-emerge. One important feature of seamless textures is that the specific UVs are irrelevant. What matters is that the UVs are consistent between multiple shapes.

In the image, a "seamless" texture has been applied to a grid of hexagons (one of the 3 possible regular polygons that can form a grid). Circled are the intersecting hexagon vertices and arrows point to the edge. Seams have been reintroduced due to the fact that the texture was designed to be seamless over a square, not a hexagon. Some textures, especially ones with higher contrast, appear much worse with these seams.

Using debug texture to highlight that, while specific UVs work for grids of squares, they don't for grids of hexagons (or triangles for that matter):

Describe the bug or feature request in detail

One way to achieve this consistency is the use a global coordinate system to drive the UVs, such as scene/world coordinates. There is a slight complication that the axis that translates world to uv coordinates is relevant to the process. Therefore, I propose that the axis be configurable. The lack of such an axis would indicate that the geometry's specific, local UVs should be used.

Potential API: material.uvsAxis = quaternion | plane | vector (normal to plane)

Alternate API: material.uvsAxis = THREE.XZ | THREE.XY | THREE.YZ (only support 3 axes)

Alternate API: geometry.uvsAxis = (see above); material.useUvsAxis = true (allow each geometry to have its own UV axis.

Both would solve my problem, with the first being highly flexible. To be clear, the implementation would project the world coordinate onto the axis to determine the resulting UV coordinate.

Not a solution: User having to manually set the UVs of each vertex of each mesh at all times. This approach doesn't support InstancedMesh, where each instance has the same UVs. It is also needlessly difficult.

Three.js version

As long as I can remember

Browser

• All of them

OS

• All of them

[donmccurdy]

You can accomplish this with custom shaders, or with the node material system in the examples:

const material = new THREE.StandardNodeMaterial();
const uv = new SwitchNode( new PositionNode(), 'xy' );
material.color = new TextureNode( texture, uv );

I don't think world coordinates as UVs is likely to be supported in a way other than the ideas above — the problem of selecting an existing UV set for specific textures has been stalled for a while, even though it is easier and more common.

[finnbear]

Thanks for the reply! The node material thing might solve the problem, but I don't see any documentation on that feature. Guessing its still in dev. Somethings I would like to know from the documentation is if it would support fog, which I use extensively, and whether it would support normal maps. I'm guessing the answer to both is yes.

I know I could make a custom shader but that would lose the benefit of fog and normal maps. I was hoping the functionality I'm suggesting seems worth it to integrate into the engine itself. If not, that's ok. I'll have to find another way, or live with the seams.

[Mugen87]

The node material thing might solve the problem, but I don't see any documentation on that feature. Guessing its still in dev.

Yes, but there are already issues tracking the development and documentation: #7522 and #17971.

And yes, fog and normal maps are supported. There is a list of examples that demonstrate the various features of this new material: https://threejs.org/examples/?q=nodes

[finnbear]

This issue is pretty important to me so I wrote a PR, my first for this repository. Please tell me how I can increase the chances of getting it merged 😃

[Mugen87]

TBH, I don't vote to add this logic to the renderer. It's too application specific. Consider to use the presented solution with node materials or modify the shaders on application level via Material.onBeforeCompile().

I'm also not aware of any other engine that supports such a mode.

[finnbear]

Material.onBeforeCompile() looks promising, I'll give that a try. Unfortunately, such materials can't be cloned, which is something my application uses.

As for other engines:

• I use world coordinates extensively in Blender (which supports it via a node, but that's Blender's primary mode for shaders)
• There is demand for the feature in Unity: https://gamedev.stackexchange.com/questions/136652/uv-world-mapping-in-shader-with-unity
• Someone went through the trouble of making several code snippets for Unity: http://wiki.unity3d.com/index.php/WorldUVs
• Shader forge page: http://www.acegikmo.com/shaderforge/wiki/index.php?title=World-Space_UV_Mapping
• Another random forum post: https://forum.unity.com/threads/mapping-texture-to-world-position-instead-of-object-position.94766/
• Random video, same use case as me: https://www.youtube.com/watch?v=s79Zu0F8fuY
• Documentation for Unreal Engine has an example of world -> uv: https://docs.unrealengine.com/en-US/Engine/Rendering/Materials/CustomizedUVs/index.html
  

You might be right that there are no other engines that have such a "mode" (in the literal sense of the word) but that doesn't mean there is not demand for the feature. Others have clearly spent time working around the limits of their respective engines to use this feature.

Edit: For what it's worth, one reason I like the Material.onBeforeCompile() approach is that it allows the same texture to be shared between materials with world mapping and those with UV mapping (my scene uses this). While writing my PR, it seemed appropriate to use Texture.mapping because that already existed. However, the API might be better if it was on a Material level (like my original feature request). On a related note, mapping, repetition, and offset are currently properties of Texture but might be better as properties of Material. The rendering code currently derives all of these properties from a single texture per material, according to a priorities hierarchy.

[Mugen87]

You might be right that there are no other engines that have such a "mode" (in the literal sense of the word) but that doesn't mean there is not demand for the feature.

The node material approach presented here #18992 (comment) is equivalent to what other engines/tools offer like e.g. Blender. I would say that is sufficient and does not justify a modification of the core materials.

[WestLangley]

Here is how to implement this feature request in NodeMaterial, based on the example proposed in #19010.

EDIT: Actually, this was a first-attempt -- without instancing.

var hexMaterial = new Nodes.StandardNodeMaterial();
hexMaterial.side = THREE.DoubleSide;
hexMaterial.fog = true;

var scale = new Nodes.FloatNode( .1 );

var uv = new Nodes.SwitchNode( new Nodes.PositionNode( Nodes.PositionNode.WORLD ), 'xz' );

var uvPos = new Nodes.OperatorNode( uv, scale, Nodes.OperatorNode.MUL );

hexMaterial.color = new Nodes.TextureNode( worldUvTexture, uvPos );

[WestLangley]

In the end, it was trivial to add this feature and support instancing by using code-injection in the built-in material. NodeMaterial was not used.

In meshphysical_vert.glsl.js:

#include <worldpos_vertex>

vec4 wPos = instanceMatrix * vec4( transformed, 1.0 );
vUv = ( uvTransform * vec3( wPos.xz, 1 ) ).xy;

[finnbear]

@WestLangley could you elaborate on what you mean by 'code-injection'? It looks very promising. Is that something I could do programmatically?

Do you mean defining a onBeforeCompile hook that replaces an entire function with that?

Also, 1) doesn't <worldpos_vertex> already multiply by the instanceMatrix? 2) you are not using the output of worldPosition, so why import <worldpos_vertex> in the first place?

three.js/src/renderers/shaders/ShaderChunk/worldpos_vertex.glsl.js

Lines 1 to 15 in 7c1424c

	export default /* glsl */`
	#if defined( USE_ENVMAP ) || defined( DISTANCE ) || defined ( USE_SHADOWMAP )
	vec4 worldPosition = vec4( transformed, 1.0 );
	#ifdef USE_INSTANCING
	worldPosition = instanceMatrix * worldPosition;
	#endif
	worldPosition = modelMatrix * worldPosition;
	#endif
	`;

The reason I would prefer modifying a normal material is that, having just tried the node approach, it has significant drawbacks (despite solving this one problem). Most notably, the resulting materials are not drop-in replacements that have familiar properties like transparent, opacity, and color tint. My application is not set up to handle two different types of materials, or at least it isn't yet. Other problem is, as you mention, the approach doesn't support instancing.

[finnbear]

Here's where I am now:

material.onBeforeCompile = (shader) => {
    shader.vertexShader = shader.vertexShader.replace('#include <uv_vertex>', 'vec4 worldPos = vec4(position, 1.0);\n#ifdef USE_INSTANCING\nworldPos = instanceMatrix * worldPos;\n#endif\nvUv = ( uvTransform * vec3( worldPos.xz, 1 ) ).xy;');
};

I guess my question is: Were you suggesting something different/better than this?

[WestLangley]

In my proof-of-concept, I just hacked the shader by adding two lines after <worldpos_vertex>.

But, yes, in practice, you would use material.onBeforeCompile().

[finnbear]

In my proof-of-concept, I just hacked the shader by adding two lines after <worldpos_vertex>.

But, yes, in practice, you would use material.onBeforeCompile().

Thanks for the clarification. The issue I'm now dealing with is that onBeforeCompile() executes on the shader before three.js templates in everything. As a result, the #define USE_INSTANCING doesn't seem to be present. I would have liked to use the same material for instanced objects and non-instanced objects (world coordinates were supposed to make it universal) but maybe that's not an option.

Edit: Nvm, just remembered that that information isn't needed in the JS runtime - its used by the shader compiler. Not sure why the math didn't work then...retrying.

Edit: Turns out the matrix multiplication with instanceMatrix was occuring but that doesn't mean the shader works for some reason. Fixing this seam issue is slowly becoming a nightmare 😉 \

Edit: Looks like I was trying to insert code too early, and position hadn't been multiplied by the associated matrices to get world position.

Edit: Working nicely

Thanks for the advice! (It might be ideal to have this logic in the engine but this solution works)

For reference, my final solution was:

material.onBeforeCompile = (shader) => {
    shader.vertexShader = shader.vertexShader.replace('#include <uv_vertex>\n', '').replace('#include <worldpos_vertex>', 'vec4 worldPosition = vec4( transformed, 1.0 );\n#ifdef USE_INSTANCING\nworldPosition = instanceMatrix * worldPosition;\n#endif\nworldPosition = modelMatrix * worldPosition;\nvUv = (uvTransform * vec3(worldPosition.xz, 1)).xy;');
};