Displacement Effect #5573

defektu · 2023-08-17T18:57:20Z

@mvaligursky requested PR from #5554 to look into it.

Known issues with PR is in issues thread.

It might be a poor implementation in terms of engine's workflow.

LitShader-forward-proc & LitShader-ShadowPass_1_0-proc does not get #define DISPLACEMENT argument possibly because of my poor implementation.

Project file in #5554 works perfectly (probably due to shader chunk rebuilds when replacing a chunk)

Did not implemented checks for implementation cause of draft PR

I confirm I have read the contributing guidelines and signed the Contributor License Agreement.

mvaligursky · 2023-08-17T19:17:35Z

Would you have any way to use this? Engine example or Editor project?

defektu · 2023-08-17T19:31:31Z

Scene for changing transformVS
and engine example with use_local_engine

mvaligursky · 2023-08-18T09:37:08Z

I had a quick look, and I think:

you should use heightMap for this instead of introducing new texture channel. The useDisplacement would then use this map as displacement instead of for parallax.
I think you need something like this to have normals attribute added to the vertex shader:

Give these a go please and let me know if there are further issues - there might be with supplying UVs to the vertex shader too, but maybe not with the heightMap.

mvaligursky · 2023-09-11T08:50:36Z

Hi @defektu - any thoughts / progress on this? It'd be great to finish off if possible, thanks.

defektu · 2023-09-11T17:35:02Z

Sorry @mvaligursky, nowadays are a bit rough workwise.

Using heights was possible but uv's and texture is not passing through to shadowpass. (vertex's are displacing but shadow pass won't in this case.)

Shadows are rendered through Standart material shader as far as I inspected. Build process won't take the litOption arguments (like texture_shadowMap) while shadows are processing (useHeights=null). Or I might doing something wrong.

I can explore any thoughts in my spare time.

defektu · 2023-09-12T13:22:54Z

Today I noticed standart.js has // all other passes require only opacity part. So ShadowPass is not getting any info about anything except opacity.

mvaligursky · 2023-09-12T13:26:16Z

Yep, we'll need to adjust this to pass on the height if used as well.

defektu · 2023-09-12T13:30:46Z

Also vertex_normals needed. But it does not pass into shadow pass as semantics as well Vertex shader attribute "vertex_normal" is not mapped to a semantic in shader definition, shader [Shader Id 2 LitShader-ShadowPass_1_0-proc]

defektu · 2023-09-12T14:16:32Z

Shaderpass does not get any litOptions (turns out to be undefined)
getShaderVariant passes this.shaderOptBuilder.updateMinRef if it is not FORWARD_PASS. So do you have any suggestions about this topic. Besides shadowPass it is working as intended (displacing the surface).

mvaligursky · 2023-09-12T14:29:58Z

I suspect the StandardMaterialOptionsBuilder will need to be updated to generate required options in the shadow pass - probably in _updateSharedOptions, which I think is shared between forward and shadow pass, but not sure now based on you seeing litOptions being undefined.

Perhaps update the PR with what you have, using the height texture, and I'll see how I can fix the shadows using your example.

mesh-displacement example added.

defektu · 2023-09-14T19:21:07Z

Rolled back previous changes and added mesh-displacement example draft.

defektu · 2023-09-14T21:04:52Z

Somehow I got it working with shared options. I will cleanup everything and send PR.

This change is for temporary purposes. DISPLACEMENT definition is must (otherwise displacementVS must be created) minimalOptions is disabled due to shaderpass definitions.

defektu · 2023-09-14T21:51:17Z

Pushed working version with minimalOptions = false. It is working as intended like this.

defektu · 2023-09-15T21:47:54Z

Effect fully works for now but noticed env goes blank (weird frustrum culling behaviour causes that)
Can't state if it is caused by PR or something else.

mvaligursky · 2023-09-15T22:06:22Z

I fixed the culling that few days ago .. you're just probably not up to date.
Is it all ready for the review you reckon? Ping me when it is, I'll check it out, thanks!

defektu · 2023-09-15T23:39:58Z

Yes it is up and running @mvaligursky. UV transformations are not implemented. displacementOffset is nice to have and can be implemented by introducing new material attribute.

Displacement offset is introduced as material attribute.

defektu · 2023-09-15T23:56:53Z

Pushed new commit for displacementOffset too.

forgot console.log

mesh-displacement example added.

This change is for temporary purposes. DISPLACEMENT definition is must (otherwise displacementVS must be created) minimalOptions is disabled due to shaderpass definitions.

defektu · 2023-09-18T13:17:52Z

There is specularityFactor. Since effect uses heightMap, displacementFactor and displacementOffset is good?

src/scene/materials/standard-material.js

mvaligursky · 2023-09-18T13:22:35Z

src/scene/shader-lib/chunks/common/vert/transform.js

 #endif
 vec4 getPosition() {
    dModelMatrix = getModelMatrix();

    #ifdef DISPLACEMENT
    vec4 dMapSample = texture2D(texture_heightMap, vertex_texCoord0.xy);
-    float displacer = dMapSample.r * material_heightMapFactor * 10.;
+    float displacer = dMapSample.r * material_heightMapFactor * 10. + material_displacementOffset;


I'm not sure about the *10 part here?

That can be removed when material_heightMapFactor is replaced with new attribute.

Wouldn't it be more useful for vertex displacement to be implemented as an overridable shader chunk instead?

I am not sure, you are the guys to call the shot.

mvaligursky · 2023-09-18T13:24:04Z

There is specularityFactor. Since effect uses heightMap, displacementFactor and displacementOffset is good?

yeah actually, I missed you added an offset too. Sounds good in that case to me.

useDisplacement flag is setting from `displacementFactor !== 0`

- Fixed useDisplacement flag - Minified mesh-displacement example

defektu · 2023-09-19T00:30:38Z

Hey @mvaligursky, Updated PR & ready for review.

Decided to keep useDisplacement attribute for code readability. It sets based on displacementFactor. Cleaned up mesh-displacement example.

I guess it is good to keep useDisplacement attribute in material parameters for debugging?

Not a code dev mainly so ignore commit chains <3

examples/src/examples/graphics/mesh-displacement.tsx

src/scene/materials/standard-material-options-builder.js

src/scene/materials/standard-material.js

src/scene/shader-lib/chunks/common/vert/transform.js

src/scene/materials/standard-material-options-builder.js

defektu · 2023-09-28T08:26:54Z

Added draft docs for attributes.

Things to consider:

This effect uses UV0 and R channel of height map.
You can't offset or transform map yet.
If there is no heightMap, useDisplacement will break vertex chunk.

willeastcott · 2023-10-29T12:36:13Z

Hey @mvaligursky and @slimbuck - can we sync on this PR? I'm keen to make a decision on what to do with it.

Maksims · 2023-10-29T15:09:05Z

Is there a way to change heightMap channel for displacement?
Also will it apply tiling and offsets?

Another question, will it be viable/common when devs use displacement in combination with parallax? In such case the heightMap textures might be different or the same?

defektu · 2023-10-31T18:22:27Z

Hi @Maksims, I think this vertex shader needs some special attributes for offsetting and tiling (which is totally possible). Concurrent options are bound to fragment shader. Don't think that back propagation is not possible unless changing where uniforms defined.

Not really experienced with engine, so this needs crosscheck.

defektu · 2023-12-16T07:10:13Z

Back from service. Ready to do changes if needed.

feature(Displacement): Init

cb69ae3

defektu mentioned this pull request Aug 17, 2023

Feature Request - Displacement Map #5554

Open

fix(displacement): rollback & heightMap replacement

bab5a3d

mesh-displacement example added.

fix(displacement): working state with temporary flag

778ed42

This change is for temporary purposes. DISPLACEMENT definition is must (otherwise displacementVS must be created) minimalOptions is disabled due to shaderpass definitions.

defektu added 2 commits September 15, 2023 21:47

chore(mesh-displacement): updated mesh-displacement example

7f7b099

fix(standart-material): displacement effect fully works now.

8571a0c

feature(displacement-effect): displacement offset

4321fb1

Displacement offset is introduced as material attribute.

defektu added 6 commits September 16, 2023 03:03

fix(standart-material): lint

70eb2f7

forgot console.log

feature(Displacement): Init

0771893

fix(displacement): rollback & heightMap replacement

c6daa4c

mesh-displacement example added.

fix(displacement): working state with temporary flag

480edb2

This change is for temporary purposes. DISPLACEMENT definition is must (otherwise displacementVS must be created) minimalOptions is disabled due to shaderpass definitions.

chore(mesh-displacement): updated mesh-displacement example

a9fa42c

fix(standart-material): displacement effect fully works now.

744516c

mvaligursky reviewed Sep 18, 2023

View reviewed changes

src/scene/materials/standard-material.js Outdated Show resolved Hide resolved

mvaligursky reviewed Sep 18, 2023

View reviewed changes

defektu added 3 commits September 18, 2023 17:52

feat(displacement): displacementFactor added

8289413

useDisplacement flag is setting from `displacementFactor !== 0`

chore(mesh-displacement.tsx): added displacementFactor variable

3738ca2

- Fixed useDisplacement flag - Minified mesh-displacement example

fix(types-fixup): no fixup needed

312bed2