Various visual effects #14508

GefullteTaubenbrust2 · 2024-03-31T18:14:38Z

This PR adds various effects intended to improve the visual experience. This includes changes to the shader code, as well as the engine source code, Lua API and settings.

List of Features and Changes

Tinted sunlight during dusk and dawn
Options for warmer artificial light colors
"soft" clouds look
Slight overall changes to the post processing shaders
Crude water reflections (sky and sun) and waves
Color grading
Vignette
Translucent foliage
Bump maps
Node "reflections" (just reflections of sunlight)
Adjusted fog color (more saturated where the fog is lighter)

Lua API additions

Cloud parameters: shadow field controls the color of the cloud's base
Lighting: shadow_tint controls shadow color
- default would be black { r = 0, g = 0, b = 0}
- ambient light from the sky could be emulated by a blue color like { r = 0, g = 50, b = 180 }

Settings

Name	Technical Name	Location	Type	Default
Tinted sunlight	tinted_sunlight	Graphics>Graphics Effects	bool	false
Artificial light color	artificial_light	Graphics>Graphics Effects	enum	"default"
Soft clouds	soft_clouds	Graphics>Clouds	bool	false
Liquid reflections	enable_water_reflections	Shaders>Waving Nodes	bool	false
Color grading	enable_color_grading	Shaders>Post Processing	bool	false
Vignette	enable_vignette	Shaders>Post Processing	bool	false
Translucent Foliage	enable_translucent_foliage	Shaders>Other Effects	bool	false
Bump maps	enable_bump_maps	Shaders>Other Effects	bool	false
Node reflections	enable_node_reflections	Shaders>Other Effects	bool	false

Screenshots

To do

This PR is Ready for Review.

How to test

For most features, simply change the game settings. Of course, the game you use should support dynamic shadows and volumetrics.
The added Lua properties can be used like so:

player:set_clouds({shadow = { r = 120, g = 160, b = 240 }})
player:set_lighting({shadow_tint = { r = 0, g = 50, b = 180 }})

Little fix

Most notably, (hopefully) fixed network protocols

Oops that part is redundant

Use the adjusted fog rendering from the nodes shader

oong819 · 2024-04-01T06:41:55Z

Do this work on OpenGL ES 2?

GefullteTaubenbrust2 · 2024-04-01T07:06:41Z

Do this work on OpenGL ES 2?

I haven't checked thoroughly but it should

Main fix is to the translucency effect, as it didn't take into account nodes that are tinted by the vertex color

This is hardly worthy of a commit but it bugged me

lhofhansl · 2024-04-01T16:35:46Z

Nice.

Note, we already have tinted sunlight (as long as volumetric light and shadows are enabled). Is this in addition?

GefullteTaubenbrust2 · 2024-04-01T17:18:40Z

Nice.

Note, we already have tinted sunlight (as long as volumetric light and shadows are enabled). Is this in addition?

We do? I mean the volumetric light is tinted, but the light illuminating nodes is always white/grey isn't it? I'm confused
Edit: #14091 implements basically this, except that it can be modified using Lua. That quite possibly should be the way to implement the effect instead. But still, as it stands I don't believe the game has this feature.

lhofhansl · 2024-04-01T20:00:36Z

the light illuminating nodes is always white/grey

Sorry, yes, misunderstood.
The effects are nice and I am glad someone is looking at this again.
I'll do a code review hopefully soon!

lhofhansl · 2024-04-02T01:12:42Z

builtin/settingtypes.txt

+#   -   default: Neutral, white light.
+#   -   medium: Slightly warmer color than default.
+#   -   warm: Warm (yellowish-orangy) light.
+artificial_light (Artificial light color) enum default default,medium,warm


This needs shaders on, let's move it in that section, and require shaders

lhofhansl · 2024-04-02T01:13:44Z

builtin/settingtypes.txt

+#   When enabled, liquid reflections are simulated.
+#
+#   Requires: shaders, enable_waving_water, enable_dynamic_shadows
+enable_water_reflections (Liquid reflections) bool false


Hmm... Should this be here, or also under shaders? I feel like this into the shader section.

lhofhansl · 2024-04-02T01:20:19Z

client/shaders/nodes_shader/opengl_fragment.glsl

+}
+
+float wave_noise(vec3 p, float off) {
+	return snoise(p + vec3(0.0, 0.0, off)) * 0.4 + snoise(2.0 * p + vec3(0.0, off, off)) * 0.1 + snoise(3.0 * p + vec3(0.0, off, off)) * 0.075 + snoise(4.0 * p + vec3(-off, off, 0.0)) * 0.05;


Is there a simpler way? We call snoise 4 times here, then later we call wave_noise 3 times.
In the vertex shader we're only calling snoise once to get the wave pattern.

Good point, the 4 calls in wave_noise i know seem a bit much and it’s one more than would technically be needed for regular fractal noise - however, especially with these reflections, any artifacts in the noise show very clearly and these 4 just look significantly better.
The fact wave_noise has to be called three times is because we need normal information in the fragment shader, which we don’t need in the vertex shader. I did implement it for the vertex shader in my „shader pack“ for 5.8.0 and that does almost definitely improve performance, but also shows very distinct artifacts.
With many functions you can get their gradient as an analytical function, but I‘m not sure how that would work with noise like this. I could look into using analytical functions like sine waves instead of noise, which would reduce the calls to effectively 2 calls to the „noise“ function, but it would introduce periodicity and I‘m not sure about the performance improvement.
If anyone else has ideas as to how to optimize this, let me know.

Edit: Actually, looking at the noise function again… the smooth noise samples a not so smooth function several times already, so in principle the normal information is there… I might be able to work something out.

lhofhansl · 2024-04-02T01:22:03Z

client/shaders/nodes_shader/opengl_vertex.glsl

@@ -262,14 +265,14 @@ void main(void)

 		if (f_timeofday < 0.2) {
 			adj_shadow_strength = f_shadow_strength * 0.5 *
-				(1.0 - mtsmoothstep(0.18, 0.2, f_timeofday));


Can you explain the magic number changes?

Simple reason is the old values don't look very good imo, especially during sunsets the shadows fade far too quickly. This is basically a function for shadow strength over time, so I've plotted that here. Blue is the original, red is the changed one.

It should be pretty apparent why sunsets would be especially washed out. The old function is also very strangely asymmetric, which is odd if you think about the physical implications of that, and the new one isn't. I suppose it makes sense that the shadows change faster when the moon rises and sets compared to the sun, so that might make sense to change back to be fair.

lhofhansl · 2024-04-02T01:22:49Z

client/shaders/second_stage/opengl_fragment.glsl

 	const float gamma = 1.6;
 	const float exposureBias = 5.5;
 	color.rgb = uncharted2Tonemap(exposureBias * color.rgb);
 	// Precalculated white_scale from
 	//vec3 whiteScale = 1.0 / uncharted2Tonemap(vec3(W));
 	vec3 whiteScale = vec3(1.036015346);
 	color.rgb *= whiteScale;
-	return vec4(pow(color.rgb, vec3(1.0 / gamma)), color.a);


Why do you no longer need the light-space transformations?

They looked very strange to me and created very harsh contrast and oversaturated colors, which as far as I know is exactly what the gamma correction was there to prevent. I’ll post a comparison later.

So actually this has to do with #14109, I'll have to look deeper into that...

I promised screenshots, here they are.
The PR as is (errors aside):

With gamma correction before color grading:

I suppose the solution here might be to return everything to how it was and move the color grading step into the tone mapping function?

sfan5 · 2024-04-11T19:24:01Z

Also, was the uniform removed because it wasn't used? If so it might make sense to reintroduce it...

Yes, simply for that reason.

GefullteTaubenbrust2 · 2024-04-12T20:01:36Z

Yes, simply for that reason.

I might have some time this weekend, I suppose I'll just add it back.

reintroduces eyePosition uniform

Infernia829 · 2024-04-17T00:07:37Z

Water reflections don't work on windows, neither does volumetric lights, bloom, or just about any other shader besides the shadows.

Infernia829 · 2024-04-17T01:26:15Z

Using OpenGL3 render engine on the debug build results in

2024-04-16 20:24:10: [Main]: Automatically selecting world at [C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\worlds\Test]
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open vertex shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\Solid.vsh
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open pixel shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\Solid.fsh
2024-04-16 20:24:11: ERROR[Main]: Irrlicht: GLSL shader program failed to link
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open vertex shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\Solid.vsh
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open pixel shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\TransparentAlphaChannel.fsh
2024-04-16 20:24:11: ERROR[Main]: Irrlicht: GLSL shader program failed to link
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open vertex shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\Solid.vsh
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open pixel shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\TransparentAlphaChannelRef.fsh
2024-04-16 20:24:11: ERROR[Main]: Irrlicht: GLSL shader program failed to link
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open vertex shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\Solid.vsh
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open pixel shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\TransparentVertexAlpha.fsh
2024-04-16 20:24:11: ERROR[Main]: Irrlicht: GLSL shader program failed to link
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open vertex shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\Solid.vsh
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Could not open pixel shader program file: C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\OneTextureBlend.fsh
2024-04-16 20:24:11: ERROR[Main]: Irrlicht: GLSL shader program failed to link
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Warning: Missing shader files needed to simulate fixed function materials:
C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\Renderer2D.vsh
Shaderpath can be changed in SIrrCreationParamters::OGLES2ShaderPath
2024-04-16 20:24:11: ERROR[Main]: Irrlicht: GLSL shader program failed to link
2024-04-16 20:24:11: WARNING[Main]: Irrlicht: Warning: Missing shader files needed to simulate fixed function materials:
C:\Users\Admin\Desktop\minetest-Eyecandy-Effects\bin\MinSizeRel\..\..\client\shaders\Irrlicht\Renderer2D.vsh
Shaderpath can be changed in SIrrCreationParamters::OGLES2ShaderPath
2024-04-16 20:24:11: ERROR[Main]: Irrlicht: GLSL shader program failed to link

lhofhansl · 2024-04-17T04:18:34Z

@Infernia829 You verified it works without this applied? You built the same way?

Infernia829 · 2024-04-17T04:21:27Z

@Infernia829 You verified it works without this applied? You built the same way?

Baseline MT works, yes. However none of the listed shaders/visual effects work in any way, shape, or form. I tried over 20 different combinations of settings, cmake settings, and more, no matter what I did I was unable to get your shaders working.

What I sent above was the only error messages I was able to get, otherwise the shaders just didn't load, it looked the same as it did with them disabled.

oong819 · 2024-04-17T06:10:12Z

Yes, simply for that reason.

I might have some time this weekend, I suppose I'll just add it back.

The nodes_shader error are fixed now =D
(still suffer from crash due to master branch bug)

lhofhansl · 2024-04-18T16:04:33Z

Please make a clear statement :)

Has MT build from master (without any other changes) broken shaders on Windows or not?
Is MT build from master with this PR applied broken or not?

Finally, I have a local change that brings back the needed parts removed in #14516 combined with this, and that is all working fine (on Linux)

Infernia829 · 2024-04-18T16:07:18Z

Please make a clear statement :)

Has MT build from master (without any other changes) broken shaders on Windows or not?

Is MT build from master with this PR applied broken or not?

Finally, I have a local change that brings back the needed parts removed in #14516 combined with this, and that is all working fine (on Linux)

MT build from master works fine, but again it is lacking all the features that don't work anyway.

MT build from master with this PR applied is broken.

MirceaKitsune · 2024-04-21T15:51:10Z

This sounds absolutely beautiful and I can't wait to see the PR approved if there are no issues blocking it! Water reflections are a long awaited last piece of the puzzle ever since the other effects were finally added: All that will be left is support for specular and normal maps in some form, it can be done with our lighting system but will likely require an original approach to get a 3D reflection volume right.

Realistic sun / moon colors that get yellow then red near the horizon are another thing I've always hoped to see changed, thank you for addressing that! The only huge limitation left as far as lights go is that in nearly 20 years of Minetest, no attempt has been made to support colored light sources so you can for example have a green lamp: I'm hoping that someday this final blocker can be lifted which will open the doors to a lot of new possibilities.

grorp · 2024-04-21T16:50:19Z

Does this add actual water reflections like in GreenXenith's bfs_ssr branch though? From what I can see, it's "just" some fancy lighting.

lhofhansl · 2024-04-22T16:52:09Z

Does this add actual water reflections like in GreenXenith's bfs_ssr branch

Nope it just adds simple sun reflection.
If I may... Let's not do the "Minetest Thing" again, where we wait for some perfect solution somewhere out there that never materializes.

grorp · 2024-04-22T17:44:50Z

Let's not do the "Minetest Thing" again, where we wait for some perfect solution somewhere out there that never materializes.

I did not suggest that. I also prefer imperfect solutions over no solutions. My goal was to correct possibly false expectations expressed in #14508 (comment) and #14508 (comment).

lhofhansl · 2024-04-23T00:39:49Z

@grorp I also like real reflection. :) Did ever get @GreenXenith's ssr branch to work? I tried half-heartedly and all I get is blank screen as soon as the game starts.

GreenXenith · 2024-04-23T00:43:11Z

If I may... Let's not do the "Minetest Thing" again, where we wait for some perfect solution somewhere out there that never materializes.

I agree, I would rather migrate my SSR branch (to replace the simple reflections probably) after this is merged.

lhofhansl · 2024-04-24T04:14:58Z

I suggest that we could include the following effects with relatively little discussion with just client settings:

Tinted sunlight during dusk and dawn
"soft" clouds look
Crude water reflections (sky and sun) and waves
Node "reflections" (just reflections of sunlight)
Adjusted fog color (more saturated where the fog is lighter)

I don't know what the following do:

Slight overall changes to the post processing shaders
Color grading
Translucent foliage

The following perhaps need some discussion:

Options for warmer artificial light colors
Vignette
Bump maps

I will say that overall this quite increases the immersion. The colors are better, and the simple sun reflection makes it look like real water.

tigercoding56 · 2024-04-30T00:08:01Z

the sand is a bit green

grorp · 2024-04-30T11:58:01Z

The split proposed by lhofhansl seems sensible to me.

Re "vignette": A vignette can be implemented as a mod or even a client mod using the HUD API. Some games already do this, e.g. Repixture. Therefore, I think having it as a client-side setting is unnecessary and may result in multiple vignettes being applied on top of each other.

Infernia829 · 2024-05-01T04:40:37Z

The split proposed by lhofhansl seems sensible to me.

Re "vignette": A vignette can be implemented as a mod or even a client mod using the HUD API. Some games already do this, e.g. Repixture. Therefore, I think having it as a client-side setting is unnecessary and may result in multiple vignettes being applied on top of each other.

That is kinda dumb, a client side vignette would be so much better then some sort of hack hud api, why would you purposfully remove it when its already been made

grorp · 2024-05-01T08:47:16Z

That is kinda dumb, a client side vignette would be so much better then some sort of hack hud api, why would you purposfully remove it when its already been made

So you're saying that a shader vignette is better than a HUD API vignette? Why?

The benefit of vignettes via the HUD API is that they can be customized (e.g. their strength) by games/mods depending on player status effects, and that this works without any explicit support from Minetest. Surely we could also make a shader vignette customizable, but that would need another API.

Re "a client side vignette would be so much better": Strictly speaking, a vignette implemented via a client mod is also client-side.

GreenXenith · 2024-05-01T09:38:28Z

Re "a client side vignette would be so much better": Strictly speaking, a vignette implemented via a client mod is also client-side.

Strictly speaking, once you send the vignette HUD element from a regular mod it is also client-side.

GefullteTaubenbrust2 · 2024-05-01T14:18:23Z

The split proposed by lhofhansl seems sensible to me.

Re "vignette": A vignette can be implemented as a mod or even a client mod using the HUD API. Some games already do this, e.g. Repixture. Therefore, I think having it as a client-side setting is unnecessary and may result in multiple vignettes being applied on top of each other.

True that, I just moved that one over from my shaderpack without realizing there is a vignette effect already. Although I think the hud based vignette suffers from banding, so I may at some point look into that instead.

GefullteTaubenbrust2 · 2024-05-01T14:23:01Z

the sand is a bit green

How did that happen? Any specific things you did that caused this, or is the sand just always green?
It looks though as though the tinted sunlight causes this and I probably won't implement it for the split PRs anyway...

Zughy · 2024-05-01T14:40:02Z

Although I think the hud based vignette suffers from banding

OT but: same with formspecs. Just put a perfect shade as a formspec image and you'll see the banding effect. I discovered this the other day (MT 5.8)

AbduSharif · 2024-05-02T03:24:53Z

What about user choice? In lots of cases, HUD vignette looks way worse than the post process alternative.

There also games that don't add the effect, what if the user wants to enable it while playing them?

GefullteTaubenbrust2 · 2024-05-02T19:06:30Z

What about user choice? In lots of cases, HUD vignette looks way worse than the post process alternative.

There also games that don't add the effect, what if the user wants to enable it while playing them?

There's actually another issue, which is the HUD is applied after everything is rendered out, but color grading for example should actually apply after the vignette... Not sure how to solve this, I suppose I could also add lua options for a vignette as part of the second stage shaders? That would also allow the user to still add mods that add the vignette. None of this will be in the first PR, but it's still important to think about...

GefullteTaubenbrust2 added 6 commits March 29, 2024 11:17

Add everything I've done so far

6767ae8

A few extra settings and fixes

f0cd910

Update mapblock_mesh.cpp

d14111e

Little fix

Various changes

1ba97bf

Most notably, (hopefully) fixed network protocols

Update clientpackethandler.cpp

037c707

Oops that part is redundant

Update opengl_fragment.glsl

c7a595e

Use the adjusted fog rendering from the nodes shader

GefullteTaubenbrust2 added 3 commits April 1, 2024 14:26

Various small fixes

3ba746c

Update opengl_fragment.glsl

e7f1e86

Main fix is to the translucency effect, as it didn't take into account nodes that are tinted by the vertex color

Update opengl_fragment.glsl

8776072

This is hardly worthy of a commit but it bugged me

SmallJoker added @ Script API @ Client / Audiovisuals Feature ✨ PRs that add or enhance a feature Shaders and removed @ Script API @ Client / Audiovisuals Feature ✨ PRs that add or enhance a feature labels Apr 1, 2024

lhofhansl self-requested a review April 1, 2024 20:00

lhofhansl reviewed Apr 2, 2024

View reviewed changes

Fix nodes_shader error

0f125d9

reintroduces eyePosition uniform

GefullteTaubenbrust2 marked this pull request as draft May 1, 2024 18:18

GefullteTaubenbrust2 mentioned this pull request May 4, 2024

Visual Effects Vol. 1 #14610

Open

GefullteTaubenbrust2 closed this May 4, 2024

Various visual effects #14508

Various visual effects #14508

Conversation

GefullteTaubenbrust2 commented Mar 31, 2024 • edited

Lua API additions

Settings

To do

How to test

oong819 commented Apr 1, 2024

GefullteTaubenbrust2 commented Apr 1, 2024

lhofhansl commented Apr 1, 2024

GefullteTaubenbrust2 commented Apr 1, 2024 • edited

lhofhansl commented Apr 1, 2024

lhofhansl Apr 2, 2024

Choose a reason for hiding this comment

lhofhansl Apr 2, 2024

Choose a reason for hiding this comment

lhofhansl Apr 2, 2024 • edited

Choose a reason for hiding this comment

GefullteTaubenbrust2 Apr 2, 2024 • edited

Choose a reason for hiding this comment

lhofhansl Apr 2, 2024

Choose a reason for hiding this comment

GefullteTaubenbrust2 May 4, 2024

Choose a reason for hiding this comment

lhofhansl Apr 2, 2024

Choose a reason for hiding this comment

GefullteTaubenbrust2 Apr 2, 2024

Choose a reason for hiding this comment

GefullteTaubenbrust2 Apr 7, 2024

Choose a reason for hiding this comment

GefullteTaubenbrust2 Apr 11, 2024 • edited

Choose a reason for hiding this comment

sfan5 commented Apr 11, 2024

GefullteTaubenbrust2 commented Apr 12, 2024

Infernia829 commented Apr 17, 2024 • edited

Infernia829 commented Apr 17, 2024 • edited by SmallJoker

lhofhansl commented Apr 17, 2024

Infernia829 commented Apr 17, 2024

oong819 commented Apr 17, 2024

lhofhansl commented Apr 18, 2024

Infernia829 commented Apr 18, 2024

MirceaKitsune commented Apr 21, 2024

grorp commented Apr 21, 2024

lhofhansl commented Apr 22, 2024

grorp commented Apr 22, 2024 • edited

lhofhansl commented Apr 23, 2024

GreenXenith commented Apr 23, 2024

lhofhansl commented Apr 24, 2024

tigercoding56 commented Apr 30, 2024 • edited

grorp commented Apr 30, 2024

Infernia829 commented May 1, 2024

grorp commented May 1, 2024

GreenXenith commented May 1, 2024

GefullteTaubenbrust2 commented May 1, 2024

GefullteTaubenbrust2 commented May 1, 2024 • edited

Zughy commented May 1, 2024 • edited

AbduSharif commented May 2, 2024 • edited

GefullteTaubenbrust2 commented May 2, 2024

GefullteTaubenbrust2 commented Mar 31, 2024 •

edited

GefullteTaubenbrust2 commented Apr 1, 2024 •

edited

lhofhansl Apr 2, 2024 •

edited

GefullteTaubenbrust2 Apr 2, 2024 •

edited

GefullteTaubenbrust2 Apr 11, 2024 •

edited

Infernia829 commented Apr 17, 2024 •

edited

Infernia829 commented Apr 17, 2024 •

edited by SmallJoker

grorp commented Apr 22, 2024 •

edited

tigercoding56 commented Apr 30, 2024 •

edited

GefullteTaubenbrust2 commented May 1, 2024 •

edited

Zughy commented May 1, 2024 •

edited

AbduSharif commented May 2, 2024 •

edited