Tone mapping ruins the constant background color

[LiangliangNan]

Hi, thanks for the great project!

In the shader code in tonemap.glsl, tonemap is applied to all pixels, which is not correct if a constant background color is desired. See the effects below:

Without tone mapping:

With tone mapping:

I think whether it is background can be recorded as the w component in pathTraceTexture. Maybe there are other better ideas?

[knightcrawler25]

Hi,

The background color actually acts as a constant light source (which I was using as a furnace test) so perhaps it should be named differently to avoid confusion.

I think the standard way of doing this (that I know of) is to record transparency in a channel, as you mentioned, and then composite the rendered image over a white background or any other image.

Something like this:

Right now, the renderer doesn't track alpha/transparency and it would require some modification to get it to work.

[LiangliangNan]

Thanks for the quick reply.
Now I see. But it is indeed confusing and would be good to make it behave the same as other software.

[knightcrawler25]

I added support for background color and transparency to the dev branch. There might be some bugs and transparency doesn't work with the denoiser as of now, but you should be able to set a background color or save out an image and composite it over another image in GIMP or photoshop.

White Background:

Transparency in viewer:

Composited:

[LiangliangNan]

Thanks for the efforts. Very much appreciated!
The white color works now.

Below I summary the issues I have identified in the test and in reading the code.

1. Low-reso rendering when the mouse moves.
   I found the "cornell_box.scene" is not properly rendered. It seems the low-resolution pass is not correctly rendered, or it may be due to the composition or that the previous content was not cleared. Here is a short recording:
   https://user-images.githubusercontent.com/15526536/147749593-40d0f1f4-cdf3-427b-a555-5aee230be5ed.mp4

2. Switching scenes.
   When switching to a new scene, the original contents seem not cleared. This is really weird because the renderer has been rebuilt. Here is a snapshot (which can be reproduced by firstly zooming out the cornell_box scene and then switching to cornell_box2)
   

3. Other issues in the code.

• in uniforms.glsl: uniform useEnvMap is never used and can be removed
• in anyhit.glsl: LIGHTS -> OPT_LIGHTS
• in pathtrace.glsl: OPENGL_NORMALMAP -> OTP_OPENGL_NORMALMAP
• in Scene.cpp: missing "delete texture;" if texture creation failed, in function "int Scene::AddTexture(const std::string& filename)";
• in Scene.cpp: "resizedTex" in function "void Scene::ProcessScene()" is allocated using "new[]" and thus should be freed using "delete []".

These things so far, hope it helps and looking forward to any update. Happy New Year!

[knightcrawler25]

Thanks a lot for pointing these out. I have pushed a commit to the repo that has the changes for 3.

For 1 and 2 however, I'm unable to reproduce them on windows. Are you running this on Linux?

[LiangliangNan]

Oh, forgot to mention that.
I am using an apple laptop:

• maxOS 11.6
• CLang 13.0.0
• GPU: Intel(R) Iris(TM) Plus
• OpenGL version: 4.1 INTEL-16.5.2
• GLSL version: 4.10

To understand what goes wrong, I took snapshots for the four FBOs (every time when new contents have been written into).
https://www.dropbox.com/s/td9bfe3ijm7ou39/fbo_snapshots.zip?dl=0
I was not able to get a clue, maybe easier for you.

[knightcrawler25]

I've actually never tested the code on a mac as I don't have access to one. Unfortunately, I don't think it would be possible for me to debug this as I don't see the same issues on windows or linux. I've also tried running the code on some older hardware as well and can't replicate it.

The first issue looks rather odd, sort of like uninitialized data is being read by the shader. Do you see this issue across all scenes? Perhaps you can strip out much of the logic in pathtrace.glsl and just check if ClosestHit() is returning a proper hit

[LiangliangNan]

Now the 2nd issue has been fixed, with your latest commit and the following change:

• in TiledRenderer::Render(), after drawing into the low reso FBO, the scene->dirty should be changed to false, i.e.,
  scene->instancesModified = false;
  becomes
  scene->instancesModified = false;
scene->dirty = false;

The first issue happens only to the "cornell_box.scene", which is really weird. I couldn't find any potential cause in the shader code.

[knightcrawler25]

Thanks, I added that change in.

It's odd that you see an issue in that scene but not in cornell_box2.scene. The second scene uses the same objects (Difference being cbox_smallbox.obj and slight difference in light emission, material colors and path depth).

[LiangliangNan]

Indeed!

Below are 3 consecutive snapshots of the trace FBO (the lower left part of the "cornell_box. scene"). And we can see that many hits are missed:

[knightcrawler25]

Is cornell_box.scene the first scene that gets loaded when you run the code? What if you switch to a different scene and then switch back to cornell_box.scene, does the issue still remain?

[LiangliangNan]

Then it is gone. But why?

[knightcrawler25]

I'm guessing it could be one of these uniforms that is not being sent to the shader the first time around and when you reload the scene things might be getting initialized properly.

GLSL-PathTracer/src/core/TiledRenderer.cpp

Lines 520 to 560 in a6526c8

	pathTraceShader->Use();
	shaderObject = pathTraceShader->getObject();
	glUniform3f(glGetUniformLocation(shaderObject, "camera.position"), scene->camera->position.x, scene->camera->position.y, scene->camera->position.z);
	glUniform3f(glGetUniformLocation(shaderObject, "camera.right"), scene->camera->right.x, scene->camera->right.y, scene->camera->right.z);
	glUniform3f(glGetUniformLocation(shaderObject, "camera.up"), scene->camera->up.x, scene->camera->up.y, scene->camera->up.z);
	glUniform3f(glGetUniformLocation(shaderObject, "camera.forward"), scene->camera->forward.x, scene->camera->forward.y, scene->camera->forward.z);
	glUniform1f(glGetUniformLocation(shaderObject, "camera.fov"), scene->camera->fov);
	glUniform1f(glGetUniformLocation(shaderObject, "camera.focalDist"), scene->camera->focalDist);
	glUniform1f(glGetUniformLocation(shaderObject, "camera.aperture"), scene->camera->aperture);
	glUniform1i(glGetUniformLocation(shaderObject, "useEnvMap"), scene->hdrData == nullptr ? false : scene->renderOptions.useEnvMap);
	glUniform1f(glGetUniformLocation(shaderObject, "hdrMultiplier"), scene->renderOptions.hdrMultiplier);
	glUniform1i(glGetUniformLocation(shaderObject, "maxDepth"), scene->renderOptions.maxDepth);
	glUniform2f(glGetUniformLocation(shaderObject, "tileOffset"), (float)tile.x * invNumTiles.x, (float)tile.y * invNumTiles.y);
	glUniform3f(glGetUniformLocation(shaderObject, "uniformLightCol"), scene->renderOptions.uniformLightCol.x, scene->renderOptions.uniformLightCol.y, scene->renderOptions.uniformLightCol.z);
	glUniform1i(glGetUniformLocation(shaderObject, "frameNum"), frameCounter);
	pathTraceShader->StopUsing();
	pathTraceShaderLowRes->Use();
	shaderObject = pathTraceShaderLowRes->getObject();
	glUniform3f(glGetUniformLocation(shaderObject, "camera.position"), scene->camera->position.x, scene->camera->position.y, scene->camera->position.z);
	glUniform3f(glGetUniformLocation(shaderObject, "camera.right"), scene->camera->right.x, scene->camera->right.y, scene->camera->right.z);
	glUniform3f(glGetUniformLocation(shaderObject, "camera.up"), scene->camera->up.x, scene->camera->up.y, scene->camera->up.z);
	glUniform3f(glGetUniformLocation(shaderObject, "camera.forward"), scene->camera->forward.x, scene->camera->forward.y, scene->camera->forward.z);
	glUniform1f(glGetUniformLocation(shaderObject, "camera.fov"), scene->camera->fov);
	glUniform1f(glGetUniformLocation(shaderObject, "camera.focalDist"), scene->camera->focalDist);
	glUniform1f(glGetUniformLocation(shaderObject, "camera.aperture"), scene->camera->aperture);
	glUniform1i(glGetUniformLocation(shaderObject, "useEnvMap"), scene->hdrData == nullptr ? false : scene->renderOptions.useEnvMap);
	glUniform1f(glGetUniformLocation(shaderObject, "hdrMultiplier"), scene->renderOptions.hdrMultiplier);
	glUniform1i(glGetUniformLocation(shaderObject, "maxDepth"), scene->dirty ? 2: scene->renderOptions.maxDepth);
	glUniform3f(glGetUniformLocation(shaderObject, "camera.position"), scene->camera->position.x, scene->camera->position.y, scene->camera->position.z);
	glUniform3f(glGetUniformLocation(shaderObject, "uniformLightCol"), scene->renderOptions.uniformLightCol.x, scene->renderOptions.uniformLightCol.y, scene->renderOptions.uniformLightCol.z);
	pathTraceShaderLowRes->StopUsing();
	tonemapShader->Use();
	shaderObject = tonemapShader->getObject();
	glUniform1f(glGetUniformLocation(shaderObject, "invSampleCounter"), 1.0f / (sampleCounter));
	glUniform1i(glGetUniformLocation(shaderObject, "enableTonemap"), scene->renderOptions.enableTonemap);
	glUniform1i(glGetUniformLocation(shaderObject, "useAces"), scene->renderOptions.useAces);
	glUniform1i(glGetUniformLocation(shaderObject, "simpleAcesFit"), scene->renderOptions.simpleAcesFit);
	glUniform3f(glGetUniformLocation(shaderObject, "backgroundCol"), scene->renderOptions.backgroundCol.x, scene->renderOptions.backgroundCol.y, scene->renderOptions.backgroundCol.z);
	tonemapShader->StopUsing();

[LiangliangNan]

I've checked those uniforms and the values are the same.

Look at the result. There are uniform regions with straight boundaries. I guess the issue might be due to the transformation of each mesh.

[knightcrawler25]

Not really sure where the issue is but could you try the latest code in the dev branch and see if that fixes anything? I've refactored some of the code.

[LiangliangNan]

Thanks for the effort. Just tried and it is still the same:

Two files that do not compile:

• Loader.cpp, line 483:
  printf("Unable to load gltf %s\n", filename);
  should be
  printf("Unable to load gltf %s\n", filename.c_str());
• EnvironmentMap.cpp: missing "#include "

[tigrazone]

One more issue.
When I wrote in command line
PathTracer -s c:\temp\glTF-Sample-Models\2.0\WaterBottle\glTF-Binary\WaterBottle.glb
program fails.
in Main.cpp from line 524 must be rewriten

if (!sceneFile.empty())
    {
        scene = new Scene();

        GetEnvMaps();
        LoadScene(sceneFile);
    }
    else
    {
        GetSceneFiles();
        GetEnvMaps();
        LoadScene(sceneFiles[sampleSceneIdx]);
    }

When is no EnvMap loaded program fails also.
TiledRenderer.cpp lines 251-252 and 274-275 must be rewriten like this

       glUniform2f(glGetUniformLocation(shaderObject, "envMapRes"), scene->envMap == nullptr ? 0.0: (float)scene->envMap->width, scene->envMap == nullptr ? 0.0 :(float)scene->envMap->height);
        glUniform1f(glGetUniformLocation(shaderObject, "envMapTotalSum"), scene->envMap == nullptr ? 0.0 : scene->envMap->totalSum);

[tigrazone]

I check tests from https://github.com/KhronosGroup/glTF-Sample-Models

glTF-Sample-Models\2.0\SpecularTest\ not passed. Specular parameter not changed

glTF-Sample-Models\2.0\SpecGlossVsMetalRough\ not passed. Right bottle not texture mapped like left bottle

right result by https://github.com/SaschaWillems/Vulkan-glTF-PBR

glTF-Sample-Models\2.0\AttenuationTest

where is blue?

glTF-Sample-Models\2.0\TransmissionRoughnessTest

where is blue?

[knightcrawler25]

@tigrazone : GLTF material support is not complete yet. Only metallic roughness workflow is used and transmissionFactor from the extensions:

GLSL-PathTracer/src/loaders/GLTFLoader.cpp

Lines 245 to 279 in ddbba9a

	// Albedo
	material.baseColor = Vec3((float)pbr.baseColorFactor[0], (float)pbr.baseColorFactor[1], (float)pbr.baseColorFactor[2]);
	if (pbr.baseColorTexture.index > -1)
	material.baseColorTexId = pbr.baseColorTexture.index + sceneTexIdx;
	// Opacity
	material.opacity = (float)pbr.baseColorFactor[3];
	// Alpha
	material.alphaCutoff = static_cast<float>(gltfMaterial.alphaCutoff);
	if (strcmp(gltfMaterial.alphaMode.c_str(), "OPAQUE") == 0) material.alphaMode = AlphaMode::Opaque;
	else if (strcmp(gltfMaterial.alphaMode.c_str(), "BLEND") == 0) material.alphaMode = AlphaMode::Blend;
	else if (strcmp(gltfMaterial.alphaMode.c_str(), "MASK") == 0) material.alphaMode = AlphaMode::Mask;
	// Roughness and Metallic
	material.roughness = sqrtf((float)pbr.roughnessFactor); // Repo's disney material doesn't use squared roughness
	material.metallic = (float)pbr.metallicFactor;
	if (pbr.metallicRoughnessTexture.index > -1)
	material.metallicRoughnessTexID = pbr.metallicRoughnessTexture.index + sceneTexIdx;
	// Normal Map
	material.normalmapTexID = gltfMaterial.normalTexture.index + sceneTexIdx;
	// Emission
	material.emission = Vec3((float)gltfMaterial.emissiveFactor[0], (float)gltfMaterial.emissiveFactor[1], (float)gltfMaterial.emissiveFactor[2]);
	if (gltfMaterial.emissiveTexture.index > -1)
	material.emissionmapTexID = gltfMaterial.emissiveTexture.index + sceneTexIdx;
	// KHR_materials_transmission
	if (gltfMaterial.extensions.find("KHR_materials_transmission") != gltfMaterial.extensions.end())
	{
	const auto& ext = gltfMaterial.extensions.find("KHR_materials_transmission")->second;
	if (ext.Has("transmissionFactor"))
	material.specTrans = (float)(ext.Get("transmissionFactor").Get<double>());
	}

For the last screenshot, you'll have to increase number of bounces to get the blue color

[tigrazone]

Agreed with blue in scenes - increase number of bounces helps to render blue colored boxes.
But other is not solved yet

[knightcrawler25]

@LiangliangNan : Could you also try the debug branch? It has just enough code to render cornell_box.scene and would be easier to check.

[LiangliangNan]

Now there are fewer "black" things, but still similar. See the recording below:

Untitled.mp4

Now it seems the top and bottom planes are semi-transparent (but still partially).

[knightcrawler25]

That's good news lol. I added a couple of commits each progressively removing a feature until its down to just bvh traversal and triangle intersection. Let me know if any of the commits ends up working, that way I'll know where to look.

[LiangliangNan]

Thanks for the efforts 👍
"Normals" is correct and "Reflect" has the problem. See below:

Disable NEE

Remove materialTex

Reflect

Normals

[knightcrawler25]

(Edit: Ignore this, its just an effect of clamping negative values)

Normals don't look right. Was the screenshot from 9a09313?

This is what I see from that commit:

[LiangliangNan]

Yes. it is from 9a09313.
The figure captions correspond to your comments

[knightcrawler25]

Do you see the same image with the last two commits where mesh transformations have been removed?

[LiangliangNan]

Here are the results from the last three commits:

hit dist:

remove transformation from mesh

Remove transformation from BVH