Feature Request: FidelityFX Super Resolution

[Zi7ar21]

Currently it appears that the image renders at a lower-than-native resolution. The image upscaling could be improved by switching to FidelityFX Super Resolution. FSR has already been been done on Shadertoy! https://www.shadertoy.com/view/stXSWB It shouldn't be difficult to add it as an option.

[erichlof]

Hello @Zi7ar21

Thank you for the heads-up about this technique and the Shadertoy link. As with everything else currently in this project, I first have to fully understand the overall problem that the algo is trying to solve, and, in general non-math terms, how it plans on solving it. This often means going back to the original research paper where these algos/techniques oiginated from. Only then can I start adding an algo (which admittedly sometimes involves copying and pasting source code) to the project.

At first glance at the Shadertoy demo, I do have some initial concerns: in the comments the author admits that he himself doesn't fully understand the algo that he has ported to Shadertoy. Also, looking at the BufferA tab, it looks like there are quite a number of texture taps required. Normally by itself, this would be ok, but I do already have 25 taps per frame per pixel to be able to do diffuse denoising properly.

The reason that the three.js-PathTracing-Renderer is set to 0.5 PixelRatio(half resolution) is purely for real time performance reasons. Each demo has a dedicated .js setup file with matching name, and if you change the pixel ratio back to the default 1.0 (full resolution), it will slow down the frame rate, but will give perfect results like you would expect from Maya, Blender Cycles, VRay, Arnold, and the like. This is due in part to subpixel SuperSampling (that is already in place), and Peter Shirley's (author of Ray Tracing in One Weekend series) brilliant 'tent filter', which takes all those precious sub-pixel samples and combines them in the most effective way that I have ever seen in any rendering package.

So we do already have algos in place for sub-pixel SuperSampling and filters and denoising, etc., - whether or not I can use this new Fidelity system and make it play nice with the systems that are already in place, I will have to investigate further.

But at any rate, thanks again for the suggestion- I'm always looking for new and effective approaches to real time path tracing! :-)

[Zi7ar21]

You can find all the details about FSR here: https://gpuopen.com/fidelityfx-superresolution

FSR is for upscaling lower-resolution frames to native resolution. This is only useful for real-time rendering. If you are rendering a full quality image, you would not want to use this. Therefore, you might want to make it optional. The goal is to match the native resolution frame as closely as possible.

Basically FSR is AMD's Open-Source competitor to NVIDIA's proprietary DLSS. FSR doesn't use deep learning, and so it's super compatible. Like you said, the path-tracer currently renders at 0.5x native resolution to increase performance. The problem is that currently it looks like it's being upscaled with bilinear which is pretty bad.

It's just postprocessing, and so it shouldn't break anything. They say to feed it the frame before tone mapping and stuff. You can check out the source code and see for yourself how it works: https://github.com/GPUOpen-Effects/FidelityFX-FSR/tree/master/ffx-fsr

[erichlof]

Ah ok, I think I understand what it's trying to do now. Thank you for the clarification! Yes this would be a nice addition to the renderer indeed. As previously mentioned, I feel the need to understand at least some of the algos and code involved with all the various pieces of this project, in order to feel confident and worthy enough to copy and paste sections of someone else's code into the renderer. Looking at the source code you linked to, it is pretty complex, but not impossible to wade through eventually. It might take a while, but I will investigate and give it a go.

Btw, I enjoyed taking a peek at your path marcher in C project ! The code is quite well organized and well-commented when it matters. I appreciate good, readable code like this. :-) My programming journey including a multi-year tour in OpenGL 1 and C programming back in the late 1990's, so the C language holds a special place in my heart! (Although now that I have dived into the world of webGL programming where JavaScript rules the land, I'm probably too rusty to do anything meaningful in C, ha ha!). Best of luck to you in your path tracing/marching efforts!

-Erich

[Zi7ar21]

I am excited to see if it increases performance, and if it performs well on mobile. Also thanks, I need to go back into my C Path Tracer and rewrite a whole lot of it, as there is a lot of things that can be done better now, along with using a better image format.

[michaelsakharov]

https://www.shadertoy.com/view/ftlSDX Looks like a fork of that Shadertoy that's much simpler in terms of Code and faster while producing practically identical results.