[1.20]: Implement OIT Rendering.

[marchermans]

Order Independent Transparent Rendering

OIT Rendering, for short OIT, is a rendering mechanic which allows rendering several transparent objects without sorting their order first.

Existing mechanic

If we take a look at the existing rendering mechanics within Minecraft, then Mojang uses two major systems to handle layering transparent surfaces:

1. Make sure to render the furthest away vertices first during transparent and semi-transparent surface rendering.
2. Render transparent geometry after solid ones.
   This works virtually all of the time. But not in all:
   For example:

Expand to see examples of particles

As you can see, the smoke particles are semi-transparent, but the effect cloud particles in the background are not visible.

Expand to see examples of shulker bullet halos

As you can see, the entity renderer for the shulker and the clouds in the background are not rendered properly.

Expand to see examples of ice block interactions

Visible here is the main example of the rendering engine breaking transparency since the water is not visible behind the ice.

Impact

These are corner cases in vanilla, and the priority of fixing them is lower. However, these issues happen regularly in a modded environment when rendering additional components into the world. Examples of cases where this happens are custom particles, schematic previews, fake worlds, etc.

New mechanic

This new feature implements weighted order independent rendering based on the paper published by: McGuire and Bavoil¹. The idea here is to capture all rendering calls which involve transparent subsections that we are interested in and draw them to a separate system.
This system emulates the usual ALPHA <-> 1-ALPHA rendering, but more on the restrictions of this new system later.

Implementation:

This new system uses additional hooks placed into the rendering pipelines for the level, particles, and CPU Bound computed buffers to check whether the RenderType or ParticleRenderType will render transparent components. If so, it will capture the call and all related contextual information needed to retrigger and store it. Once the world has been rendered, it binds a new custom render target used to perform the OIT Rendering of the captured calls. Using a new system to adapt shaders, the system will calculate the transparent processing that normally happens when you perform an ALPHA <-> 1-ALPHA blended rendering on the GPU, but this time in the shader.

Afterward, the resulting color and background coverage, known as the reveal, is used for blitting the transparent rendered output to the main buffer.

Results

Expand to see examples of particles

As you can see, the smoke particles are semi-transparent and the effect cloud particles behind them are now visible.

Expand to see examples of shulker bullet halos

As you can see, the backgrounds behind the shulker halo are rendered properly and are visible.

Expand to see examples of ice block interactions

Visible here is the main example of the rendering engine working properly, the water is visible behind the ice.

New Hooks and Events

The original PoC for this PR implemented the hooks raw into the OIT subsystem. However, I always envisioned that this system runs on top of several hooks for the community.
These hooks or events are the following:

• RegisterGlslPreprocessorsEvent: Event fired when the preprocessors for an event are determined. Allows modders to register custom preprocessors that modify shaders. Used by Forges OIT system to wrap the shader execution and optionally handle the OIT render pass. Additional adaptations were made to
• RenderTargetEvent.Create and RenderTargetEvent.Resize: Allows for managing custom RenderTarget-instances and implementations, which shaders can use. Forges OIT system uses a custom RenderTarget implementation with different FrameBuffers and attached textures to handle the Transparency calculation.
• ShaderEvent.OnAttachmentInitialization fired when the shader instances are attached to GPU programs. Allows for the configuration of custom attributes when shaders are adapted.
• ParticleRenderType#requiresTransparencySorting to indicate to external systems is this particle render type uses transparency components and requires sorting. Setting this to true will allow the render type to be transformed by OIT.
• Custom render types for chunk Geometry: Although currently not exposed, the system now supports custom render types which are not the default transparency render type for chunk geometry sorting and OIT processing.

Current problems:

• The weighting function: The current function was adapted from the recommended weighting function in the paper. However, it still needs improvement. Some colors get a bit muted (see ice and honey blocks). Over the next few weeks, we can still modify these weights and improve the accuracy to match vanilla better.
• Not all forms of transparency are handled. I am looking at you: Weird additive transparency Mojang uses with the creepers...

Status:

The PR enables OIT rendering by default for now.
This is to gather feedback on the feature.
Currently, the shaders use an ARB extension for OpenGL, which is supposedly supported on all platforms Mojang runs on. Including Mac OSX; however, the actual impact still needs to be seen.

Performance impact:

By default, this has no negative impact.
However, depending on your environment, you can expect 10 to 15% FPS improvements (the more transparency-related rendering is needed for a scene, the bigger the impact, so oceans/beaches have a much larger impact than desserts). Besides this, the frame times get more stable, given that now no sorting needs to happen based on your position.

Footnotes

1. Weighted Blended Order-Independent Transparency ↩

[SizableShrimp]

Does this fix sorting w/ lightning + water/whatever? What is preventing this from working for Creepers/can that be easily fixed? And how much will this break mods doing custom stuff (not sure exactly what that may be)?

[marchermans]

Does this fix sorting w/ lightning + water/whatever? What is preventing this from working for Creepers/can that be easily fixed? And how much will this break mods doing custom stuff (not sure exactly what that may be)?

Yes, as far as I can see, it fixes the combination with Lightning + Water.
Mojang uses "additive transparency" for the creeper, but in reality, it is color multiplication without any transparency involved. So it is not needed. But it might be depending on what modders do. So please let me know whether I need to find a solution for this or not.
With regards to breaking modder's stuff.
I am not exactly sure. If you have a really custom rendering setup with custom render targets and custom massive shaders, it will break it. But 99% should just work. It should even take care of the custom setups people have for handling their particles which are transparent. I think netto it is an improvement and it should not effect many people.

[pupnewfster]

Bracket should be on previous line

[pupnewfster]

Bracket should be on previous line

[pupnewfster]

Bracket should be on previous line and this method should potentially be added to the .exc file? I believe that only affects parameters so I don't believe (?) it should cause issues with it overriding the method in IForgeGlslPreprocessor. Also might as well include the override annotation

[pupnewfster]

Probably worth a comment above this line along the lines of: Forge: Copy of REGEX_MOJ_IMPORT with support for custom prefixes just so that if mojang changes their regex at some point it is easier to know what part of this has to be updated.

[pupnewfster]

Bracket should be on previous line

[pupnewfster]

Bracket should be on a new line

[pupnewfster]

Bracket should be on a new line

[pupnewfster]

Bracket should be on a new line

[pupnewfster]

Given how both of these are NO-OP ShaderInstance consumers and ChunkGeometryRenderCall and ParticleRenderCall also both have NO-OP ShaderInstance consumers we should just create and store a single private static final Consumer<ShaderInstance> NO_OP_CONSUMER = shader -> {}; in the OITLevelRenderer and then reference that in all these places so that we only allocate a single object for all these consumers

[pupnewfster]

Bracket should be on a new line

[XFactHD]

Signature is missing the added RenderType parameter

[XFactHD]

I've played around with this and found a few issues with it:

• Distance fog doesn't fade out the world and instead fades the geometry to white, leading to the "rough edges" of the chunks at the border of the render distance still being visible
  
• Inner faces of the player model render through the outer faces in 3rd person
  
• Blending of multiple translucent blocks looks wrong (inverted order maybe?)
  
• Water appears to sometimes vanish behind translucent blocks
  
• Resource reloading makes all non-translucent chunk geometry, all block and item models in screens and text go black (flat textures like screen backgrounds and button backgrounds still work)
  
• Parts of the first-person hand model are invisible after joining a world, moving a few meters usually fixes it
  
• Fabulous mode breaks all translucent chunk geometry rendering (it shouldn't be possible to have Fabulous mode and OIT enabled at the same time)
  
• Drawing additional geometry in a RenderLevelStageEvent bypasses all patches that capture render calls in most, if not all stages, which leads to translucent geometry rendered there not being handled appropriately. In my case (particle stage with particle render target) this leads to the item-in-hand vanishing when the custom stuff renders

This was tested on commit 3d35fd1 and all points except the last one were reproduced in a plain Forgedev environment

[jellysquid3]

As perhaps a useful prior work, we also attempted to implement the WBOIT algorithm in CaffeineMC/sodium-fabric#519, but ran into a lot of issues with the approach. The algorithm doesn't handled layered translucency very well, and fails badly once you start mixing different colors (see some of the linked screenshots in the thread.)

My understanding is that the algorithm (as we implemented it) was correct and that our visual artifacts were within expectations of the original paper. We did a lot of tweaking with the weighting formula but didn't manage to produce results that were any better.

If you do want to try and go ahead with this approach, you may find better results with Moment-Based Order Independent Translucency (MBOIT). The performance isn't nearly as "good"-- though probably still much better than trying to sort the geometry on the CPU-- and it appears to handle these kinds of layered effects much better. It also shares many similarities with the original WBOIT paper, so your work here would prove a very useful platform for implementing the MBOIT paper.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[autoforge]

@marchermans, this pull request has conflicts, please resolve them for this PR to move forward.