useRenderLoop: render after updates

[andretchen0]

Description

Problem

Updates/renders share the same unordered-from-the-components-perspective callback "slot", leading to out-of-sync visuals.

Example

Here's the Cientos' playground demo for MeshReflectionMaterial. Notice the extra "jumps" in the reflection:

Screen.Recording.2024-03-29.at.15.12.31.mov

Here are two adjacent frames. Notice that the "real" elements haven't moved, but the reflection "jumps" from one frame to the next:

Why it happens

The "jumps" occur because the updates/renders are all firing in onLoop, but out-of-order from the perspective of the FBO reflection. So for a file like this ...

<script setup lang="ts">
import { TresCanvas, useRenderLoop } from '@tresjs/core'
import DemoComponent from './DemoComponent.vue'

const { onLoop } = useRenderLoop()

onLoop(({ elapsed }) => {
  console.log("Demo update", elapsed)
})
</script>

<template>
  <TresCanvas>
    <TresPerspectiveCamera />
    <DemoComponent></DemoComponent>
  </TresCanvas>
</template>

... a single "tick" of onLoop looks like this:

With this ordering, if DemoComponent renders the scene to an FBO, and is then rendered to the screen, it'll always be 1 frame behind Demo.

Suggested solution

We could follow R3F's lead here:

• Guarantee that rendering happens after onLoop
• Implement these:
  ** https://docs.pmnd.rs/react-three-fiber/api/hooks#taking-over-the-render-loop
  ** https://docs.pmnd.rs/react-three-fiber/api/hooks#negative-indices

For example, R3F's OrbitControls uses a negative index:

https://github.com/pmndrs/drei/blob/c147c2b1064bc4b457150f995bf714c2e43cf56f/src/core/OrbitControls.tsx#L58C1-L61C13

If I understand correctly, that means that the OrbitControls camera is updated before the rest of the scene, which is important in the case of FBOs like in the reflection material.

Alternative

As a simpler – though less flexible – solution, we could add a few callbacks:

• useRenderLoop().onBeforeRender() – for FBOs
• useRenderLoop().onRender() – for actually rendering to the screen

Additional context

No response

Validations

• I agree to follow this project's Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

[alvarosabu]

Thanks for opening this @andretchen0 is really insightful, I noticed that issue and didn't understand why it was happening but now I understand 🤔

Due to the amount of issues that useRenderLoop is giving lately I'm considering refactoring it to move away from the useRaFn and go for a more vanilla approach to ensure we have a unique loop per instance.

Any suggestions?

[andretchen0]

Any suggestions?

I don't think it'd be hard to roll our own.

The only part I haven't implemented in other projects is unbinding callbacks transparently when their Vue component is unmounted.

What timeline are you looking at? I've been holding off on discussions and eventual implementations of big changes until v4 is released.