Jittery camera movement when physics tick rate doesn't match FPS

[TranquilMarmot]

Please read follow-up comment; this may be able to just be closed!

Issue description

phantom-camera version: 0.6.4
Godot v4.2.1.stable - macOS 14.1.1 - Vulkan (Forward+) - integrated Apple M1 - Apple M1 (8 Threads)

This is a bit of a duplicate of #173 but I think that there may need to be some updated docs around it for some best practices. I spent the whole day debugging physics jitters and found out that it was the camera that was the source of it 😅

I think this comment is very relevant: #179 (comment)

Have been trying to find a way that meant you didn't need to use the smoothing-addon, but it might not be a feasible thing to automatically solve for this addon alone. Though there is still a jitter issue when you disable the damping and have a PhysicsBody a target. The obvious solution would be to use the visual representation (mesh/sprite) node as the Follow Target, but wonder if there's a way to still allow the PhysicsBody to be used as a target. Obviously, it's a fairly minor thing and won't lose any sleep over it.

This current solve does also mean that the requirement of the smoothing-addon needs to be communicated, especially to those just picking up the addon / Godot, as to not cause people to think something is fundamentally broken. Which is probably my biggest concern.

(hi, it's me, people thinking something is fundamentally broken 😄 )

The setup I have is like...

• CharacterBody3D (RigidBody3D)
  • CollisionShape3D
  • Smoothing (from smoothing-addon)
    • MeshInstance3D
• PhantomCamera3D (follow mode "Simple", follow target MeshInstance3D above)

For testing purposes, I've set Physics Ticks per Second to 10 in the project settings.

Using a static camera, you can see that the smoothing-addon is working as intended. The mesh follows the collision shape smoothly, even though the physics update is slow:

Screen.Recording.2024-03-27.at.12.16.40.AM.mov

However, if you set this to follow mode "Simple", the camera movement is jittery even if you follow the mesh instance that's a child of the Smoothing node.

Note that this behavior exists even with damping turned on! It also happens no matter what child of the rigid body you follow. It's surprising to me that this behavior happens when following the MeshInstance3D since it should be smoothed (but maybe this is happening in _physics_process so the smoothing is done before the camera updates?)

Screen.Recording.2024-03-27.at.12.15.43.AM.mov

There even appear to be issues when using smoothing-addon with the physics tick rate much higher than the FPS. Here's an example of a physics tick rate of 144, zoomed in so that you can see the "vibration" of the camera following the object:

Screen.Recording.2024-03-27.at.12.22.11.AM.mov

That same vibration is not present with the static camera, so it's something in the way that the camera is following the object:

Screen.Recording.2024-03-27.at.12.23.44.AM.mov

I wonder if there should be a setting to switch whether you want a specific camera to update inside of _process or inside of _physics_process?

Minimal reproduction project

Reproduction project: https://github.com/TranquilMarmot/phantom-camera-jitter-repro

See README for details.

[TranquilMarmot]

Actually, just tried out the 0.7 branch in the reproduction project and it seems to work as expected!!! 🎉 I didn't realize that #179 wasn't released yet 🤦

Screen.Recording.2024-03-27.at.12.57.01.AM.mov

Following the RigidBody3D is jittery, but following instead the MeshInstance3D works totally fine.

---

Feel free to close this issue if it doesn't provide any value; I'd say that the upcoming release fixes the bug.

[ramokz]

Will keep this issue open until the release is out. Every test for this makes a difference, so thanks for sharing the MRP :)

[Nohac]

I experienced some jitter when I tested out 0.6 with 2D camera. Given this comment I wanted to test 0.7 as well, turns out it was much worse.
The game is running at 100fps and the physics are unchanged. I tried changing camera processing mode from Idle to Physics which has helped with standard godot camera jitter, but didn't change anything in this case. Forcing the game to run at 60fps fixes it completely (it actually looks better/smoother then standalone godot camera).

Video of jitter

jitter2d.webm

Sorry for the bad recording, the jitter did not show up with my screen recorder software, so I had to use my phone.

[ramokz]

The way the system works in 0.7 requires a bit of extra setup. So if you're only assigning the Follow Target to the character, which I'm guessing is a CharacterBody2D(?), then by default jitter, yes, is to be expected.

Would stay tuned for the release notes of 0.7, which covers what to do in more detail.

[Nohac]

I used the same setup as stated in the docs for 0.6, with the target being a CharacterBody2D.
Tried adding the camera as a child of the CharacterBody2D, which removed the jitter, but then non of the settings worked.

Looking forward to the release!

[ipinzi]

In case it helps, I turned the pixel perfect setting on the camera off and changed the _process in the camera and host gd scripts to _physics_process and it works well with both the characterbody2D and rigidbody2D.

[ramokz]

Don't recommend moving the logic to the _physics_process for the reasons mentioned in the 0.7 release notes.

Would suggest following the steps mentioned in the FAQ

[ipinzi]

That was the first thing I tried but 1. It didn't work for pixel perfect cameras and 2. the solution is not the ideal one if the user needs to use the same smoothing paradigm for networking rollback etc which will make the plugin more awkward to use.

The fundamental issue IS the tick rate in which the camera is moving if the target is a physics object. Cinemachine solves this by allowing the brain to execute its processes in FixedUpdate as well as LateUpdate (This is important as right now it seems the camera interferes with velocity/position as it locks the rigidbody/transform in place if set up incorrectly) or it has a SmartUpdate feature which will dynamically pick between the two depending on situation.

[ipinzi]

After reading your release note I understand the problem you are facing. It can be solved by having an enum that lets the code run in either physicsprocess or process then you can extend that with a smart update by detecting the phantom camera target’s node type and auto switch between the two.

[audeck]

I won't pretend like I know what, how and most importantly why Cinemachine does all the things it does, so correct me if I'm wrong.

Conceptually, cameras (and, by extension, phantom cameras) shouldn't ever be physics objects. Every game object (i.e. ones that interact with the world) you want to render to your game's window should have a) a physics representation and b) a visual representation.

There is then a clear separation of concern with a) handling where the physics representation should be after interacting with other physics objects at physics tick intervals (i.e. in _process_physics for Godot), and b) handling where a visual representation is depending on only where it's corresponding physics representation is at frame rate intervals in (i.e. in _process).

In this case, correctly handling networking rollback, or even the object teleporting instantly, is up to b) (albeit obviously with some help from your networking code, etc.). Having a separate node handling solely where the object should be rendered is beneficial for many reasons.

As an example, imagine you have a 2D game, and you want a true pixel-perfect camera (i.e. no pixel misalignments - even for moving objects). However, you have a moving object, intentionally with no friction, with a vector of (0.3, 0) getting applied every physics tick. If you had only a) and a camera tracking it, you'd be forced to round the position of a) down every physics tick, essentially eliminating it's motion. Once you introduce b), you can let a) keep track of it's position with decimal precision, and let b) handle the rounding down/up.

As another example, you can have the physics engine running at 20 ticks per second (wink), and still have smooth motion of objects on the screen at 1000+ fps if b) uses techniques such as interpolation using frame times.

So then cameras - objects that should concern themselves with only the visual representations of objects - should update at frame rate intervals (i.e. in _process). Allowing cameras to update in _physics_process is possible, but it hides an underlying issue with how we structure and think about our game objects, as I understand it.

Sorry for the wall of text 🤓.

[ramokz]

To back up what @audeck says, jittery in this context isn't an issue with the physics node being jittery as it moves around, but rather the thing that visually represents it, i.e. the Sprite / Model inside of it.

I could not for the life of me record actual examples of how _process and _physics_process differs as the exports always smoothed out the result, so have instead attached 2 nearly identical projects — one using the current approach with _process, and another that uses _physics_process to move the camera.

_process_example.zip

_physics_process_example.zip

Each project includes 2 example scenes, one for 2D and one for 3D, which can be found in the root directory. More concretely, the _physics_process project has all the camera movement done in the _physics_process and a disabled repositioning of the PlayerVisual inside the player_controller.gd (attached to the playable character nodes). The last bit is how the smooth visual representation gets achieved in the _process project.

The 2D and 3D scenes both contain a node, Sprite2D and MeshInstance3D respectively, that moves back and forth using a built-in Godot Tween animation. Code can be found inside scripts attached to their respective scene's root node.

Both projects use an identical project setup of:

• PhantomCamera3D Node
  • Follow Target = PlayerVisual
  • Follow Damping = true
  • Follow Damping Value = Vector3(0.1, 0.1, 0.1)
• Project Settings
  • Physics Ticks per Second = 31
  • Viewport Window (width/height) = 1920 x 1080

The observation from the above projects with those settings should show the one using the _physics_process has very jittery animation, particularly when it comes to the Tweening node in the background. Whereas the scenes from the project that runs in the _process should be a lot smoother.

If you modify the Physics Tick per Second to be, say, 60 in the project settings for the _physics_process project, then that should smooth out the player movement. But what it doesn't resolve is anything that is animated using the _process such as any Tween animations, which will remain jittery if you move while they're moving.

So I am not confident that a dynamic system that sets the camera to move in _process or _physics_process depending on its target type will resolve the underlying issue of it impacting other animations that might be happening in a scene.

[ipinzi]

I understand. I think the issue stems from Godot not having any underlying interpolation for physics. 3.5 had this but is yet to be implemented from what I have read. The engine itself should be responsible for this interpolation under the hood in process_physics but as of 4.2 I still don’t think that has happened. Essentially doing what you guys are doing with smoothing, this should be happening between the rigidbody node and the sprite node with the user having no idea.

When it does eventually happen, mimicking the way cinemachine handles this would work.
You could test the theory by testing it out with a more sophisticated physics system like rapier2D. That should have underlying physics interpolation. (Don’t know for sure haven’t tried it yet)

[Nohac]

Just found out that 2D physics interpolation has been implemented for godot 4.3 godotengine/godot#88424
I'm just going to hold out on using this addon until that is released

[ramokz]

I understand. I think the issue stems from Godot not having any underlying interpolation for physics. 3.5 had this but is yet to be implemented from what I have read. The engine itself should be responsible for this interpolation under the hood in process_physics but as of 4.2 I still don’t think that has happened. Essentially doing what you guys are doing with smoothing, this should be happening between the rigidbody node and the sprite node with the user having no idea.

100%, it ideally shouldn't be on the user to figure out how to set it up correctly — as most will be confused until looking it up otherwise. Should hopefully just be a case of waiting for the engine to add that in to improve that usability.

When it does eventually happen, mimicking the way cinemachine handles this would work.
You could test the theory by testing it out with a more sophisticated physics system like rapier2D. That should have underlying physics interpolation. (Don’t know for sure haven’t tried it yet)

Good idea, will try to run some tests with that and Jolt out at some point.

[blai30]

I am on the 4.3 Dev 6 snapshot and the issue is still happening when using PhantomCamera2D. I tried changing Camera2D's Process Callback to Physics, Physics Interpolation Mode from Inherit to On, still seeing the issue. If I remove PhantomCamera2D and PhantomCameraHost from the scene entirely, the issue is gone.

https://godotengine.org/article/dev-snapshot-godot-4-3-dev-6/

[Nohac]

@blai30 Did you enable the settings mentioned in the PR?

Make sure to enable the project setting physics/common/physics_interpolation and set physics/common/physics_jitter_fix to 0.0 when testing.

[blai30]

@Nohac Yes, that did not seem to help

[audeck]

@blai30 Fixed the repository's 2D dev scene for 4.3 dev6 with physics interpolation on audeck/physics-interpolation-fix (don't mind the first commit). Basically, I changed the following:

• Inside project settings (as per @Nohac):
  • Changed physics/common/physics_interpolation to true
  • Changed physics/common/physics_jitter_fix to 0
• Inside phantom_camera_2d.gd:
  • Changed _process to _physics_process
  • Changed every occurence of get_process_delta_time to get_physics_process_delta_time to get follow_damping working
• Inside phantom_camera_host.gd:
  • Changed _process to _physics_process
• In the dev_scene_2d.tscn scene:
  • Set the active PlayerPhantomCamera2D's follow target to just the main CharacterBody2D player node, not the PlayerVisuals node
• In the playable_character_2d.tscn scene (i.e. the player scene):
  • Moved the PlayerSprite so that it's parent is the main CharacterBody2D node, not the PlayerVisuals node

What's important is not mixing the previous manual interpolation (using either the project's PlayerVisuals or lawnjelly/smoothing-addon) with the physics one.

From the upstream discussions it seems that the Godot team wants to prioritize making things as easy as possible for people just getting into Godot/game dev in general. @ramokz To support your previous comment, I think this addon should mirror that and support using _physics_process as well (with a switch and/or auto-detection to decide, as @ipinzi already mentioned). Not a pressing matter, as only dev6 is out so far, but it'd be nice to have a major release prepared before the official release of 4.3.

[Nohac]

I tested this now, and it does not work out of the box as @blai30 says, applying @audeck solution fixes it.
What I don't understand then, shouldn't physics interpolation (fixed timestep interpolation) sync physics "movement" with the "visual" representation (_process)?

So I'm not completely sure what the interpolation actually does. With @audeck changes and interpolation disabled, it is a bit more jittery, but it's not terrible. It would be nice to get the godot devs to give an "official" recommendation of what to do in this case.

[ramokz]

Tried out the 4.3 dev6 build as well.
By the looks of it, the change in Godot 4.3 appears to resolve the issue I posted earlier.

@Nohac if you comment out the following code in addons/phantom_camera/examples/scripts/2D/player_character_body_2d.gd, or select the script attached to the playable character, that should solve the jittery character movement as well.

func _process(delta) -> void:
	_player_visuals.global_position = _physics_body_trans_last.interpolate_with(
		_physics_body_trans_current,
		Engine.get_physics_interpolation_fraction()
	).origin

This code exists as a manual solution to the lawnjelly's smoothing-addon, so it likely interferes with the other setup.

[ramokz]

Have attached a short demo of having the physics_interpolation (inside Project Settings) enabled and, at the end, disabled — the difference is very stark. It's obviously only a tiny test sample, but so far, it looks like it addresses the issue.

physics_interpolation_demo.mp4

To go back to what @ipinzi proposed, do agree that with this change in Godot 4.3, and assuming it works consistently for others, the PCam2D and PCamHost should support automatically applying its movement logic in the _physics_process rather than the _process if it follows a physic-based node.

[audeck]

This code exists as a manual solution to the lawnjelly's smoothing-addon, so it likely interferes with the other setup.

Yes. The built-in physics interpolation's inherent limitation is that it makes the game's physics run 1 tick behind (in reality it now runs 2 ticks behind, but that's irrelevant for this), since instead of just calculating the physical position and setting the object's global_position to it, now the calculated position is where it should be and the actual global_position needs to interpolate to the calculated one inbetween the current and the next tick.

That means that during _physics_process, the global_position getting read in player_character_body_2d.gd is effectively 1 tick in the past. The script then interpolates the visuals to that position; but since it is inherently constrained in the same way the built-in physics interpolation is, the visuals' position ends up being 1 additional tick behind.

So say we have three physics ticks - #0, #1 and #2 - with the game just after finishing #2. Inbetween #2 and the next tick, the physics node is interpolating between positions at #1 and #2, while the visual node is interpolating between positions at #0 and #1. It is then very easy to introduce jitter with any sort of discrepancy in these positions (mainly due to differing tick rates and FPS), since the camera is following the "invisible" physical node, while the sprite is lagging behind at different distances each frame.

... shouldn't physics interpolation (fixed timestep interpolation) sync physics "movement" with the "visual" representation (_process)?

So yes, I'm pretty sure it does, as long as the visual representation isn't trying to do the same thing as well, but with different data.

On another note, with the dev scene using physics interpolation, the camera position seems to lag behind the player position by 1 tick (noticeable on very low physics tick rates), but that's material for a separate issue after 4.3.

[ipinzi]

That's great news! I'm glad this came together so quickly. It should also help the camera not fight against the interpolation of other physics systems.

[TranquilMarmot]

Now we just need the same interpolation to come to 3D 😉

[ramokz]

Have made a draft PR with the changes to support the Godot 4.3 phyiscs interpolation #294

It technically all works and supports both 4.2 and 4.3, but due to an issue (described in the PR) I won't be merging this in until that gets resolved. From the test included in the PR description, it points to it being a bug in the current Godot implementation of the physics interpolation.

[ramokz]

The PR should now be updated to resolve the aforementioned issue(s).
Any help testing the changes in 2D projects for Godot 4.3 would be greatly appreciated!
PR can be found in #294

Testing in Godot 4.2 would also be beneficial, mainly just to confirm that there's no regression.