Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ratchet & Clank: Tools of Destruction/Resistance: Fall of Man instanced mesh system broken #10754

Closed
JimScript opened this issue Aug 23, 2021 · 34 comments · Fixed by #15483
Closed
Assignees

Comments

@JimScript
Copy link

JimScript commented Aug 23, 2021

Quick summary

In some early Insomniac games, only one foliage instance can be visible at a time

Details

In Ratchet & Clank: Tools of Destruction, New minor foliage will replace rather than supplement other older foliage.
Foliage fades in as normal:
screenshot-2021_08_23_15_09_26
screenshot-2021_08_23_15_09_37

And then when a new foliage loads in it completely replaces the old one:
screenshot-2021_08_23_15_09_48

Now this would be a minor nitpick, if Resistance: Fall of Man (another insomniac game) didn't use the system more extensively and for a gameplay purpose like hiding mines.

Here is an RSX Capture:
BCUS98127_20210823151612_capture.zip

And here is the system:

  • SYS: RPCS3 v0.0.17-12659-d0451932 Alpha | HEAD
  • SYS: AMD Ryzen 5 3600 6-Core Processor | 12 Threads | 15.95 GiB RAM | TSC: 3.600GHz | AVX+ | FMA3
  • SYS: Operating system: Windows, Major: 10, Minor: 0, Build: 19043, Service Pack: none, Compatibility mode: 0
  • RSX: Found vulkan-compatible GPU: 'GeForce RTX 2070 SUPER' running on driver 456.38.0.0
  • SYS: Firmware version: 4.85
@JimScript JimScript changed the title Early Insomniac games Foliage problem Ratchet & Clank: Tools of Destruction/Resistance: Fall of Man foliage sorting broken Nov 20, 2021
@JimScript
Copy link
Author

Well I might as well show an example of Resistance: Fall of Man:
BCUS98107_screenshot_2021_11_20_04_05_48
BCUS98107_screenshot_2021_11_20_04_05_52
BCUS98107_screenshot_2021_11_20_04_05_58

RSX Capture:
BCUS98107_20211120040735_capture.zip

@JimScript
Copy link
Author

Sorry if this is confusing, here is a video of it in action:

RFOMFoliageProblem.mp4

@JimScript
Copy link
Author

Here is a visual aid on the differences between ps3 and rpcs3 foliage sorting logic in Ratchet and Clank Future: Tools of Destruction.
PS3:
FoliageDemoPS3
RPCS3:
FoliageDemoRPCS3

Sorry I couldn't get a photo or video of a real ps3, I don't own a capture card or something like that. But there is a surprising amount of foliage missing in these games because of this error, more so than I originally thought.

@JimScript
Copy link
Author

JimScript commented Feb 12, 2022

Ok, so I figured my phone picture was good enough to compare so...
PS3:
0212221204 1

RPCS3:
BCUS98127_screenshot_2022_02_12_12_08_42

And with this I see that some of the foliage is in clusters that are never seen even in the culling order that RPCS3 produces for some reason. I wonder what causes it to only show one mesh at a time, but 2-3 meshes in Resistance: Fall of Man?

@JimScript
Copy link
Author

Along with the previously stated sorting issue, it appears that some foliage is rendering only as an outline. This example is in Quest for Booty, but it has much the same rendering as ToD does, so it is applicable to both games:
NPUA80145_screenshot_2022_02_15_14_14_41
RSX Capture:
NPUA80145_20220215141422_capture.zip

@JimScript
Copy link
Author

OK, so I have been playing around with renderdoc to see what the problem is and this is what I found. The missing foliage is well and truly not there, no draw calls except for one solitary plant in this foliage system, which leads me to believe this is a cpu problem. I would guess this is an spu problem as a task like this seems too minor for the ppu. What I think what is suppose to happen is this spu function is to go around and update the foliage to different levels of opacity as the camera gets closer, but in RPCS3 the function stays at the first foliage it can find until it has to make a unique mesh fully opaque wherein it goes to the new foliage and dumps the old one.

As for the outline shader, I may not know much about these things, but I don't think a single minor mesh is suppose to draw over a thousand times in a single frame. A third of the frame render time is dedicated to that one plant, and to add to that the first several hundred draws just draw the same few pixels on the screen. Although that could be just how the game does it, but that seems like a horrible optimization.

Anyway that is just some speculation on my part.

@kd-11
Copy link
Contributor

kd-11 commented Feb 18, 2022

Needs testing on AMD/INTEL. NVIDIA is notorious for having rounding issues that can cause strange visuals like this.

@Shadowsplace
Copy link

I tested it on two different machines in Tools of Destruction and got the same result.

  • SYS: RPCS3 v0.0.20-13288-4be9e0d3 Alpha | HEAD
  • SYS: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz | 8 Threads | 15.92 GiB RAM | TSC: 2.394GHz | AVX+ | FMA3
  • SYS: Operating system: Windows, Major: 10, Minor: 0, Build: 19043, Service Pack: none, Compatibility mode: 0
  • RSX: Found vulkan-compatible GPU: 'GeForce GTX 770M' running on driver 425.31.0.0
  • SYS: Firmware version: 4.88
Nvidia.Capture.mp4
  • SYS: RPCS3 v0.0.20-13288-4be9e0d3 Alpha | HEAD
  • SYS: AMD Ryzen 3 2200U with Radeon Vega Mobile Gfx | 4 Threads | 7.66 GiB RAM | TSC: 2.495GHz | AVX+ | FMA3
  • SYS: Operating system: Windows, Major: 10, Minor: 0, Build: 19043, Service Pack: none, Compatibility mode: 0
  • RSX: Found vulkan-compatible GPU: 'AMD Radeon(TM) Vega 3 Graphics' running on driver 2.0.173
  • SYS: Firmware version: 4.88
AMD.Capture.mp4

Both using Vulkan but OpenGL doesn't seem to change anything (on Nvidia at least).

@JimScript
Copy link
Author

JimScript commented Feb 19, 2022

I think there are two separate issues in this issue, the foliage sorting and the outline shader. I think what kd-11 is talking about is the shader issue. What you have tested is the sorting issue. If you want an example of the shader problem in action, go to hoolifar island in QFB. If you don't have QFB, Planet Mukow's foliage has the same problem, so you can check it there.

@JimScript
Copy link
Author

Hey @Shadowsplace, can you check the foliage in the imperial fight festival with your AMD machine? It is different than the ones on cobalia and I was wondering if @kd-11 is right about this particular part of the issue is vendor related. And sorry I couldn't do it, I don't own any AMD gpus. Here is an example of the problem on Nvidia hardware:
BCUS98127_screenshot_2022_02_27_14_49_44
If your AMD machine still makes the foliage here an outline, then it isn't vendor related.

@Shadowsplace
Copy link

Shadowsplace commented Mar 6, 2022

Sorry about the delay. Same specs (RPCS3 v0.0.21-13333-aafd74f9 Alpha | HEAD build)

  • Nvidia
    Nvidia Plant

  • AMD
    AMDplant

It appears the issue isn't vendor related.

@JimScript
Copy link
Author

JimScript commented Mar 6, 2022

Thank you for testing this, Shadowsplace, with this we now know both issues are not vendor related. I still think that the sorting part is a spu(or ppu) problem, but I don't know what could cause this particular shader to render as an outline.

@JimScript
Copy link
Author

I was looking at an old resistance video when I noticed that the amount of foliage missing is worse than Tools of Destruction's. Here is a screenshot of RPCS3 in roughly the same location as 0:28 in the Somerset(40p) map:
BCUS98107_screenshot_2022_05_11_15_06_20

@homembarata
Copy link

Wish I had seen this thread before...

It's not just grass, but small stuff like small rocks are also missing. Here's a later level in Resistance:
rpcs3 exe Screenshot 2021 07 10 - 02 08 59 27

In the original hardware there are a bunch of small debris on the floor (not exactly the same screen but the same area):
Resistance_ Fall Of Man _ PS3 _ Full Game Playthrough Walkthrough _ No Commentary 3-57-4 screenshot

My guess is that this is an instanced mesh system, which is common for this kind of small objects and foliage. And the emulator is only rendering one instance at a time. Just throwing some ideas here.

@JimScript
Copy link
Author

It probably is that since I made a Renderdoc of this issue and didn't find any of the missing meshes anywhere in it. But it also seems to be different than the fade in/out as that works correctly with the one mesh that does render. Also certain multiplayer maps can run more than one at a time for some reason.

@JimScript JimScript changed the title Ratchet & Clank: Tools of Destruction/Resistance: Fall of Man foliage sorting broken Ratchet & Clank: Tools of Destruction/Resistance: Fall of Man instanced mesh system broken May 19, 2022
@isJuhn
Copy link
Contributor

isJuhn commented May 19, 2022

I believe ratchet games culls lots of stuff in case of low fps before the rendering stage at all. I've checked a lot of ToD renderdocs and if it runs too slow a lot of stuff is not even in the renderdoc. But at the same time the foliage is never there no matter the fps so it might be some stage before the render stage that removes them, and not some rsx culling.

@JimScript
Copy link
Author

JimScript commented May 25, 2022

I have thought a bit about the possibility of this being related to a performance cull, and now I think that this culling system might not be receiving a proper(or any) performance report. That would explain why there is generally only one mesh rendered at a time, because to the instanced mesh system the PS3 is getting absolutely slammed, so it renders the bare minimum amount of meshes it can(only one). That would also explain why Grimsby(40p) in CTF can render more meshes at once, because it is getting a lighter report. Now this is all guesswork, but it is the only thing that I can think of that would cause this.

@Darkhost1999
Copy link
Contributor

I have to host the video somewhere else because GitHub will say the video is corrupt or too big.
Oblivion (looks better upscaled and if I load all of the environment first.) Trees lose leaves as you approach them.
https://youtu.be/rBs_uqE2V1k
RPCS3.log

OpenGL Comparison
https://youtu.be/8nowDtWDMJY
RPCS3.log

I know this is a different game. Spooky. But the issues look the same.
I mean let's compare Oblivion only loses leaves not entire plants.
All 3 games lose foliage at similar distances to each other.
The other games plants are on the ground. Oblivion's leaves are off the ground.
If you want a separate issue ticket just lmk.

I can test on my Intel uhd 620 igpu laptop later for a comparison there? Would that be helpful or because igpus are not officially supported would it be meaningless?

@kd-11 kd-11 self-assigned this Oct 17, 2022
@kd-11
Copy link
Contributor

kd-11 commented Oct 17, 2022

This one never got triaged. I've added it to backlog.

@JimScript
Copy link
Author

I have to host the video somewhere else because GitHub will say the video is corrupt or too big. Oblivion (looks better upscaled and if I load all of the environment first.) Trees lose leaves as you approach them. https://youtu.be/rBs_uqE2V1k RPCS3.log

OpenGL Comparison https://youtu.be/8nowDtWDMJY RPCS3.log

I know this is a different game. Spooky. But the issues look the same. I mean let's compare Oblivion only loses leaves not entire plants. All 3 games lose foliage at similar distances to each other. The other games plants are on the ground. Oblivion's leaves are off the ground. If you want a separate issue ticket just lmk.

I can test on my Intel uhd 620 igpu laptop later for a comparison there? Would that be helpful or because igpus are not officially supported would it be meaningless?

I'll admit, I don't see the similarity. In TOD & R:FOM they both can only show one or few pieces of foliage at a time with them popping around as you get closer or farther, this seems more gradual(except for the trees suddenly popping up in the background in the Vulkan example). I wouldn't even say it was the outlined plants part of this issue, as that remains consistent at any distance with any renderer. But maybe the underlining renderer could be used differently?

@Darkhost1999
Copy link
Contributor

Darkhost1999 commented Oct 17, 2022

I'll admit, I don't see the similarity

You never see similarity to ANYTHING I say so IDK
OGL did same thing as vulkan tree pop in I just needed to load the water so there actually is no difference between renderers

@Darkhost1999
Copy link
Contributor

I definitely see as you described this
RPCS3: FoliageDemoRPCS3

@JimScript
Copy link
Author

I'll admit, I don't see the similarity

You never see similarity to ANYTHING I say so IDK OGL did same thing as vulkan tree pop in I just needed to load the water so there actually is no difference between renderers

I'm sorry about that. But issues with Oblivion were fixed with the PC uptime problem I posted, so maybe it is similar and would be fixed with this.

@JimScript
Copy link
Author

JimScript commented Mar 8, 2023

I think the reason why some of the ground plants look hollowed out and spindly is that that the alpha is limited to around 127 and anything past that is all zeroed out for that shader, making the bulk of it not render. This is the alpha of the QFB plant edited to demonstrate:
QFBPlantBrokenAlpha
And the original to compare:
QFBPlantOriginalAlpha

@JimScript
Copy link
Author

I've found something interesting about the hollowed out shader:
BCUS98127_screenshot_2023_09_24_22_10_18
It doesn't block shadows, even though it does on a real PS3. It makes sense since this is a transparent material, but what if it can block shadows on PS3 because it's filled with an opaque material, like it fades from transparent into opaqueness. And what if it's hollowed doing around a thousand draw calls because it's caught in a loop trying to get the opaque version but it's just getting the transparent version over and over again until the emulator gives up? I guess I'm making a lot of assumptions here.

@JimScript
Copy link
Author

Turns out it does draw the opaque part first in another color pass, but it only does so on a very slim threshold. That must have been the few pixels I saw being drawn first before, I could only really see it now when the LOD bias offset is set to 10 since the threshold is so tight.
Opaque Shader Drawn:
ToDPlantOpaque

Translucent Shader and Shadow Drawn:
ToDPlantTranslucent

Still don't know why both the opaque and translucent parts draw hundreds of times each frame though.

@kd-11
Copy link
Contributor

kd-11 commented Apr 17, 2024

Test #15471. It's not directly aimed at this issue (I haven't deep-dived this one yet) but the cause could be related.

@rogeriodomingos
Copy link

I can still replicate the issue in Resistance: Fall of Man as of version 0.0.31-16351-ea891829.

  • SYS: RPCS3 v0.0.31-16351-ea891829 Alpha | master
  • SYS: AMD Ryzen 5 3600 6-Core Processor | 12 Threads | 15.93 GiB RAM | TSC: 3.600GHz | AVX+ | FMA3
  • SYS: Operating system: Windows, Major: 10, Minor: 0, Build: 22631, Service Pack: none, Compatibility mode: 0
  • RSX: Found Vulkan-compatible GPU: 'AMD Radeon RX 6600 XT' running on driver 2.0.299
  • SYS: Firmware version: 4.90
Untitled.video.mp4

@JimScript
Copy link
Author

You know, with Resistance: Fall of Man showing multiple clusters rather than a single plant, could that in fact be a different issue? I mean this issue turned out to be different from the one that affected Resistance 2 and ACIT on NVIDIA, who's to say this isn't a different issue either? Still, this is only speculation, I won't make another issue until we know for sure.

@rogeriodomingos
Copy link

rogeriodomingos commented Apr 18, 2024

It might be the same issue, but it has improved. Here is a screenshot with version 0.0.30-15956:
Screenshot 2024-04-18 135822

And version 0.0.30-16096 with the same behavior as the previous video I posted:
image

#15471 doesn't seem to have made any difference in this case. I'll try to narrow down the commit that made the improvement.

@rogeriodomingos
Copy link

rogeriodomingos commented Apr 18, 2024

It was actually this pull request that did it:
#15065

I see that you also reported an improvement in Tools of Destruction so it might be related.

Summing things up: #15065 improved the issue but #15471 had no effect.

@kd-11
Copy link
Contributor

kd-11 commented Apr 18, 2024

This one is weird. I'll have to investigate assuming it's multiple issues. Hopefully it's not related to depth precision since that can only be correctly resolved with software rendering.

@kd-11
Copy link
Contributor

kd-11 commented Apr 18, 2024

Ok, nothing too special here. It's what I expected. PS3 doesn't have dedicated instancing hardware, but it can do instancing through like a million different methods by maniupulating the FIFO puller. We just didn't have a pattern match for the registers this game is using to implement it.
Fix incoming.

@kd-11
Copy link
Contributor

kd-11 commented Apr 19, 2024

#15483 implements the feature, though in a very suboptimal way right now. A lot more work is needed to get this ready for merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants