WIP: GS: Add Pre-Round Sprite hack

[refractionpcsx2]

⚠️ THIS IS WIP - Things are gonna change and hopefully improve over time! This PR is currently broken, do not use ⚠️

Description of Changes

This attempts to preproduce hardware behaviour of texel coordinates (0.5 centre), but falls down in a bunch of cases, hence the hack. But does fix a couple of long standing bugs.

Rationale behind Changes

In some cases, the PS2 will round down coordinates which PCSX2 does not, causing some issues with rendering. A prime example of this is Manhunt which draws a square over an area to hide lights, but uses a single pixel low alpha square in order to do so, and PCSX2 current misses this pixel by 1. This PR makes it pick the right pixel.

Suggested Testing Steps

Try games that have graphical errors (in software) and see if this resolves them. I've already checked Mercinaries, and Gran Turismo 4 also gets broken by this.

Fixes #257
Resolves the water effect mentioned here #1986

This PR kind of fixes the UI on Dark Cloud but causes some misaligned backgrounds.

Showcase of the two main fixes of this PR:

Beyond Good and Evil:

Manhunt:

[5d-0]

Works perfectly! Thank you for doing this :)

[refractionpcsx2]

Works perfectly! Thank you for doing this :)

Thanks :) there's still a lot to do, I want to make it less hacky and based more on hardware test results (confirming a theory on manhunt led to this) but we need more data, but it should provide some interesting testing.

I should Also note if anybody is testing manhunt in hardware mode, you also have to enable auto flush.

[5d-0]

pcsx2:

pic from real hardware for reference:

[HelicopterP]

I don't know whether it's worth to report bugs at his point but here it is anyway. Maybe it will help somehow...
Pre-round sprite hack breaks shadow rendering in this games:
gs_20220701210748_Harry Potter and The Prisoner of Azkaban_SLUS-20926.zip
gs_20220701211609_Ratchet _ Clank_SCUS-97199.zip
gs_20220701212102_Jak and Daxter - The Precursor Legacy_SCUS-97124.zip
gs_20220701212855_Jak II_SCUS-97265.zip

[refractionpcsx2]

Thanks, I mean it's worth checking those when we get more data to check, so it's appreciated. But as said in it's current state it WILL break stuff, I kind of expect it.

[refractionpcsx2]

I haven't checked the other dumps, but I think I'm on to something here, using some information gathered on how STQ works, and used some theory on UV's (I believe we've come across this problem before)

Dark Cloud 2 seems to work in this, along with Manhunt and Beyond Good and Evil, so it feels a bit more promising.

Edit: Wild Arms 3 no longer needs the Wild Arms hack, so that's nice? At least at the time of writing this.

No doubt something else broke :D

[HelicopterP]

ALL GS dumps posted above are fine now.

[refractionpcsx2]

Thanks, but now Manhunt isn't fixed again xD

Edit: Also noticed God of War doesn't like this also.

[refractionpcsx2]

Note: This PR is still NOT right, but it's working okay.

Here's the current state of things that get fixed by this PR (unless it says "Not currently fixed" after it)

3-Nen B-Gumi Kinpachiu Sensei - sprite alignment for scene + character sprites (unknown if UV or ST, probably UV)
Armored Core 2 garbage next to energy bar and radar (UV)
Battle Gear 2 UI line ordering (interlacing) (UV)
Beyond Good and Evil water (UV)
Cho Aniki high scores font misalignment (UV)
Dark Cloud 2 UI alignment (1 pixel more than TBW, possible clamping weirdness?)
Devil May Cry 2 gap in the health bar when using SW mode (UV?)
Dynasty Warriors 5 lines in floor (UV)
Dynasty Warriors 6 lines in floor (UV)
DBZ BT3 outline positioning (UV, possibly don't need upscaling hacks anymore/need adjusting)
Everblue screen alignment/blur (UV)
Gallop Racer 6 UI alignment
Gallop Racer Lucky7 UI alignment
Manhunt lights (STQ)
Mercenaries lines in software (unknown if UV's or STQ at this point)
New Roomania Shadow connection (bilinear'd UV's)
Shadow of the colossus screen alignment (neither really better)
Shin Megami Persona 3 UI element alignment/clarity during battles
Ratatouille post process alignment (UV)
Ratchet - Deadlocked (Wild arms redundant, use HPO Vertex and nothing else)
Wild Arms 3 UI positioning (Wild arms hack redundant, use round sprite half for upscaling)
WRC 2 Extreme UI some fonts (UV)

Affected but not currently working:
Shin Megami Persona 3 minimap (STQ, 8bit trunc, not 9) - Not currently fixed.

[mirh]

I timidly believe this may have busted "Half Pixel Offset".
Nevermind...

[refractionpcsx2]

To a point, yes. They kinda do similar jobs, just at different points in the render.

[piau9000]

Dynasty Warriors 6 also suffers from the same lines that 5 do, so it's worth checking too.

[refractionpcsx2]

Dynasty Warriors 6 also suffers from the same lines that 5 do, so it's worth checking too.

do you have Dynasty Warriors 6 to check? I do not, unfortunately

[piau9000]

Dynasty Warriors 6 also suffers from the same lines that 5 do, so it's worth checking too.

do you have Dynasty Warriors 6 to check? I do not, unfortunately

I do. What do you a want me to test?

Here's a gsdump showing the bug:
(SLUS 21774) Dynasty Warriors 6 - Line Bug.zip

[refractionpcsx2]

Normal:

Preround:

For upscaling, along with preround it needs Half Pixel Offset (Normal) and Round Sprite: Half in order to get rid of the line, otherwise it's still there.

[piau9000]

Over the years, I gathered some info about DW 5 and 6. What I noticed is that, if you put half pixel offset to vertex, it would make the lines thinner, making it less obvious (but they are still there). But if you put round sprite to half (to fix the vertical lines when upscaling), it would make the lines wider again, which is curious.

[refractionpcsx2]

Over the years, I gathered some info about DW 5 and 6. What I noticed is that, if you put half pixel offset to vertex, it would make the lines thinner, making it less obvious (but they are still there). But if you put round sprite to half (to fix the vertical lines when upscaling), it would make the lines wider again, which is curious.

That is interesting, the HPO I understand as that corrects the upscaling issue, but of course it's wrong to begin with (As can be seen in native/software), so it kind of compounds the issue.

So hopefully once I've finished with this PR, that it will correct that part of it, so only HPO is required for upscaling (as above)., maybe round too, i dunno..

[RedDevilus]

Also should fix #257

[refractionpcsx2]

Also should fix #257

That was already in the OP

[Jakey757]

Fixes lines in FMV's for Xenosaga Episode III at native resolution, and when upscaling when used with round sprite full.

Also fixes lines in UI for Xenosaga Episode II when upscaling, together with round sprite full.

[refractionpcsx2]

oh well, the update I just did must have stopped it fixing the line, there was a bug so it was doing it on the wrong items in some cases.

However software mode is still receptive to it, but hardware mode doesn't care for me

[Jakey757]

hmmm, i had HPO disabled and i was also using 5x, seems like different scales break it again.

4x for example the faint line is back.
EDIT: yeah HPO is breaking it like that example you posted for me.

[refractionpcsx2]

I mean in the new version which is currently building, it seems to be acting differently to you and link

[refractionpcsx2]

Sorry, I'm a dingus, apparently I was overriding the global settings with a per-game upscale and somehow didn't notice.

Yes HW mode is fixed in native with that.

[nesaruM]

Hi, dont know if it is useful, but "007 Russia with love" has similar problem with lights like Manhunt. On 1.7.5253 lights clearly are visible through walls

[refractionpcsx2]

Hi, dont know if it is useful, but "007 Russia with love" has similar problem with lights like Manhunt. On 1.7.5253 lights clearly are visible through walls

there can be many reasons for this, the best thing to do would be to open an issue (if there isn't one) and post a GS dump with your issue, then we can look at it.

Thanks

[Jakey757]

Fixes Genji's distorted image in the software renderer #4401.

The hardware renderers show improvement, but still not quite right.