Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Jet Set) Static output for OSBs / analysis of incorectly handled SFX #1

Open
Laserdisc9X opened this issue Sep 25, 2023 · 9 comments
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@Laserdisc9X
Copy link

Laserdisc9X commented Sep 25, 2023

This is a two-parter, first expanding upon the Loop flag I mentioned on twitter and second my theory on why sounds are handled incorrectly, based on my decade+ modding experience with this game.


Firstly, some sound banks output static when a specific index is extracted. For this demonstration, we will use E_POLICE.OSB, primarily focusing on indexes 08, 09 and 10, in E_SND_SE.OSB.

These 3 indexes are extracted as static, or corrupted audio; I have deduced that these sounds, among the others that extract improperly, have one thing in common: They're flagged to "loop" indefinitely.

For example, when the Police chase the player, a "HUP HUP HUP" sound is looped until the player escapes,, If you import the OSB into audacity as raw, pitch/speed shift it a bit, you can somewhat hear this sound and the two others.

Furthermore, I've deduced that this "looping flag" is done within the binary, not in-game. I can prove this by swapping the police sounds with any of the playable characters, none of which have looping audio. I switched out Tab's (CV04_YOYO (yes, it's named wrong)) and it just so happens the "HUP HUP HUP" is the same index as one of Tab's jumps, so the sound is easy to produce.

Jumping with Tab, after swapping the sfx banks, produces the police "HUP", and it indeed DOES loop, proving it's handled in-binary, since none of the players have looping audio.

Something is being read wrong, perhaps a pattern that hasn't been noticed; Something is being read improperly.


Secondly, about the incorrectly handled audio: I'm convinced it's actually correct.. (Not the static but the sounds that are the wrong pitch / speed.)

I've observed this for sometime, and played around with swapping sounds/putting new sounds; I think this was done to save space on the GD-ROM, even though the game is only like 800mb normally.

Sounds for Characters like Combo or Goji are pitched improperly, some even more complex with changed speeds, and I believe the game applies an algorithm to re-pitch them depending on their size in the binary. The manatee driver for the Dreamcast clearly has this ability given the sewers adds revert to the voices, and they're definitely not baked in given how they can glitch. (buggy port).

I swapped all of Combo's sounds, which extract """wrong""" as +9% (pitched high), with accurately pitched versions that I did in Audacity; This resulted in the game slowing down the sounds, If I pitch them properly, but speed them up, they play how I pitched them but now at a different speed.

I did the same with Goji, same result.

There is some sort of algo in the game, or it's one of the unknown header definitions, it all seems overly engineered; I cannot say i'm surprised, every level has duplicate textures, and textures from other areas. There are about 22 files that have textures for characters and enemies that never get loaded.

Just a mess.

@biggestsonicfan
Copy link
Owner

There's a lot of "extra data" in the OSB files that simply aren't documented properly yet. AMKR, NITS, PALP, EMUN, IPTC, RPCE, and ETTX metadata are all skipped, however, this doesn't apply to E_POLICE.OSB as it doesn't have any of of this metadata.

Adding some quick debug console output gives us a hint:

Extracting e_police.osb
DEBUG - e_police- 0 Offset data: 00-01-CC-03
DEBUG - e_police- 1 Offset data: 00-01-74-27
DEBUG - e_police- 2 Offset data: 00-01-FC-4D
DEBUG - e_police- 3 Offset data: 00-01-84-7E
DEBUG - e_police- 4 Offset data: 00-01-4C-A5
DEBUG - e_police- 5 Offset data: 00-01-B4-C9
DEBUG - e_police- 6 Offset data: 00-01-DC-E6
DEBUG - e_police- 7 Offset data: 01-01-04-02
FileLen issue with e_police- 8 - Original length: 570056 New length: 11456
DEBUG - e_police- 8 Offset data: 01-03-2C-16
FileLen issue with e_police- 9 - Original length: 570856 New length: 11648
DEBUG - e_police- 9 Offset data: 01-03-F4-42
FileLen issue with e_police- 10 - Original length: 570776 New length: 11648
DEBUG - e_police- 10 Offset data: 01-03-7C-70
FileLen issue with e_police- 11 - Original length: 194647389 New length: 5472
DEBUG - e_police- 11 Offset data: 01-01-04-9E
DEBUG - e_police- 12 Offset data: 01-01-6C-B3
FileLen issue with e_police- 13 - Original length: 200382435 New length: 6144
DEBUG - e_police- 13 Offset data: 01-01-94-C0

Looking at how file the offset data is parsed, we can see a pattern to our bad data chunks:

// ADPCM flag
// This is what that XX in the file offset is used for
// 00 = PCM
// 01 = ADPCM
// 03 = ???

Currently, I am handling that value 03 as incorrectly, which I've had a gut feeling has been wrong for some time. The original game the original OSBExtract was created for, Puyo Puyo~n, never had this flag.

If we look at the lengths for hints, for file 8, 11456 becomes 2CC0 in hex, and file 9's 11648 becomes 2D80. I don't see an obvious correlation to the offset data here, unfortunately.

HOWEVER, simply setting isADPCM to true when the second byte in the offset data is 03 does indeed fix the issue and the samples play correctly. So they were just ADPCM data all along that I just did not set the flag for, oops...

So the 01 in the first byte of the offset data could indicate it should loop, however I do not know if audio 11, 12, and 13 are supposed to loop. If you can confirm they do loop for me, I will append loop in the filenames of extracted audio which do loop.

@biggestsonicfan biggestsonicfan added bug Something isn't working good first issue Good for newcomers labels Sep 27, 2023
@Laserdisc9X
Copy link
Author

ADPCM this whole time? Certainly makes us feel silly.

11-13 don't loop, they should just be weapon sounds that are unused (Police don't have guns in JGR), actually they're identical to Onishima's as well, which those are single-use event sounds.

@biggestsonicfan
Copy link
Owner

As for the other half of this issue then, I do support different sample rates:

//Get Sample Rate
byte[] bSampleRate = BitConverter.GetBytes(sampleRate);

Do you have a specific example of a pitched-up extraction so I can look at the metadata?

@Laserdisc9X
Copy link
Author

A good example, from both regions (e_snd_se, snd_se) would be Combo's voice lines (cv09_combo).

In audacity, Combo can be pitched properly with -9%, Japanese -30%; This was the first one I tried to fix manually.

A majority of the US SFX seem to be from 9-11%, Japanese being 26-30%; Some of the gangs like Love Shockers also change speed, not just pitch, so they come out a little warbly if you have one aspect corrected but not the other.

A good example for that would be Japanese Tab (cv09_yoyo), when he goes "YOOOOO"; Sounds terrible if you only pitch fix it. I don't know what speed it's at, I did it by ear comparing in game and it still sounds terrible, warbly.... Same thing with a few Love Shock sounds.

@biggestsonicfan
Copy link
Owner

Analyzing cv09_combo.osb, I noticed that index 14 is actually has both the correct pitch and speed!

So, let's take a look at the "unknown data" within the OSP header for each file:

DEBUG - cv09_combo- 00 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-D6-7A-00-00-00-00-00-00
DEBUG - cv09_combo- 01 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-59-5E-00-00-00-00-00-00
DEBUG - cv09_combo- 02 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-AF-6C-00-00-00-00-00-00
DEBUG - cv09_combo- 03 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-78-41-00-00-00-00-00-00
DEBUG - cv09_combo- 04 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-DB-3D-00-00-00-00-00-00
DEBUG - cv09_combo- 05 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-8A-5A-00-00-00-00-00-00
DEBUG - cv09_combo- 06 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-C2-2C-00-00-00-00-00-00
DEBUG - cv09_combo- 07 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-89-27-00-00-00-00-00-00
DEBUG - cv09_combo- 08 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-0F-5C-00-00-00-00-00-00
DEBUG - cv09_combo- 09 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-89-59-00-00-00-00-00-00
DEBUG - cv09_combo- 10 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-15-8D-00-00-00-00-00-00
DEBUG - cv09_combo- 11 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-F5-A4-00-00-00-00-00-00
DEBUG - cv09_combo- 12 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-FA-7B-00-00-00-00-00-00
DEBUG - cv09_combo- 13 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-93-72-00-00-00-00-00-00
DEBUG - cv09_combo- 14 unknown data: 1F-00-1F-00-00-00-00-00-00-00-00-0E-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-87-5D-00-00-00-00-00-00
DEBUG - cv09_combo- 15 unknown data: 1F-00-1F-00-CD-79-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-42-73-00-00-00-00-00-00-EB-70-00-00-00-00-00-00

If we assume that only the data that's identical between 0-13 and 15 is where the pitch/speed change is different from 14, we get this:

Bytes 5-8
High: CD-79-00-00
Regu: 00-00-00-00

Bytes 32-35
High: 42-73-00-00
Regu: 3C-00-00-00

Now let's go back and look at our friend e_police:

DEBUG - e_police- 00 unknown data: 1F-00-12-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-22-47-00-00-00-00-00-00
DEBUG - e_police- 01 unknown data: 1F-00-12-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-FD-4C-00-00-00-00-00-00
DEBUG - e_police- 02 unknown data: 1F-00-12-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-E5-60-00-00-00-00-00-00
DEBUG - e_police- 03 unknown data: 1F-00-12-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-49-4D-00-00-00-00-00-00
DEBUG - e_police- 04 unknown data: 1F-00-12-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-93-48-00-00-00-00-00-00
DEBUG - e_police- 05 unknown data: 1F-00-12-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-0B-3A-00-00-00-00-00-00
DEBUG - e_police- 06 unknown data: 1F-00-12-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-1A-36-00-00-00-00-00-00
DEBUG - e_police- 07 unknown data: 1F-00-12-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-00-00-00-00-31-28-00-00-00-00-00-00
DEBUG - e_police- 08 unknown data: 1F-00-1F-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-04-00-00-00-64-59-00-00-00-00-00-00
DEBUG - e_police- 09 unknown data: 1F-00-1F-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-04-00-00-00-F4-5A-00-00-00-00-00-00
DEBUG - e_police- 10 unknown data: 1F-00-1F-00-00-00-00-00-00-00-00-0D-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-3C-00-00-00-04-00-00-00-CC-5A-00-00-00-00-00-00
DEBUG - e_police- 11 unknown data: 1F-00-12-00-00-78-00-00-00-00-00-0F-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-48-00-00-00-34-17-00-00-BA-2A-00-00-00-00-00-00
DEBUG - e_police- 12 unknown data: 1F-00-12-00-00-78-00-00-00-00-00-0C-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-48-00-00-00-00-00-00-00-34-1A-00-00-00-00-00-00
DEBUG - e_police- 13 unknown data: 1F-00-12-00-00-78-00-00-00-00-00-0F-00-00-F8-1F-F8-1F-F8-1F-F8-1F-F8-1F-19-19-19-19-00-00-00-00-48-00-00-00-E3-17-00-00-C6-2F-00-00-00-00-00-00

We can see that bytes 32-35 (with the exception of 11, 12, and 13) also have bytes of 3C-00-00-00, so I am going to guess for now this byte area has something to do with it.

I'm gonna see if I can figure anything out based on this data.

@Laserdisc9X
Copy link
Author

I suppose you could put one pattern over the other, in the OSB, and see how the sound plays in game. Take the one from 14 that plays properly and put it over a pitched index. "HEY!" should be a common combo jump sound, easy to replicate.

@biggestsonicfan
Copy link
Owner

It's not a solution, but changing the sample rate to 32 kHz seems to do the trick for most high-pitched audio.

For now I can add a parameter for sample rate to override the default 44.1 kHz, so audio can at least be extracted and listened to properly.

I can't promise a timeframe on the next release, but it should be soon. I want to try to move the project to .net core so I can automate build releases here on GitHub.

@Laserdisc9X
Copy link
Author

When you get around it, all be sure to test further.

@biggestsonicfan
Copy link
Owner

Sorry for the wait @Laserdisc9X , but I decided to do a lot of changes with the latest release. Give it a shot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants