[Request] Improved Harmonix MOGG support (proper mixing and decryption)

[SCOTT0852]

I noticed that while there is support for Harmonix MOGG audio (src/meta/mogg.c), it seems to only render as mono at full volume. The games do keep track of what audio channels are for which instrument and render them in stereo accordingly. This metadata is stored in a songs.dta, in ../../config/songs.dta for the v10 files found in GH2 X360 on-disc and DLC, ../songs.dta for Rock Band series DLC, and ../gen/songs.dta for extracted on-disc Rock Band files (requires decompiling scripts when extracting, only checked in RB3 as that's the only game I have handy). The "tracks" datanode determines if a track is stereo or mono, "pans" can be used to affect how the audio is panned, and "vols" is used to decrease track volume for mixing purposes (all tracks in the mogg are as loud as possible without clipping and turned down in-game). An example is as follows (some nonessential lines trimmed for length reasons) from the Rock Band 2 export:

(
   'everlong'
   (
      'song'
      (
         'name'
         "songs/everlong/everlong"
      )
      (
         'tracks'
         (
            (
               'drum'
               (0 1 2 3 4 5)
            )
            (
               'bass'
               (6 7)
            )
            (
               'guitar'
               (8 9)
            )
            (
               'vocals'
               (10 11)
            )
            (
               'keys'
               ()
            )
         )
      )
      ('crowd_channels' 14 15)
      (
         'pans'
         (-1.0 1.0 -1.0 1.0 -1.0 1.0 -1.0 1.0 -1.0 1.0 -1.0 1.0 -1.0 1.0 -2.5 2.5)
      )
      (
         'vols'
         (-1.5 -1.5 -2.5 -2.5 -3.5 -3.5 -4.5 -4.5 -3.5 -3.5 -3.0 -3.0 -4.5 -4.5 -4.5 -4.5)
      )

Each track in this song is stereo, although if there is only one track number provided that track will play in mono (such as ('bass' (5) )). I believe any tracks that aren't assigned will automatically be part of the backing track, so those must be accounted for as well. If an odd number of tracks is provided for an instrument I believe the last one is mono and the rest are stereo, this is mainly seen with drums having stereo toms/cymbals and mono kick.

Another feature I would personally like to see is MOGG decryption to support additional audio types. The encryption has been reverse engineered and is documented at https://github.com/DarkRTA/themethod3, which is also set up to be usable as a library. Only MOGG version 10 is unencrypted, the rest of the versions from 11 to 17 use the same encryption method with varying ways to derive the decryption keys. One of the main reasons I would like encrypted file playback is to use in conjunction with metadata parsing for RPCS3 DLC libraries, as the PS3 versions of the games use custom PS3-optimized MOGG loading code that appears to not support unencrypted v10 due to an oversight. Decrypted files can't be in the same spot as the encrypted files, and you can't just replace encrypted with decrypted outright without making the game unplayable. RPCS3 is currently the best emulator for the games, so it's something that a lot of people already have set up.

My current workflow for listening to these tracks is to decrypt the MOGG, throw it in a folder full of other decrypted MOGGs, open it in Audacity, and then spend time marking tracks as stereo and adjusting their volume manually. As vgmstream already has very basic v10 support, I'd love to see it polished up and extended to other MOGG versions.

[bnnm]

For the first point, vgmstream mainly does decoding and that seems more of a text parser's job, so I feel it's better handled by some script/program that reads .dta (in original or decompiled forms) and creates .txtp files for vgmstream, more flexible overall (see wwiser for a similar concept). TXTP can mix instrument layers and change volumes but can't do panning at the moment, probably could be added later.

For the second point, would need samples to check. Even if decryption is added keys may need to be kept in external files though.