[Request] Nintendo Mobiclip/Actimagine (.mo/.moflex/.fv/.vx/.mods) support

[Sukotto-1999]

Nintendo Europe/Actimagine developed a series of formats called Mobiclip/Actimagine for FMV playback on 3DS/DS/Wii, Nintendo Europe also developed a special codec called FastAudio which vgmstream does not support. Mobiclip also supports multitrack audio (useful for multiple languages or other)

I have a bit of spectification what would look like:

.vx/fv (VXDS/FVDS) (FastAudio/A3 V1) (NDS)
.mods (MODSN2/MODSN3) (NDS IMA-ADPCM, PCM16, FastAudio/A3 V1, FastAudio/A3 V2) (NDS)
.moflex (L2) (MOFLEX IMA-ADPCM, PCM16, DSPADPCM, FastAudio/A3 V2) (3DS)
.mo (MOC3/MOC5) (IMA-ADPCM, PCM16, FastAudio/A3 V2, Custom Vorbis (Nintendo Channel, Wii no Ma)) (Wii)

The audio codec for Actimagine .VX and .FV formats, and .mods if using original audio format are a variant of LPC (known as FastAudio V1) with 128-sample frames coded in 20/14/12/10 bytes and these are all stored in codebook data (3124 bytes) stored in the container.

There are 2 versions of the Mobiclip DS Video Codec. .mods files created with AvitoMobiclipDS version from 2007, start with MODSN2. while the others begin with MODSN3. MODSN2 has the audio in a different location or specification. Another ones are for .mo. one is MOC3, one is MOC5.

Layout:
interleave (possibly blocked?) (mods/moflex/mo)
blocked (mods/vx/fv)

Games affected for example:
Final Fantasy Crystal Chronicles Ring of Fates (NDS, uses VX)
Nintendo Channel (Wii, uses Vorbis audio)
Phantasy Star Zero (NDS)
eCrew Development Program (rare NDS game)
Rise of the Guardians (Wii, multitrack and FastAudio)
Yo-Kai Watch Demo (3DS)

Multiple samples:
https://drive.google.com/file/d/1OJwuAgOTo9kA1N3on8gr-L9L-bVa5eam/view?usp=sharing

The source code for MobiclipDecoder would be useful for support for Mobiclip:
https://github.com/Gericom/MobiclipDecoder

Sources: https://wiki.multimedia.cx/index.php/Actimagine_Video_Codec

[richardpl]

See FFmpeg.

[bnnm]

I think it's possible to add but in the backburner due to some stuff.

[Sukotto-1999]

Maybe a demuxer for Mobiclip video is impossible, or support for Mobiclip audio and the FastAudio codec sourced from FFmpeg is i guess possible?

[bnnm]

Demuxing is possible but these formats are more complex than usual and ned to sort out various issues first. ffmpeg doesn't support all the variants though.

[modusc896d352]

ffmpeg has a mailing list where the developers exchange ideas about new oncoming developments to ffmpeg code, including new formats, fixes to existing stuff, etc.
if you have some mobiclip files that ffmpeg doesn't support, go to the mailing list and subscribe so you can send a message there, and get a thread going in ffmpeg-devel.

[Sukotto-1999]

okay, but how about making a .bms script would be possible to demux Mobiclip video but as the Mobiclip formats are EXTREMELY COMPLEX than usual, How about making a .txth for demuxed Mobiclip audio would be impossible.

Firstly, i found why. the process of playing mobiclip files is simple when used in NDS games: There are chunks of video data alongside the audio data at a common sample rate. The interleave for stereo Audio is used before the last chunk of audio is at the final interleave block.

[bnnm]

The problem with demuxing those files is the same as playing them. Format use rather complex bitstreams, so understanding them will take a while. Mainly moflex/mo, maybe mods is simpler

[Sukotto-1999]

Okay, so, after understanding complex bitstreams and demuxing mobiclip video, instead of implementing support for audio from Mobiclip video data, instead how about taking a look at the mobiclip parts of FFmpeg's source code for implementing those two codecs: MOFLEX 4-bit IMA ADPCM (https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200828214833.27650-1-onemda@gmail.com/) and Mobiclip FastAudio Codec (https://ffmpeg.org/doxygen/trunk/fastaudio_8c_source.html), figuring out the interleave and the blocked parts can be difficult. How about making a .TXTH file ready when making an impossible demuxer in quickbms .bms script unless support can be possible to add.

[Sukotto-1999]

Okay, you know what? I give up. i think the first part of the bitstream means video and the second is, of course audio. The audio for .mods with IMA-ADPCM audio means NDS_IMA (NDS-style 4-bit IMA ADPCM) or IMA 4-bit ADPCM. Demuxing complex bitstreams takes a while.

[Sukotto-1999]

I also found out that there was an early format made in 2004 called FV (FVDS) and was used only for Metroid Prime Hunters First Hunt Demo.

[Sukotto-1999]

Okay, continuing the discussion. I found more research.

Here's how source files are converted to mobiclip. Developers on the DS process source files, and then convert them to .avi with a special Mobiclip/ActImagine VFW video and audio codec, and then converted to Mobiclip format using a privately hosted tool (from WarioWorld dev site) by Nintendo. I found this in the manual of Mobiclip/Actimagine dev kit. We are not tolerating issues related to converting audio to properitary formats. We only tolerate issues related to demuxing and decoding audio.

and by the way, here are the first couple of bytes of a .VX file taken from Rune Factory DS:
56 58 44 53 ED 0A 00 00 00 01 00 00 C0 00 00 00
00 00 0F 00 18 00 00 00 80 BB 00 00 02 00 00 00
F4 34 00 00 78 97 87 00 E0 AF 87 00 01 00 00 00
3E 03 19 00 BB 70 58 9C 08 00 0B 23 01 00 61 04

At offset 0x16, the hex numbers are 0x0000BB80, which itself is actually the sample rate (48000)

And one more. We have a plan. How about a VGMToolbox style demuxer that removes video chunks from Mobiclip files, similar to demuxing .xmv or Mobiclip .mo formats, or a very-experimental .bms or .py script that may do the same thing

EDIT: I guess there's no way or evidence support can be added now. Mobiclip formats are more complex than Hudson Soft HVQM4 (.h4m) formats.

[Sukotto-1999]

Hi @bnnm. I have noticed, in the moflex videos from YKW demo on 3DS, which i actually included in my samples list. It turns out they do not have audio in it. Thanks.

[Sukotto-1999]

Update: Pleonex documented more of the Mobiclip for DS (not Actimagine VX/FV) specs, according to his project PlayMobic on Github. (https://www.pleonex.dev/PlayMobic/docs/specs/container-MODS.html)

Offset 0x18 is audio codec ID

Codec ids:
0: No audio
1: FastAudio/A3 V1 (with codebook)
2: FastAudio/A3 V2
3: NDS IMA-ADPCM
4: PCM16

Information should be useful, Also, the size of the audio blocks after each video frame bit-chunk is fixed for some encoders:

FastAudio is 0x28 bytes
IMA ADPCM is 0x84 bytes if keyframe otherwise is 0x80 bytes
PCM16 is 0x200 bytes

I'll find some sources later, should reply with a comment now.

Thanks @bnnm.

[bnnm]

I made a demuxer for moflex (3ds) here (see comments on top for extra info). Only playable via ffmpeg but probably can be added to vgmstream later. It was pretty complex but seemed used the most and I actually had enough samples.

As for others:

• .vx: probably simple but I don't think it's common (the single, tiny sample above doesn't even have audio).
• .mo: on a further inspection seems rather doable without bitstream and simple packet sizes, not sure if actually used much
• .mods: seems simple at first, but seemingly (from those linked docs) you need to read the full bitstream to parse audio = much harder

[Sukotto-1999]

it sure is! @bnnm

There you go. one almost down. 4 to go. VX, MODS, MO, and FV.

I'll add more samples with audio if i have time. Also, .moflex files created with Mobiclip Multicore Encoder don't work with the demuxer, maybe support should be added in the future.