wasapi + wine/alsa + surround51rear -> no sound from the rear speakers

[DirtYiCE]

I don't have audio in the back channels when trying to use openal-soft under wine. (The native linux version with alsa works fine). I'm trying this example: https://github.com/aib/openal-surround-test/ (originally I tried NFS3 with dsoal, but then I realized the problem is with openal-soft).

Possibly relevant part of the log (WINEDEBUG=trace+alsa ALSOFT_LOGLEVEL=3)

[ALSOFT] (II) Got message "Reset Device" (0x0001, this=021e5e88, param=00000000)
011c:trace:alsa:AUDDRV_GetAudioEndpoint {c74ae9e7-d379-4cab-bd07-fb1a5132db88} 0x462100 0x2bbfc78
011c:trace:alsa:get_alsa_name_by_guid Found matching device key: L"0,default"
011c:trace:alsa:AudioClient_AddRef (0x462a58) Refcount now 1
011c:trace:alsa:AudioClient_GetMixFormat (0x462a58)->(0x2bbfc7c)
011c:trace:alsa:dump_fmt wFormatTag: 0xfffe (WAVE_FORMAT_EXTENSIBLE)
011c:trace:alsa:dump_fmt nChannels: 2
011c:trace:alsa:dump_fmt nSamplesPerSec: 48000
011c:trace:alsa:dump_fmt nAvgBytesPerSec: 384000
011c:trace:alsa:dump_fmt nBlockAlign: 8
011c:trace:alsa:dump_fmt wBitsPerSample: 32
011c:trace:alsa:dump_fmt cbSize: 22
011c:trace:alsa:dump_fmt dwChannelMask: 00000003
011c:trace:alsa:dump_fmt Samples: 0020
011c:trace:alsa:dump_fmt SubFormat: {00000003-0000-0010-8000-00aa00389b71}
[ALSOFT] (II) Device mix format:
    FormatTag      = 0xfffe
    Channels       = 2
    SamplesPerSec  = 48000
    AvgBytesPerSec = 384000
    BlockAlign     = 8
    BitsPerSample  = 32
    Size           = 22
    Samples        = 32
    ChannelMask    = 0x3
    SubFormat      = {00000003-0000-0010-8000-00aa00389b71}
[ALSOFT] (II) Requesting playback format:
    FormatTag      = 0xfffe
    Channels       = 6
    SamplesPerSec  = 48000
    AvgBytesPerSec = 1152000
    BlockAlign     = 24
    BitsPerSample  = 32
    Size           = 22
    Samples        = 32
    ChannelMask    = 0x60f
    SubFormat      = {00000003-0000-0010-8000-00aa00389b71}
011c:trace:alsa:AudioClient_IsFormatSupported (0x462a58)->(0, 0x2bbfc88, 0x2bbfc7c)
011c:trace:alsa:dump_fmt wFormatTag: 0xfffe (WAVE_FORMAT_EXTENSIBLE)
011c:trace:alsa:dump_fmt nChannels: 6
011c:trace:alsa:dump_fmt nSamplesPerSec: 48000
011c:trace:alsa:dump_fmt nAvgBytesPerSec: 1152000
011c:trace:alsa:dump_fmt nBlockAlign: 24
011c:trace:alsa:dump_fmt wBitsPerSample: 32
011c:trace:alsa:dump_fmt cbSize: 22
011c:trace:alsa:dump_fmt dwChannelMask: 0000060f
011c:trace:alsa:dump_fmt Samples: 0020
011c:trace:alsa:dump_fmt SubFormat: {00000003-0000-0010-8000-00aa00389b71}
011c:trace:alsa:map_channels Mapping mmdevapi channel 0 (0x1) to ALSA channel 0
011c:trace:alsa:map_channels Mapping mmdevapi channel 1 (0x2) to ALSA channel 1
011c:trace:alsa:map_channels Mapping mmdevapi channel 2 (0x4) to ALSA channel 4
011c:trace:alsa:map_channels Mapping mmdevapi channel 3 (0x8) to ALSA channel 5
011c:trace:alsa:map_channels Mapping mmdevapi channel 4 (0x200) to ALSA channel 6
011c:trace:alsa:map_channels Mapping mmdevapi channel 5 (0x400) to ALSA channel 7
011c:trace:alsa:map_channels need_remapping: 1, alsa_channels: 8
011c:trace:alsa:AudioClient_IsFormatSupported returning: 00000000
011c:trace:alsa:AudioClient_Initialize (0x462a58)->(0, 40000, 927c0, 0, 0x2bbfc88, (null))
011c:trace:alsa:dump_fmt wFormatTag: 0xfffe (WAVE_FORMAT_EXTENSIBLE)
011c:trace:alsa:dump_fmt nChannels: 6
011c:trace:alsa:dump_fmt nSamplesPerSec: 48000
011c:trace:alsa:dump_fmt nAvgBytesPerSec: 1152000
011c:trace:alsa:dump_fmt nBlockAlign: 24
011c:trace:alsa:dump_fmt wBitsPerSample: 32
011c:trace:alsa:dump_fmt cbSize: 22
011c:trace:alsa:dump_fmt dwChannelMask: 0000060f
011c:trace:alsa:dump_fmt Samples: 0020
011c:trace:alsa:dump_fmt SubFormat: {00000003-0000-0010-8000-00aa00389b71}
011c:trace:alsa:map_channels Mapping mmdevapi channel 0 (0x1) to ALSA channel 0
011c:trace:alsa:map_channels Mapping mmdevapi channel 1 (0x2) to ALSA channel 1
011c:trace:alsa:map_channels Mapping mmdevapi channel 2 (0x4) to ALSA channel 4
011c:trace:alsa:map_channels Mapping mmdevapi channel 3 (0x8) to ALSA channel 5
011c:trace:alsa:map_channels Mapping mmdevapi channel 4 (0x200) to ALSA channel 6
011c:trace:alsa:map_channels Mapping mmdevapi channel 5 (0x400) to ALSA channel 7
011c:trace:alsa:map_channels need_remapping: 1, alsa_channels: 8
011c:trace:alsa:AudioClient_Initialize ALSA period: 512 frames
011c:trace:alsa:AudioClient_Initialize ALSA buffer: 2048 frames
011c:trace:alsa:AudioClient_Initialize MMDevice period: 480 frames
011c:trace:alsa:AudioClient_Initialize MMDevice buffer: 2880 frames
011c:trace:alsa:AudioClient_GetDevicePeriod (0x462a58)->(0x2bbfcb0, (nil))
011c:trace:alsa:AudioClient_GetBufferSize (0x462a58)->(0x2bbfc80)
011c:trace:alsa:AudioClient_SetEventHandle (0x462a58)->(0x9c)
[ALSOFT] (II) Post-reset: 5.1 Surround, Float32, 48000hz, 960 / 2880 buffer

Openal-soft request channel-format 0x60f (which has SPEAKER_SIDE_LEFT/RIGHT instead of SPEAKER_BACK_LEFT/RIGHT) which wine maps to channel 6-7 https://github.com/wine-mirror/wine/blob/3f4f116dc52c2d37b98e62c3d9bdd8a79e44ccc1/dlls/winealsa.drv/alsa.c#L686 which makes sense with 7.1 audio, but I only have 6 channels...
If I'm right, this isRear51 variable gets initialized based on the Device mix format part in the above dump, but that only reports a stereo device (not sure whether this is a wine bug, or it's a feature).

openal-soft/alc/backends/wasapi.cpp

Line 1700 in 23cc00e

bool isRear51{false};

Anyway, later it looks like it determines the channel mask based on this, and completely ignores whether I have channels = surround51 or channels = surround51rear in the config.

As a workaround I can set channels = surround71 in the config, then I get audio from the back speakers, but that also means the audio for the side speakers go into /dev/null...

[kcat]

It's a combination of issues. Ultimately the problem is various software disagree on what "5.1" is. In particular, the non-front/lfe channels are labelled "surround" in the actual specifications, and on PCs, they were originally given the labels "back" or "rear" in various APIs. But at some point, people realized the "back"/"rear" naming was improper and "side" was more appropriate (which also offered better compatibility, since if you're playing 5.1 content on a 7.1 system, you want the 5.1 "surround" signals to go to the 7.1 side speakers).

Windows added a second 5.1 mode that used the side labels instead of the back labels, and started updating their APIs and stuff to prefer using them when possible. ALSA, however, uses raw channel numbers that are assumed to be in a particular order for particular modes, with stereo, quad, 5.1, and 7.1 building off of each other, so can't just change the order of the 5.1 channels when setting the device to "6 channels".

OpenAL Soft used to have an explicit surround51rear channel configuration, though it was troublesome to manage since it would confuse itself (e.g. what labels should AL_FORMAT_51CHN* buffer formats feed to, how to handle 5.1 decoders). So surround51rear and surround51 were combined, the 5.1 surround channels are always mapped internally to side channels, and it's left to each of the backends to work out whether to map OpenAL Soft's side channels to the side or rear outputs for 5.1. This is why native ALSA still works, since the ALSA backend knows what order the channels need to be in for 5.1 output.

For its part, native WASAPI is pretty strict with playback streams matching the device format. Without any APOs installed to work differently, a WASAPI playback stream has to have the same channel configuration as the device, so 5.1 output will only be used with a device configured for 5.1 or better, where OpenAL Soft is able to detect if it's using the side or rear channels and behave appropriately. Wine, however, will allow streams with other channel configurations that it maps directly to ALSA channels, causing the app to be at the mercy of ALSA behavior.

Given the circumstances, ideally winealsa would detect if a 5.1side stream is being requested and transparently behave as if it was 5.1rear (the actual channel order from the app is the same), since it's more inline with how Windows behaves. It might also be possible to explicitly set the device to 5.1 Surround in winecfg's Audio tab, which would make WASAPI report the expected channel configuration. Though I'm not sure if that option is available or would be honored with winealsa.drv.

[DirtYiCE]

It might also be possible to explicitly set the device to 5.1 Surround in winecfg's Audio tab, which would make WASAPI report the expected channel configuration.

I already did that, but no effect (though I'm practically stuck at wine 6.19, since with newer versions the app I want to use doesn't work...)

Even though my rear speakers is at the rear, not at the side. Should I instead switch to 7.1 and simply ignore the side channels?

[kcat]

Depending on how far back you mean by "rear" (the specification for the surround speakers is between +/-90 and +/-110 degrees), if they're farther back with the lefts and rights creating more of a square shape, you could instead use channels=quad. Otherwise, I'll need to add an option to make 5.1 use rear channels when it would otherwise use the side channels.

[DirtYiCE]

It's more like +-150 degrees honestly, making it more of a rectangle than a square. Anyway, in the meantime I played around with that adt tool mentioned in the ambdec documentation (man, I really hoped that after finishing uni I'll never have to touch that matlab crap again, sigh) and generated something that may or may not match my speaker layout. Combined with an alsa workaround, it works now, even if not exactly ideal:

pcm.wine51fix {
    @args [ SLAVE ]
    @args.SLAVE {
        type string
        default dmix6
    }
    type route
    slave.pcm $SLAVE
    slave.channels 6

    ttable.0.0 1
    ttable.1.1 1
    ttable.6.2 1
    ttable.7.3 1
    ttable.4.4 1
    ttable.5.5 1
}

I'm not sure how common this layout is, if it's some rare one we can probably leave it like this (even though I don't think I've ever seen a 5.1 layout in real life matching that spec...)

[kcat]

It's more like +-150 degrees honestly, making it more of a rectangle than a square.

You could use the rectangle.ambdec preset for that, assuming it was installed with your build or distribution. It won't use the front-center speaker for 3D sound panning, but it may actually be better for creating a continuous soundfield if it doesn't. Though you could also just stick with the default for 5.1 output, it shouldn't be any worse than what it's already like with other 5.1 content.

Anyway, in the meantime I played around with that adt tool mentioned in the ambdec documentation (man, I really hoped that after finishing uni I'll never have to touch that matlab crap again, sigh) and generated something that may or may not match my speaker layout.

One of these days, I'd like to make a simpler tool for generating decoders from a user-specified layout. I actually have the basis for such a tool, but it has a dependency on the lapack library (in particular, LAPACKE_dgesvd to get a matrix's singular value decomposition, and LAPACKE_dgeev to get the eigenvalues; I'd need to make replacements for that functionality to avoid the dependency). Though even with a simpler interface it can still be tricky to properly generate a decoder, since there's different methods to generate them with their own pros and cons.