How to continuously record audio using OpenAL?

[Swimburger]

Description

I'm not too familiar with OpenAL and audio in general, so I'm a little out of my depth here.
I'm trying to capture continuous audio from a microphone using .NET across Windows/Mac/Linux.
I looked at ALTest and tried to modify it to continuously listen to audio, but I don't know how I should modify it for that purpose.

I also found a lot of the examples online using the AudioCapture class, but it isn't included in the NuGet package.

Here's what I got:

const int sampleRate = 16_000;
var cts = new CancellationTokenSource();
Task.Run(async () =>
{
    var buffer = new byte[sampleRate];
    var captureDevice = ALC.CaptureOpenDevice(null, sampleRate, ALFormat.Mono16, sampleRate);
    {
        ALC.CaptureStart(captureDevice);

        while (!cts.IsCancellationRequested)
        {
            ALC.CaptureSamples(captureDevice, ref buffer[sampleRate], sampleRate);
            processBuffer(buffer);
        }

        Console.WriteLine("Press any key to exit.");
        Console.ReadKey();

        Console.WriteLine("Stopping recording");
        ALC.CaptureStop(captureDevice);
    }
}, cts.Token);

Console.WriteLine("Press any key to exit.");
Console.ReadKey();
await cts.CancelAsync();

Inside of processBuffer I'm sending the audio to a transcription engine.

Related information

• Operating system: Mac
• SDL or Native backend (if using OpenTK 3.x): Not sure
• What version of OpenTK (Installer, NuGet, Source Code): NuGet
• What runtime (.NET Framework, CoreCLR, Mono): CoreCLR

If you could point me to a good resource, that'd be greatly appreciated! 🙇

[NogginBops]

It's unclear here what the actual problem is here. What is the code above do when run? What do you expect to happen and what actually happens?

[Swimburger]

When I send the audio to the transcriber engine it should return me with partial transcripts, but currently, it is stuck on line ALC.CaptureSamples
Ideally, the transcriber needs 100+ ms of audio in 16000 sample rate, continuously (in 16-bit signed integer PCM).

[NogginBops]

Ok so I'm seeing a few issues.

One is that you are not waiting for new samples to become available, you are reading them as fast as you can so most of the time you are only getting a few samples.
Here in the ALTest code is the part that waits for new samples: https://github.com/opentk/opentk/blob/master/tests/OpenALTest/ALTest.cs#L79-L90
Importantly you can note the if statement checking the number of available samples.

Another issue is that you are writing to the same part of the buffer every time. ref buffer[sampleRate] is also an index out of bounds issue. You want to index into a party of the buffer you want to write to.

Something not part of the code you are showing that could cause a potential issue is how you are handling OpenAL contexts (I don't remember if you need a context to record audio). A context is thread specific so running from a task could need you to make the context current from within the task.

[Swimburger]

Sorry for the delay. Thank you for your feedback.

I am now getting transcripts back, although less accurate than usual. I'd say 20% less accurate than I'm used to.
I still think something is missing causing the degraded accuracy.

This is what the code looks like now:

const int sampleRate = 16_000;
const int bufferSize = 1024;
var captureDevice = ALC.CaptureOpenDevice(null, sampleRate, ALFormat.Mono16, bufferSize);
ALC.CaptureStart(captureDevice);

var buffer = new byte[bufferSize];
while (true)
{
    var current = 0;
    while (current < buffer.Length && !ct.IsCancellationRequested)
    {
        var samplesAvailable = ALC.GetInteger(captureDevice, AlcGetInteger.CaptureSamples);
        if (samplesAvailable < 512) continue;
        var samplesToRead = Math.Min(samplesAvailable, buffer.Length - current);
        ALC.CaptureSamples(captureDevice, ref buffer[current], samplesToRead);
        current += samplesToRead;
    }

    if (ct.IsCancellationRequested) break;

    // copy  buffer so that it can be used in another thread
    transcribeAudio(buffer.ToArray());
}

This code is running in an async function, another async function cancels the cancellation token when a key is pressed.

[Swimburger]

I'm also frequently getting this error: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
I'm assuming this has to do with the thread context issues. The code that I'm currently using only uses one thread without tasks. I'm also copying the buffer filled by ALC to a separate buffer.

[NogginBops]

Any updates on this issue?
It does sounds like you might have some threading issues, alternatively issues with async.

[Swimburger]

Unfortunately, I didn't get anywhere after this roadblock. It mostly works, but crashes after a short period, between 30s to multiple minutes.