Avoiding stalls (allocations) in the audio thread

[Boscop]

The Plugin::process() method is called in the audio thread of the host which should never do anything that stalls, like allocation, page faults etc. to avoid buffer dropouts which can destroy speakers at loud volume and are bad in general (you can't expect people to use your plugin if it even has the potential to stall in certain cases).

So we should eliminate the dynamic allocation of midi note memory for processing midi events.
(Maybe there are more potentials of stalls in methods that the host calls in the audio thread.)
(interfaces::process_events() is called by both hosts and plugins.)

Apparently most developers pre-allocate a generous fixed size array for the midi input and only re-allocate when the size doesn't suffice. (Or truncate midi events per frame to that generous limit.)

We could do this by passing a &mut Vec to process_events(), which the caller (plugin and host) pre-allocates to a generous size outside of the audio thread.
Or do you have a better idea?

(The init and reset opcodes are supposed to be for allocations.)

[overdrivenpotato]

Good catch, we can just use a lazy static Mutex<Vec>. Ideally that way, it would only reallocate as needed, calling .clear() to reset the Vec. Any suggestions for what we should set the default size to?

[zyvitski]

Going with that solution I would opt for a buffer size somewhere in the range of 512-2048 depending on the amount of traffic expected.

I have some concern as to whether using a dynamically resizable collection is a good idea even if allocations are reduced to a minimum. This scenario seems like it could be better suited to using a fixed length circular buffer (With a similar size as suggested above). I would prefer an implementation that would be more likely to drop a midi message due to a full message queue than an implementation that would attempt to dynamically allocate memory and potentially stall execution.

Assuming a fixed length circular buffer is used, we could potentially set it up to where the length of the buffer can be specified by the plugin allowing the plugin designer to balance the amount of memory used against the possibility of dropping midi messages.

If it is agreed that a circular buffer could be the way to go I would be glad to find/implement the needed collection in order to make this work.

[Boscop]

Should it be a Mutex because static variables are shared between all instances of a dll and because this could cause problems when the host uses multi threading when two instances access the same static variable at the same time?
But a Mutex has overhead too and can also stall the thread, and can get poisoned...

[zyvitski]

Ideally it would be a Lock Free Implementation of some kind.

[Boscop]

I agree that a fixed size buffer would be preferable, but it doesn't have to be a ring buffer, just a Vec allocated during init().
I think we can put an acceptable upper limit on 2048 midi events per frame, but that has to be multiplied by the midi event size.

[zyvitski]

In that case would a Vec actually be the correct data structure to use? what advantages would we gain from using a Vec that is supposed to be fixed sized over a fixed size array? Looking at how the Current implementation is using the Vec it seems that all is needed is the ability to iterate over the collection, so a fixed length array would do the job if the size is fixed at compile time. This would also ensure that no allocation could happen at runtime.

[zyvitski]

An article worth reading on the topic: http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing

[Boscop]

yeah we could use ArrayVec

[zyvitski]

I think that would be a good choice.

[Boscop]

There are also allocations in interfaces::read_string() and the collect()s in AudioBuffer.

The string length is bounded so we can just preallocate that mem, but more important is removing the allocs in the audio thread. I think we can assume that the number of input and output channels will be constant over time, less than or equal (maybe always equal) to the number of inputs and outputs that the plugin reported to the host (until it changes the number via setNumInputs / setNumOutputs).
So we can also prealloc the Vecs in AudioBuffer and resize it only when the number of inputs/outputs changes (when the plugin is paused).

[zyvitski]

Would it be a good idea to allow the plugin to request additional buffers for intermediate usage within the plugin? Adding that could allow the plugin to handle complex processing situations without having to come up with a way to allocate the memory on their own. Additionally we could see some possible performance benefits if we were to attempt to keep all of the buffer as one large sliced up array(could theoretically improve cache locality and similar issues).

[Boscop]

Plugins allocate memory for their specific needs in their init() method.

[zyvitski]

I know that, my thought was wether having all of the luging buffers localized to one block could be beneficial or not?

[Boscop]

It depends on how to buffers are used.
Putting them all together when they are used in different contexts doesn't improve cache locality..
So it's really specific to each plugin..

[zyvitski]

Okay, I figured it was at least worth proposing.