Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use two separate RtAudio streams when input != output device #1235

Merged
merged 10 commits into from Jan 15, 2024

Conversation

mikedickey
Copy link
Collaborator

RtAudio supports the creation of streams that are input-only, output-only, or input + output (duplex). Previously, we always used a single duplex stream even when the input and output devices differ. This is problematic because each device may have its own clock, and trigger callbacks at different points in time.

RtAudio therefore needed to wait for both audio interfaces to trigger callbacks before it could trigger its own, reducing the interval of time that JackTrip has to perform its work. If the delta between the interface callbacks is small enough, relative to the work JackTrip is performing, everything work just fine. However, over time we've added more CPU intensive work in the JackTrip callbacks, such as PLC. If the delta is too short, callbacks get missed causing audio glitches.

By using separate (simplex) RtAudio streams for input and output, we are able to decouple the two audio interfaces from one another, ensuring that JackTrip always has the maximum amount of time available to "do its thing."

RtAudio supports the creation of streams that are input-only,
output-only, or input + output (duplex). Previously, we always
used a single duplex stream even when the input and output
devices differ. This is problematic because each device may
have its own clock, and trigger callbacks at different points
in time.

RtAudio therefore needed to wait for both audio interfaces to
trigger callbacks before it could trigger its own, reducing
the interval of time that JackTrip has to perform its work.
If the delta between the interface callbacks is small enough,
relative to the work JackTrip is performing, everything work
just fine. However, over time we've added more CPU intensive
work in the JackTrip callbacks, such as PLC. If the delta is
too short, callbacks get missed causing audio glitches.

By using separate (simplex) RtAudio streams for input and
output, we are able to decouple the two audio interfaces from
one another, ensuring that JackTrip always has the maximum
amount of time available to "do its thing."
@mikedickey
Copy link
Collaborator Author

Note that this requires and incorporates a patch for an RtAudio bug as described here thestk/rtaudio#417

on first call to AudioInterface::audioOutputCallback.

When using two different devices, the input device could start
triggering callbacks before the output device. This can cause
several pushes into the monitor queue before the output callback
performs its first pop. The net result is random variable latency
added to the monitor signal.

Draining the queue ensures that the two will also be as in sync
as possible.

Note that for duplex devices, the queue would never grow larger
than a size of 1, and drop down to zero before the audio callback
returns.
@mikedickey
Copy link
Collaborator Author

@cchafe note that this appears to solve a problem with bufstrategy 4 (PLC in callback, or no worker thread), where using two different audio interfaces on OSX would cause audio artifacts. With this, I think we can finally do away with the extra queue & worker thread (bufstrategy 3), shaving off an extra buffer or more of latency.

@mikedickey mikedickey force-pushed the bugfix/simplex-rtaudio branch 12 times, most recently from 445d1f7 to 2776d09 Compare January 11, 2024 23:02
Since the pip version uses outdated version of meson where diff_files is broken
@mikedickey mikedickey marked this pull request as ready for review January 11, 2024 23:21
@cchafe
Copy link
Collaborator

cchafe commented Jan 12, 2024

very cool news!

@mikedickey mikedickey force-pushed the bugfix/simplex-rtaudio branch 2 times, most recently from c7aeef5 to 6baacac Compare January 13, 2024 00:45
@nwang92
Copy link
Contributor

nwang92 commented Jan 13, 2024

There's a few times when opening the app built from this branch where the "Play Test Tone" button doesn't seem to do anything the first time I click it. If I click it again, it plays as expected. I'm not sure if it's because there's still some setup stuff going on but I haven't been able to consistently reproduce this.

Also there are some times when I get this RtAudio error when I change the number of input channels (typically when going from 1 channel, either 1 or 2, to 2 channels):
Screenshot 2024-01-13 at 2 07 47 PM

@mikedickey
Copy link
Collaborator Author

There's a few times when opening the app built from this branch where the "Play Test Tone" button doesn't seem to do anything the first time I click it. If I click it again, it plays as expected. I'm not sure if it's because there's still some setup stuff going on but I haven't been able to consistently reproduce this.

Also there are some times when I get this RtAudio error when I change the number of input channels (typically when going from 1 channel, either 1 or 2, to 2 channels):

It sounds like your build has the rtaudio bug I had to fix. Wipe out your subprojects/rtaudio-6.0.1 directory, your builddir directory, and re-run meson setup & compile. You can also look at the RtAudio.cpp file to verify that the patch was applied by meson.

@mikedickey mikedickey merged commit 3c5e72f into dev Jan 15, 2024
15 checks passed
@mikedickey mikedickey deleted the bugfix/simplex-rtaudio branch January 15, 2024 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants