jbuf: adaptive playout time calculation (Work in progress)

[sreimers]

Experimental playout time calculation for jitter buffer. It replaces frame calculation, since it uses a local time reference offset (based on clock rate).

TODO:

• Add unit tests
• Introduce empty frames (if packet loss is detected a empty frame should scheduled as replacement)
• Use tmr decode to prevent jbuf overflow on jitter spike (jbuf needs constant offloading)
• Compare with current solution and jbuf frame completeness #2754
• Cleanup/refactoring
• Testing WebRTC

It's based on suggestions from this book: https://csperkins.org/standards/rtp-book.html

Future improvements:

• Maybe the spike/jitter algorithm can be improved (we can test with different modes JBUF_ADAPTIVE_SPIKE...)
• Currently the jitter and clock skew from soundcard (auplay) is ignored, at least cheap cards/drivers maybe perform badly. A tick based offloading could help to calculate within jbuf. But good skew adaption needs samplerate conversion instead of removing/adding complete frames. Maybe we can use a audio filter module with libsamplerate.

[cspiel1]

Frame completeness is not checked. Is it already ensured by skew and jitter?

[sreimers]

Yes, it's ensured by jitter compensation, it's now a time based buffering delay, this should give the lowest delay which is needed to buffer.

[cspiel1]

okay, we can compare the plots if you are ready.

[sreimers]

Sure, give me some time to prepare and ensure the algorithms are working correctly (wrap-around calculations are tough ;-))

[cspiel1]

When jitter and/or skew reduces, then jbuf should return EAGAIN --> seems to be done already in this branch.

Then aubuf will increase number of buffered audio frames. Since jbuf already does the jitter compensation and packet re-ordering these extra frames can be dropped from aubuf. Maybe immediately dropped, or after some short time. This should reduce the latency.

Threads involved for audio:
Thread 1 - main thread, or the RX thread in near future

• jbuf_put() is called
• jbuf_get(), maybe by a timer. Or maybe by a wait condition (mqueue) from Thread 2.
• does also decoding
• pushes to aubuf

Thread 2 - audio playback thread

• only reads the audio frames from aubuf

Edit: Your PR looks promising for me. I also think that the jitter compensation should be done in jbuf. This will reduce the overall latency because jbuf is able to handle jitter and out-of-order packets.

But unit tests will become difficult or have less coverage because jbuf_get() will depend on timing.

[sreimers]

• jbuf_get(), maybe by a timer. Or maybe by a wait condition (mqueue) from Thread 2.

I implemented a fallback tmr solution here (fallback since we should avoid to many event loop wakeups): 80b2758

[sreimers]

But unit tests will become difficult or have less coverage because jbuf_get() will depend on timing.

I think we can use a virtual clock rate and small frames for tests. But yes this increases test time, at least for some milliseconds.
Edit: Or we can try to mockup tmr_jiffies somehow.

[cspiel1]

But unit tests will become difficult or have less coverage because jbuf_get() will depend on timing.

I think we can use a virtual clock rate and small frames for tests. But yes this increases test time, at least for some milliseconds. Edit: Or we can try to mockup tmr_jiffies somehow.

Okay, let's try this!

Also we could try a from jbuf_put() completely independent timer. E.g. ptime triggered.

[alfredh]

please resolve the conflicts ..

[alfredh]

what is the status of this PR ?

Can you please rebase it on top of GIT HEAD ?

[cspiel1]

I think we should prefer this approach over mine #2754.

Currently, studying the mentioned book.

[sreimers]

I will try to complete/cleanup this work soon. Maybe later this week.

[cspiel1]

I'll try to fix the sanitizer warning. I think it is not related to this PR.

[cspiel1]

PR for sanitizer warning: #2867

[sreimers]

The failing valgrind looks also unrelated: #2873