Ignore missing DASH media segments in a live stream

[alexg-axis]

When playing a DASH live stream, the player will stop at missing video segments and retry the download before failing. Due to network latencies or other issues, segments may be available in an incorrect order or not be available at all.

Instead of stopping, some use cases would benefit from continuing to play whatever segments are in the pipeline and simply show a black image when a video segment is missing or "play" silence if an audio segment is missing. That is, if a 1s video segment is missing, 1s of empty video should be shown.

A current workaround is to implement a custom segment loader and return an empty ArrayBuffer upon receiving 404s from the content server. This makes the player skip at least that segment until the buffer contains enough media. How much is skipped does not always seem to be deterministic, however. Furthermore, the stutter and inability to play audio if it's available, is not preferable.

I tried to identify better workarounds, partly by looking at how #695 is done, which allows audio to be played without video. I also looked at ways to make the segment inventory identify missing segments as taking up time without having any content. However, I failed to find any feasible workaround.

Are there existing workarounds today, or would it require new features?

[peaBerberian]

Hi,

I'm unsure of what should be done on the Manifest-side when segments are skipped, but I would assume that the Manifest should not announce non-existent segments.
In that situation, the RxPlayer would just skip the resulting holes in the content.

---

With that said, I understand that such situations can still happen and you may want to totally skip a segments if many consecutive requests for it fails with the same error.
However just doing that may lead to some unwanted edge cases: if the server really has issues and is returning 404 for all segments, we risk skipping multiple segments after everything return in order.
So the optimal algorithm may be complex: for example checking the next segment and re-doing the first one if it also fails. For consecutive groups of segments failing, I'm unsure what the right approach would be.
Moreover, other applications might prefer to never skip any segment, so even at the API level, it may be complicated to please everyone, maybe through another option?
How this logic would be influenced by a non-Infinity networkConfig.segmentRetry configuration is also unclear to me, as in that case, the application wants to fail after a set number of retries.

Anyway, to properly skip the segment, I guess instead of relying on the SegmentInventory by adding to it [fake] empty segments, it may be more straightforward to include another concept directly linked to the fetching logic instead (the SegmentInventory being linked to the buffers - it just lists which segments are currently buffered) which would signal in some way which segments are not downloadable.
The RepresentationStream, which is the part of the code deciding which segments to download, would then rely on that list to prevent requesting them again and announcing them as discontinuities to skip.

---

As for displaying a black screen or a silenced audio instead of skipping it, I'm not sure that this is easily doable without disabling the corresponding SourceBuffer which leads to a temporary black screen (while the RxPlayer is in the RELOADING state).

That is how audio-only works:

1. the previous MediaSource (on which the content is buffered) is removed. A black screen begin to be displayed
2. A new MediaSource is created and new buffers are associated to it, for now empty.
3. As media data begin to be loaded and pushed, content can restart / the frame can be shown etc.

And this would have to happen before and after the segment skip (to remove and re-add the buffer). An alternative could be to push data corresponding to a black screen for video and to silence for audio, but this may not be straightforward as we would need to generate segments for this for all potential codecs/profiles we encounter to ensure compatibility.
Another alternative would be for the browser to come with an API to allow filling the buffer in that way, but I don't know if it exists.