MPEG-DASH manifest size grows like avalanche

[alex-rsk]

GPAC version 2.0-rev2.0.0+dfsg1-2

We are making stream, consequently feeding MP4Box with new video-fragments

MP4Box -dash 10000 -dash-ctx ./stream/context.txt  -rap   -profile dashavc264:live  \
-mpd-refresh 6 -dynamic -bs-switching no \
 -segment-timeline -url-template -time-shift 5 \
./chunks/segment_0001.mp4#video ./chunks/segment_0001.mp4#audio -out ./stream/live.mpd 

Everything is working good, except that fact, that MPD manifest grows very quickly.
Hour after it weighs 9 megabytes, which causes video freeze.
On the other hand, when I use Nginx to stream MPD, it keeps manifest size about 25 kilobytes.

Currently I don't see any workaround, except cleaning it from time to time. But, perhaps, there is some option to keep size in reasonable borders?

[rbouqueau]

Your command-line asks for 10s segments, a MPD refresh every 6s and a time-shift buffer depth of 5s. The units may not be the one you expected.

[rbouqueau]

I've tried to reproduce to no avail:

$ gpac avgen:dur=2 @#video c=avc -o v.mp
$ MP4Box -dash 2000 -dash-ctx context.txt  -rap   -profile dashavc264:live  \
-mpd-refresh 10 -dynamic -bs-switching no -segment-timeline -url-template -time-shift 10 v.mp4 -out ./stream/live.mpd
$ ...
$ $ MP4Box -dash 2000 -dash-ctx context.txt  -rap   -profile dashavc264:live  \
-mpd-refresh 10 -dynamic -bs-switching no -segment-timeline -url-template -time-shift 10 v.mp4 -out ./stream/live.mpd

And I end up with this manifest (which seems fine to me):

<?xml version="1.0"?>
<!-- MPD file Generated with GPAC version 2.3-DEV-rev1031-g5d7731c4b-master at 2024-04-11T16:06:20.622Z -->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" minBufferTime="PT1.500S" type="dynamic" availabilityStartTime="2024-04-11T16:05:49.777Z" publishTime="2024-04-11T16:06:20.622Z" minimumUpdatePeriod="PT0H0M10.000S" timeShiftBufferDepth="PT0H0M10.000S" maxSegmentDuration="PT0H0M2.000S" profiles="urn:mpeg:dash:profile:isoff-live:2011,http://dashif.org/guidelines/dash264">
 <ProgramInformation moreInformationURL="https://gpac.io">
  <Title>live.mpd generated by GPAC</Title>
 </ProgramInformation>

 <Period id="DID1" start="PT0H0M0.000S">
  <AdaptationSet segmentAlignment="true" maxWidth="1280" maxHeight="720" maxFrameRate="25" par="16:9" startWithSAP="1">
   <SegmentTemplate media="v_dash$Time$.m4s" initialization="v_dashinit.mp4" timescale="25" startNumber="1">
    <SegmentTimeline>
     <S t="450" d="50" r="5"/>
    </SegmentTimeline>
   </SegmentTemplate>
   <Representation id="1" mimeType="video/mp4" codecs="avc1.64001F" width="1280" height="720" frameRate="25" sar="1:1" bandwidth="1022360">
   </Representation>
  </AdaptationSet>
 </Period>
</MPD>

Note that I'm using the latest GPAC. Would you consider updating and retrying? If you still see the issue, could you try the same as mine?

[alex-rsk]

Thank you for response.
I have tested with latest GPAC, and nothing changed.

Then I thoroughly checked my command-line params, and it seems I found the reason.

It seems it was a bug in code, when I set time-shift to the same value as dash (i.e. milliseconds instead of seconds).

Also I have aligned mpd-refresh and dash.

I started the stream and left it for 8 hours.

After that, context.txt was about 100 kbytes, and manifest.mpd weighed 44 kbytes.

I tend to think, that problem is solved now. We continue to check manifest size, though.

Thank you very much!

UPD: No, sadly, files continue growing, I just didn't noticed that my colleaugue enabled auto clean again... (

[alex-rsk]

Just found, that, if I rename new input segment to the same name, e.g. "input.mp4" - then it generates small manifests and it doesn't grow.
But such manifest not works in DASH IF player.

Example:
I called :

MP4Box ..../chunks/segment_0001.mp4#video ./chunks/segment_0001.mp4#audio -out ./stream/live.mpd
MP4Box ..../chunks/segment_0002.mp4#video ./chunks/segment_0002.mp4#audio -out ./stream/live.mpd
MP4Box ..../chunks/segment_0003.mp4#video ./chunks/segment_0003.mp4#audio -out ./stream/live.mpd

^ Works but grows.

 mv segment_0001.mp4 input.mp4
 MP4Box ..../chunks/input.mp4#video ./chunks/input.mp4#audio -out ./stream/live.mpd
 
  mv segment_0002.mp4 input.mp4
 MP4Box ..../chunks/input.mp4#video ./chunks/input.mp4#audio -out ./stream/live.mpd

^ Doesn't grow, but doesn't play also!

Any help would be appreciated!

[alex-rsk]

Managed it to work playlist through gpac:

gpac -i playlist.m3u -o live.mpd:profile=live:dmode=dynamic:sreg:stl

Is there option in MP4Box that makes "sreg" like in gpac?

[jeanlf]

Is there option in MP4Box that makes "sreg" like in gpac?

Yes, just set sreg on your target MPD file, eg
MP4Box -dash N [srcs] -dynamic -out live.mpd:sreg

[alex-rsk]

This worked out partially.

MP4Box -dash 10000 -dash-ctx ./stream/context.txt -rap -profile live -mpd-refresh 5 -bs-switching no -segment-timeline -url-template -min-buffer 1000 -time-shift 1800 input.mp4#video -dynamic -out ./stream/stream.mpd:sreg

It works! But only with video track.

If I add sound track:

MP4Box -dash 10000 -dash-ctx ./stream/context.txt -rap -profile live -mpd-refresh 5 -bs-switching no -segment-timeline -url-template -min-buffer 1000 -time-shift 1800 
input.mp4#audio  input.mp4#video -dynamic -out ./stream/stream.mpd:sreg

Then stream doesn't play (more accurately, plays only 1st chunk). It seems that there's some bug, b.c. gpac streams both tracks

In second case, i''m also getting error messages:

                                                                                                                                      
2024/04/17 19:40:29 Retry error:  exit status 1
2024/04/17 19:40:29 RETRY OUTPUT [Dasher] No template assigned, using $File$_dash$FS$$Time$
[Dasher] updated period DID1 duration 0 MPD time 0                                                                                    
[MPD] Generating MPD at time 2024-04-17T16:40:29.845Z                                                                                 
[Dasher] End of Period DID1                                                                                                           
[MPD] Generating MPD at time 2024-04-17T16:40:29.846Z                                                                                 
[Dasher] AS-1 Rep 2 segment 1 done TOO LATE by 300 ms9.56s 95 %                                                                       
[Dasher] updated period DID3 duration 10000 MPD time 10000                                                                            
[MPD] Generating MPD at time 2024-04-17T16:40:29.852Z                                                                                 
[Dasher] updated period DID3 duration 10000 MPD time 10000                                                                            
[MPD] Generating MPD at time 2024-04-17T16:40:29.857Z                                                                                 
[Filter] dasher not responding properly: 100000 consecutive process with no packet discarded or sent, but 1 packets pending           
        discarding all inputs and notifying end of stream on all outputs                                                              
Error DASHing file: Internal Service Error                                                                                            
                                                                                                                                      
2024/04/17 19:40:39 SEGMENT PROCESSED 2 2
MP4Box -dash 10000 -dash-ctx ./stream/context.txt -rap -profile live -mpd-refresh 5 -bs-switching no -segment-timeline -url-template -min-buffer 1000 -time-shift 1800 ./chunks/input.mp4#audio ./chunks/input.mp4#video -dynamic -out ./stream/stream.mpd:sreg
2024/04/17 19:40:40 Mp4box err signal: aborted (core dumped) [Dasher] No template assigned, using $File$_dash$FS$$Time$
MP4Box: filters/dasher.c:5128: dasher_reload_context: Assertion `i+1==nb_p' failed.                                                   

```

[alex-rsk]

Any suggestions?

[alex-rsk]

gpac with dynamic playlist also doesn't work properly.
It just hangs when m3u file updated from outside.

Honestly , I have tested this tool during this year in different flavours and nothing works properly, it seems it is time to move ffmpeg. (

[rbouqueau]

I can understand how frustrating unresolved issues can be. Please continue to open as many issues as needed.

As you can notice GPAC has only 36 opened issues. It shows how committed we are to solving issues when you compare with other projects.

GPAC is run by volunteers during their free time. We just lack free time, which happens. Either be patient or look for professional services (https://motionspell.com/gpac-licensing/) if that's urgent.

[alex-rsk]

Thank you guys for your work, anyways, it is impressive and outstanding.

Its's sad that it doesn't suits my needs fully.

Actually, at this moment, it works somehow, with growing manifest file. Without growing it works with single video track only; adding audio representation causes dash players' malfunction.