Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turn --hls-duration into a generic segmented-stream CLI arg and add support for DASH durations #5450

Open
3 tasks done
slhck opened this issue Jul 18, 2023 · 14 comments · May be fixed by #5601
Open
3 tasks done

Turn --hls-duration into a generic segmented-stream CLI arg and add support for DASH durations #5450

slhck opened this issue Jul 18, 2023 · 14 comments · May be fixed by #5601

Comments

@slhck
Copy link
Contributor

slhck commented Jul 18, 2023

Checklist

Description

Currently the CLI runs indefinitely until an error occurs or the user terminates the command by pressing Ctrl-C (sending SIGINT).

It would be beneficial, for automated/non-interactive use, to have an option to quit after a specified amount of time, which is particularly useful for never-ending live streams.

This could be an option called --stream-duration or --quit-after, taking a value in seconds after which the program cleanly exits, once a stream has been opened and read from.

The only viable workaround would be to run something like this, although it is not precise, since the overall timeout includes the time to load the stream as well:

import signal
import subprocess
import time

process = subprocess.Popen(
    [
        "streamlink",
        "http://example.com/stream.mpd",
        "best",
        "-o",
        "tmp.mkv",
    ],
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
)

time.sleep(10)  # Wait for 10 seconds
process.send_signal(signal.SIGINT)  # Send SIGINT signal to stop the process
stdout, stderr = process.communicate()
print(stdout)

Of course, this could be implemented using the API, but it would require re-doing everything the CLI is doing anyway.

@bastimeyer
Copy link
Member

The only viable workaround

timeout --signal=INT --kill-after=10 --preserve-status 60 streamlink ...

https://www.gnu.org/software/coreutils/manual/html_node/timeout-invocation.html

@slhck
Copy link
Contributor Author

slhck commented Jul 18, 2023

… when running from within a Python script, I meant. But yes, that is a more general solution of course!

@bastimeyer
Copy link
Member

when running from within a Python script

Then you don't need streamlink_cli

@slhck
Copy link
Contributor Author

slhck commented Jul 18, 2023

The example I gave is just a dummy example to show a workaround. It could have been written in whatever other programming language. If you're interfacing with the API, sure, you can do that, but like I wrote above, it will mean you have to re-do most of the CLI work. So I was looking for a more general solution.

Feel free to close this issue if you don't think it's necessary.

@slhck
Copy link
Contributor Author

slhck commented Jul 18, 2023

Thinking about it more, I believe there is a case for an even more general clean exit. Quitting the stream with SIGINT leaves you with possibly broken output, e.g. in the above case,

ffprobe -count_frames tmp.mkv

Runs into this error:

[matroska,webm @ 0x6f3dfc0] File ended prematurely

Because the signal may abort the download somewhere in the middle of a segment being written out.
Perhaps that is the real issue, and SIGINT should be trapped and forwarded to the function that writes the data, making it wait until a segment is completely written (eventually force-closing after a second SIGINT, or with SIGKILL).

@bastimeyer
Copy link
Member

HLS streams have an option for ending the stream gracefully at specific segments:
https://streamlink.github.io/cli.html#cmdoption-hls-duration

DASH streams currently don't have such an option. And for other non-segmented types of streams, it's not possible.

@slhck
Copy link
Contributor Author

slhck commented Jul 18, 2023

Oh, I didn't see that, thanks. In that case it could probably be added for DASH. I will look into whether I can create a PR.

@bastimeyer
Copy link
Member

bastimeyer commented Jul 18, 2023

And for regular SIGINT/SIGTERM behavior, it's not possible to always end cleanly/gracefully, because Streamlink's ringbuffer is unaware of its content structure, meaning when the CLI stops reading, it can be at any point, just like you've said. The various stream implementations simply write the contents to the buffer which the CLI or API consumers are reading from. Streamlink's main purpose is not recording, so if you need to fix your files with ffmpeg after recording, so be it.

@bastimeyer
Copy link
Member

I will look into whether I can create a PR.

I can have a look at this tomorrow or so. The --hls-duration CLI argument and respective session option should be deprecated in favor of a new --segment-duration argument (or so), so we don't have two separate HLS and DASH options which do the same stuff.

Not sure yet though what this will mean to --hls-start-offset. Implementing the respective logic for DASH streams might be a bit trickier, but it doesn't necessarily have to be implemented now if we also decide to rename it (and deprecate the old name).

@slhck
Copy link
Contributor Author

slhck commented Jul 18, 2023

Sure. --segment-duration however seems ambiguous to me, as I understand that to be the duration of a single segment. --stream-duration would make more sense, or --max-stream-duration. This could simply override stream.duration and set duration_limit as before in HLS.

@bastimeyer
Copy link
Member

I want to avoid --stream-duration, because as said, it's not possible for every stream type to implement this. --stream-segmented-duration should work though and it'd align with the already existing --stream-... CLI args.

These changes however won't make it into 6.0.0. That release will be out very soon and I don't want to delay it any further.

@bastimeyer bastimeyer changed the title Add an option to quit after a certain amount of time Turn --hls-duration into a generic segmented-stream CLI arg and add support for DASH durations Jul 18, 2023
@bastimeyer
Copy link
Member

I have a local branch ready which I will submit later.

This will deprecate --hls-duration and add --stream-segmented-duration in favor of the HLS-specific one. Both HLSStreamWorker and DASHStreamWorker will support stopping the stream early when a duration has been set.

Theoretically, this could/should be implemented in the SegmentedStreamWorker base class, as it's only subclassed by these two stream implementations (and the custom UStreamTVStreamWorker implementation as well). The requirement here is that workers have to deal with segments which implement the duration attribute. I don't know though if that's a good idea, because this requirement is not specified and there's no guarantee that this always the case in custom segmented stream implementations. The design of the various stream APIs isn't ideal and lots of typing information is also missing. I've talked about that in the past a couple of times already, but making changes is really difficult. Starting anew with a clean streams implementation (ideally an async one based on httpx without threading nonsense) is also difficult and very tedious.

Then there's the issue with the related --hls-start-offset parameter. For HLS playlists, the implementation is simple, because segments are all known. In DASH manifests however, the representation.segments() method is a generator, which makes an implementation problematic and we can't figure out a start offset easily, depending on the DASH stream type. I would like to solve this in the same release, because as said, those parameters are related, and deprecating one after another in different releases is bad. I haven't had the time and motivation yet to check this.

@slhck
Copy link
Contributor Author

slhck commented Aug 9, 2023

Oof, that does not sound that easy. I've recently had my issues with a DASH stream without known durations and I can relate to this. I'm not deep into the code base but happy to test/comment on any proposed changes if that helps. Thanks for sharing your considerations!

@bastimeyer
Copy link
Member

See #5493

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants