Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugins.dogus: StarTV live TV doesn't work #5298

Open
4 tasks done
stulluk opened this issue Apr 18, 2023 · 9 comments
Open
4 tasks done

plugins.dogus: StarTV live TV doesn't work #5298

stulluk opened this issue Apr 18, 2023 · 9 comments
Labels
plugin issue A Plugin does not work correctly unconfirmed

Comments

@stulluk
Copy link

stulluk commented Apr 18, 2023

Checklist

Streamlink version

Latest stable release

Description

Thank you very much for this plugin. It works fine for NTV and KralTV.

However, it doesn't work for StarTV, log is below.

Debug log

stulluk ~ $  streamlink https://www.startv.com.tr/canli-yayin best -l all
[13:28:04.497582][cli][debug] OS:         Linux-6.2.10-060210-generic-x86_64-with-glibc2.35
[13:28:04.497678][cli][debug] Python:     3.10.6
[13:28:04.497718][cli][debug] Streamlink: 5.4.0
[13:28:04.498169][cli][debug] Dependencies:
[13:28:04.499867][cli][debug]  certifi: 2020.6.20
[13:28:04.500194][cli][debug]  isodate: 0.6.1
[13:28:04.500454][cli][debug]  lxml: 4.8.0
[13:28:04.500846][cli][debug]  pycountry: 20.7.3
[13:28:04.501073][cli][debug]  pycryptodome: 3.17
[13:28:04.501747][cli][debug]  PySocks: 1.7.1
[13:28:04.502014][cli][debug]  requests: 2.28.2
[13:28:04.504453][cli][debug]  urllib3: 1.26.5
[13:28:04.504743][cli][debug]  websocket-client: 1.2.3
[13:28:04.504979][cli][debug]  importlib-metadata: 4.6.4
[13:28:04.505076][cli][debug] Arguments:
[13:28:04.505115][cli][debug]  url=https://www.startv.com.tr/canli-yayin
[13:28:04.505152][cli][debug]  stream=['best']
[13:28:04.505191][cli][debug]  --loglevel=all
[13:28:04.505337][cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
error: No playable streams found on this URL: https://www.startv.com.tr/canli-yayin
stulluk ~ $  streamlink --version
streamlink 5.4.0
stulluk ~ $
@stulluk stulluk added the plugin issue A Plugin does not work correctly label Apr 18, 2023
@bastimeyer
Copy link
Member

The site embeds a YouTube livestream, which the plugin properly finds and hands over to the youtube plugin:

$ streamlink -l debug 'https://www.startv.com.tr/canli-yayin'
[cli][debug] OS:         Linux-6.2.11-1-git-x86_64-with-glibc2.37
[cli][debug] Python:     3.11.3
[cli][debug] Streamlink: 5.4.0+5.g5846abd6
[cli][debug] Dependencies:
[cli][debug]  certifi: 2022.12.7
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 4.9.2
[cli][debug]  pycountry: 22.3.5
[cli][debug]  pycryptodome: 3.17
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.28.2
[cli][debug]  urllib3: 1.26.15
[cli][debug]  websocket-client: 1.5.1
[cli][debug] Arguments:
[cli][debug]  url=https://www.startv.com.tr/canli-yayin
[cli][debug]  --loglevel=debug
[cli][debug]  --player=mpv
[cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
[plugins.youtube][debug] Using video ID: _F3fvsRAFF0
[plugins.youtube][debug] This video is live.
[utils.l10n][debug] Language code: en_US
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

@stulluk
Copy link
Author

stulluk commented Apr 18, 2023

@bastimeyer thank you for looking into this. However, why my debug log is much different than yours ( Due to geographic location ? )

stulluk ~ $ streamlink -l debug 'https://www.startv.com.tr/canli-yayin'
[cli][debug] OS: Linux-6.2.10-060210-generic-x86_64-with-glibc2.35
[cli][debug] Python: 3.10.6
[cli][debug] Streamlink: 5.4.0
[cli][debug] Dependencies:
[cli][debug] certifi: 2020.6.20
[cli][debug] isodate: 0.6.1
[cli][debug] lxml: 4.8.0
[cli][debug] pycountry: 20.7.3
[cli][debug] pycryptodome: 3.17
[cli][debug] PySocks: 1.7.1
[cli][debug] requests: 2.28.2
[cli][debug] urllib3: 1.26.5
[cli][debug] websocket-client: 1.2.3
[cli][debug] importlib-metadata: 4.6.4
[cli][debug] Arguments:
[cli][debug] url=https://www.startv.com.tr/canli-yayin
[cli][debug] --loglevel=debug
[cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
error: No playable streams found on this URL: https://www.startv.com.tr/canli-yayin
stulluk ~ $

Could you please let me know, what is needed for you to be able to reproduce my above situation ?

@mkbloke
Copy link
Member

mkbloke commented Apr 18, 2023

I get the same working YT live feed from a UK IP address. @stulluk are you accessing from an IP in Turkey? Maybe the site does something different for local users.

@stulluk
Copy link
Author

stulluk commented Apr 18, 2023

I get the same working YT live feed from a UK IP address. @stulluk are you accessing from an IP in Turkey? Maybe the site does something different for local users.

Exactly. Thats why we are observing different behavior.

To confirm, I also tried from a server located in germany:

root@xxx ~ # streamlink -l debug 'https://www.startv.com.tr/canli-yayin'
[cli][info] streamlink is running as root! Be careful!
[cli][debug] OS:         Linux-5.4.0-139-generic-x86_64-with-glibc2.29
[cli][debug] Python:     3.8.10
[cli][debug] Streamlink: 5.4.0
[cli][debug] Dependencies:
[cli][debug]  certifi: 2019.11.28
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 4.9.2
[cli][debug]  pycountry: 22.3.5
[cli][debug]  pycryptodome: 3.17
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.28.2
[cli][debug]  urllib3: 1.26.15
[cli][debug]  websocket-client: 1.5.1
[cli][debug] Arguments:
[cli][debug]  url=https://www.startv.com.tr/canli-yayin
[cli][debug]  --loglevel=debug
[cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
[plugins.youtube][debug] Using video ID: _F3fvsRAFF0
[plugins.youtube][debug] This video is live.
[utils.l10n][debug] Language code: en_US
error: Unable to open URL: https://manifest.googlevideo.com/api/manifest/hls_variant/expire/1681842392/ei/eIw-ZIjUNpSU-gbZ-r6QCA/ip/2a01%3A4f8%3A221%3A2f16%3A0%3A0%3A0%3A2/id/_F3fvsRAFF0.4/source/yt_live_broadcast/requiressl/yes/hfr/1/playlist_duration/30/manifest_duration/30/maudio/1/gcr/de/vprv/1/go/1/pacing/0/nvgoi/1/keepalive/yes/fexp/24007246/dover/11/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cplaylist_duration%2Cmanifest_duration%2Cmaudio%2Cgcr%2Cvprv%2Cgo%2Citag%2Cplaylist_type/sig/AOq0QJ8wRQIgSyLIWjLcE3bIZk9Y1_wv-g4sOCl-N-_G7vnI75akFSUCIQCMus-EGwygnshYaj_3YQ_D94xwYtHd05Dpcwq5nDO03A%3D%3D/file/index.m3u8 (429 Client Error: Too Many Requests for url: https://manifest.googlevideo.com/api/manifest/hls_variant/expire/1681842392/ei/eIw-ZIjUNpSU-gbZ-r6QCA/ip/2a01%3A4f8%3A221%3A2f16%3A0%3A0%3A0%3A2/id/_F3fvsRAFF0.4/source/yt_live_broadcast/requiressl/yes/hfr/1/playlist_duration/30/manifest_duration/30/maudio/1/gcr/de/vprv/1/go/1/pacing/0/nvgoi/1/keepalive/yes/fexp/24007246/dover/11/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cplaylist_duration%2Cmanifest_duration%2Cmaudio%2Cgcr%2Cvprv%2Cgo%2Citag%2Cplaylist_type/sig/AOq0QJ8wRQIgSyLIWjLcE3bIZk9Y1_wv-g4sOCl-N-_G7vnI75akFSUCIQCMus-EGwygnshYaj_3YQ_D94xwYtHd05Dpcwq5nDO03A%3D%3D/file/index.m3u8)
root@xxx ~ #

Could you please suggest me, how to fix this issue ?

@bastimeyer
Copy link
Member

Unless someone else already has, I'll have a look in a couple of hours with a VPN.

As you can see, the plugin is really simple and whatever this specific site does in your region won't be much different from what's already implemented, so basically just an additional xpath query or so.
https://github.com/streamlink/streamlink/blob/5.4.0/src/streamlink/plugins/dogus.py

@mkbloke
Copy link
Member

mkbloke commented Apr 18, 2023

In the case of your access via German IP, @stulluk, you're getting a different issue:

429 Client Error: Too Many Requests for url:

@stulluk
Copy link
Author

stulluk commented Apr 19, 2023

Unless someone else already has, I'll have a look in a couple of hours with a VPN.

As you can see, the plugin is really simple and whatever this specific site does in your region won't be much different from what's already implemented, so basically just an additional xpath query or so. https://github.com/streamlink/streamlink/blob/5.4.0/src/streamlink/plugins/dogus.py

@bastimeyer thank you. I am not a web developer, so I don't have experience on what is an xpath query, but if I understand correctly, this is to find HLS links in the webpage code ?

Additionally, would it be possible for you to check with a VPN ?

In the case of your access via German IP, @stulluk, you're getting a different issue:

429 Client Error: Too Many Requests for url:

@mkbloke I made a quick test on my Ubuntu Desktop via Proton VPN to connect to a Netherlands VPN:

stulluk ~ $  streamlink -l debug 'https://www.startv.com.tr/canli-yayin' 480p
[cli][debug] OS:         Linux-6.2.10-060210-generic-x86_64-with-glibc2.35
[cli][debug] Python:     3.10.6
[cli][debug] Streamlink: 5.4.0
[cli][debug] Dependencies:
[cli][debug]  certifi: 2020.6.20
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 4.8.0
[cli][debug]  pycountry: 20.7.3
[cli][debug]  pycryptodome: 3.17
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.28.2
[cli][debug]  urllib3: 1.26.5
[cli][debug]  websocket-client: 1.2.3
[cli][debug]  importlib-metadata: 4.6.4
[cli][debug] Arguments:
[cli][debug]  url=https://www.startv.com.tr/canli-yayin
[cli][debug]  stream=['480p']
[cli][debug]  --loglevel=debug
[cli][info] Found matching plugin dogus for URL https://www.startv.com.tr/canli-yayin
[plugins.youtube][debug] Using video ID: _F3fvsRAFF0
[plugins.youtube][debug] This video is live.
[utils.l10n][debug] Language code: en_US
[cli][info] Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)
[cli][info] Opening stream: 480p (hls)
[cli][info] Starting player: /usr/bin/vlc
[stream.hls][debug] Reloading playlist
[cli][debug] Pre-buffering 8192 bytes
[stream.hls][debug] First Sequence: 1651641; Last Sequence: 1651646
[stream.hls][debug] Start offset: 0; Duration: None; Start Sequence: 1651644; End Sequence: None
[stream.hls][debug] Adding segment 1651644 to queue
[stream.hls][debug] Adding segment 1651645 to queue
[stream.hls][debug] Adding segment 1651646 to queue
[stream.hls][debug] Writing segment 1651644 to output
[stream.hls][debug] Segment 1651644 complete
[cli.output][debug] Opening subprocess: /usr/bin/vlc --input-title-format https://www.startv.com.tr/canli-yayin -
[stream.hls][debug] Writing segment 1651645 to output
[stream.hls][debug] Segment 1651645 complete
[stream.hls][debug] Writing segment 1651646 to output
[stream.hls][debug] Segment 1651646 complete
[cli][debug] Writing stream to output
[stream.hls][debug] Reloading playlist
[stream.hls][debug] Adding segment 1651647 to queue
[stream.hls][debug] Writing segment 1651647 to output
[stream.hls][debug] Segment 1651647 complete
[cli][info] Player closed
[stream.segmented][debug] Closing worker thread
[stream.segmented][debug] Closing writer thread
[cli][info] Stream ended
[cli][info] Closing currently open stream...
stulluk ~ $  streamlink -l debug 'https://www.startv.com.tr/canli-yayin' 480p --stream-url
https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1681904784/ei/MIA_ZNToGI3RgQeEk4aICQ/ip/149.34.244.155/id/_F3fvsRAFF0.4/itag/94/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/sgoap/gir%3Dyes%3Bitag%3D140/sgovp/gir%3Dyes%3Bitag%3D135/hls_chunk_host/rr1---sn-5hne6nz6.googlevideo.com/playlist_duration/30/manifest_duration/30/gcr/nl/vprv/1/playlist_type/DVR/initcwndbps/538750/mh/t5/mm/44/mn/sn-5hne6nz6/ms/lva/mv/m/mvi/1/pl/24/dover/11/pacing/0/keepalive/yes/fexp/24007246/mt/1681882827/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,live,sgoap,sgovp,playlist_duration,manifest_duration,gcr,vprv,playlist_type/sig/AOq0QJ8wRgIhAM21e-7DsM53s8eW5PirpA28b2c4llvQBpD4IS3LcnGGAiEA7gctXoMBsEN3iQqlji4CYMOz5dr9cowUTcNpMoU_9Gk%3D/lsparams/hls_chunk_host,initcwndbps,mh,mm,mn,ms,mv,mvi,pl/lsig/AG3C_xAwRQIhAPMcWq5aVeW0aTvu7vQy_BHOfPDdTzsVehkYFh_N-dEUAiBJmgDP5rnZom_3EdxE7uHc6mK-oTqgpJbWFvjHZGdJnQ%3D%3D/playlist/index.m3u8
stulluk ~ $  

So the backed behaves differently depending on geographic location :(

So, if I understand correctly, we need to find a way to fix this plugin for Turkey locations, am I wrong ?

@bastimeyer
Copy link
Member

I had a look yesterday and I got the different stream implementation when accessing the site via a French proxy. It's a bit more difficult though because the stream URL is embedded in their player JS file in that case. This can be extracted rather easily, but there's an additional base64 encoded query string added by their player code. When accessing the stream URL without that data, it still seemed to work, but I'm not 100% sure if there will be issues when not including that data. I didn't manage to find out where the data was coming from and where it was calculated, because the site is terribly bloated and loads a ton of additional JS scripts asynchronously while initializing the player and the stream URL.

@meatab78

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin issue A Plugin does not work correctly unconfirmed
Projects
None yet
Development

No branches or pull requests

4 participants