Support npostart.nl at youtube-dl #28381
Comments
|
Maybe it's caused by (translated):
youtube-dl -v https://www.npostart.nl/jouw-schuld-dat-ik-dik-ben/21-10-2020/VPWON_1320428
[debug] System config: []
[debug] User config: ['-i', '-o', '~/Desktop/test-ytd/%(title)s.%(ext)s', '-f', 'bestvideo[height<=1080][ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best', '--no-mtime', '--add-metadata', '--embed-thumbnail', '--force-ipv4', '--external-downloader', 'aria2c', '--external-downloader-args', '-x 10 -s 10 -j 10 -k 1M --log-level=info --file-allocation=none']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.npostart.nl/jouw-schuld-dat-ik-dik-ben/21-10-2020/VPWON_1320428']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.03.03
[debug] Python version 3.6.9 (CPython) - Linux-5.4.0-66-generic-x86_64-with-LinuxMint-19.3-tricia
[debug] exe versions: ffmpeg N-101396-ga272f38-Nico-20210306, ffprobe N-101396-ga272f38-Nico-20210306
[debug] Proxy map: {}
[npo] VPWON_1320428: Downloading token
[npo] VPWON_1320428: Downloading player JSON
[npo] VPWON_1320428: Downloading hls profile JSON
[npo] VPWON_1320428: Downloading dash-widevine profile JSON
[npo] VPWON_1320428: Downloading dash-playready profile JSON
[npo] VPWON_1320428: Downloading smooth profile JSON
[npo] VPWON_1320428: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/usr/local/bin/youtube-dl.orig/youtube_dl/extractor/common.py", line 632, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 2275, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 564, in error
result = self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 756, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)On another video from www.npostart.nl I get: |
|
Tip: |
|
npo urls work for me |
|
Seems to work with "hunted" as well
|
|
In case someone besides myself is trying those sample NPO URLs from outside of The Netherlands, https://www.npostart.nl/de-lamas/20-07-2015/BNN_101376241 is not geo-fenced (non-Dutch, European IP): ... but https://www.npostart.nl/hunted/01-11-2021/AT_300001444 DOES geo-fence: Above URL works as expected via a Dutch VPN...
Well, breaking DRM was never and (in all probability) will never be supported in Taking this issue's title for what it is, methinks support for |
|
Npo is supported, but not playlists or entire seasons.That would be a cool feature.
|
That one can be downloaded via downloadgemist. :) |
|
Ok
Many thankx for your reply and tips.
Op do 2 dec. 2021 om 10:47 schreef nicolaasjan ***@***.***>:
… youtube-dl doesn't work with this
https://www.npostart.nl/nood/30-11-2021/BV_101406936 It says drm
protected.
That one can be downloaded via downloadgemist
<https://www.downloadgemist.nl/>. :)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#28381 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATERM4TUC5SMATTPEX3CHSTUO46DTANCNFSM4Y2CGSPA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
... How do they obtain those files? The files appear to be served from a
|
|
@Vangelis66 From: https://www.downloadgemist.nl/?page=opnames (machine translated):
When giving the yt-dlp -F --allow-unplayable-formats "https://nl-ams-p18-am5.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2Mzg2MjUyMDgsInVyaSI6Ilwvdm9kXC9ucG9cL3VzcFwvVEVTVFwvbnBvXC9kYXNoX2NlbmNcL0JWXzEwMTQwNjkzNlwvQlZfMTAxNDA2OTM2X3YxNjM4MjMxMTQyLmlzbSIsImNsaWVudF9pcCI6IjIxNy4xMjEuMTkzLjYzIiwidmlld2VyIjoiNGUyZjQ0YzQ5ZWQzZDVmN2EwZTA5ZjVlZTQ1MTNlMDhhZWY1OTcyYyIsInJpZCI6ImU2ODk4ODIifQ.368A4v8B3Hd6TiaMrng435hFotsRntMXDIsn6fcKask/vod/npo/usp/TEST/npo/dash_cenc/BV_101406936/BV_101406936_v1638231142.ism/stream.mpd"
WARNING: You have asked for UNPLAYABLE formats to be listed/downloaded. This is a developer option intended for debugging.
If you experience any issues while using this option, DO NOT open a bug report
WARNING: --embed-metadata is ignored since --allow-unplayable-formats was given
WARNING: --embed-thumbnail is ignored since --allow-unplayable-formats was given
WARNING: --sponsorblock-remove is ignored since --allow-unplayable-formats was given
[generic] stream: Requesting header
WARNING: [generic] Falling back on generic information extractor.
[generic] stream: Downloading webpage
[generic] stream: Extracting information
[SponsorBlock] SponsorBlock is not supported for Generic
[info] Available formats for stream:
ID EXT RESOLUTION │ TBR PROTO │ VCODEC VBR ACODEC ABR ASR MORE INFO
────────────────────────────────────────────────────────────────────────────────────────────────────────────
�[0audio=128000 m4a │ 128k dash │ audio only mp4a.40.2 128k 48000Hz DASH audio, m4a_dash
video=589000 mp4 640x360 │ 589k dash │ avc1.42C01E 589k video only DASH video, mp4_dash
video=1915000 mp4 960x540 │ 1915k dash │ avc1.4D401F 1915k video only DASH video, mp4_dash
|
|
@Vangelis66 |
|
The original error was caused by the extractor failing to find any valid streams and falling back to an older and now obsolete extraction method that fetched a token using a discontinued API (410 Gone). By the time of #28381 (comment), the domain ida.omroep.nl used in the discontinued API had also gone. The lack of valid streams is not directly caused by DRM, which would have been reported. The site leaves programme pages up even after the linked streams are no longer available (niet beschikbaar) or not found (niet gevonden) (conversely, the APIs used by the extractor may still return valid data for a programme whose page redirects to the home page). The extractor needs to detect these errors and report them like DRM, and to have the obsolete extraction fallback removed. And then: $ python -m youtube_dl -v -F 'https://www.npostart.nl/jouw-schuld-dat-ik-dik-ben/21-10-2020/VPWON_1320428'
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.npostart.nl/jouw-schuld-dat-ik-dik-ben/21-10-2020/VPWON_1320428']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 14a087ec6
[debug] Python version 2.7.17 (CPython) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[npo] VPWON_1320428: Downloading token
[npo] VPWON_1320428: Downloading player JSON
[npo] VPWON_1320428: Downloading hls profile JSON
[npo] VPWON_1320428: Downloading dash-widevine profile JSON
[npo] VPWON_1320428: Downloading dash-playready profile JSON
[npo] VPWON_1320428: Downloading smooth profile JSON
ERROR: Dit programma is niet gevonden (53)
Traceback (most recent call last):
File "youtube_dl/YoutubeDL.py", line 815, in wrapper
return func(self, *args, **kwargs)
File "youtube_dl/YoutubeDL.py", line 836, in __extract_info
ie_result = ie.extract(url)
File "youtube_dl/extractor/common.py", line 534, in extract
ie_result = self._real_extract(url)
File "youtube_dl/extractor/npo.py", line 197, in _real_extract
return self._get_info(url, video_id) # or self._get_old_info(video_id)
File "youtube_dl/extractor/npo.py", line 269, in _get_info
raise ExtractorError('; '.join(errors), expected=True)
ExtractorError: Dit programma is niet gevonden (53)
$ |
|
Using the latest release (which is already quite old, I have not tried compiling a snapshot), it prints the following error on the end: Using the following test url, from a Dutch IP address: |
|
That's expected:
But now a new setup dialogue is needed, as in the yt_dlp extractor. Then both extractors fail when extracting the streams because some unknown POST data is required. |
I wouldn't spend myself any coding time over this specific extractor 😞 ; NPO are known to have put EVERYTHING behind a full-blown DRM fence some years ago 👎 ; below screengrab is from a "legacy" browser that used to have WidevineCDM support, but Google now officially sanction only their own browser and its siblings/close relatives (i.e. Firefox) 😡 :
|
|
As the DRM issue seems to date back at least 3 years, I'll close this, along with some older related zombie issues. Ofc if someone can find a non-DRM show, we can kick off again. |
This comment was marked as outdated.
This comment was marked as outdated.
|
And here it is, as well as the new |
😄 ; probably being pedantic, but that DRM-free program is an NPO3 offering, delivered from
(from opening post); technically, NPO and NPO3 are handled by the same InfoExtractor ( |
|
... And HERE it is (take 2); an NPO ancient offering back from 1995, without DRM: https://www.npostart.nl/de-verborgen-geschiedenis/21-05-1995/WO_VPRO_035772 I doubt, though, that Dutch speaking people would want to use Later addition: Apparently, the programme mentioned above (from 2015) is still available and also DRM-free 😜 ... |
IMHO it is unacceptable that the NPO (Nederlandse Publieke Omroep; Dutch Public Broadcasting), paid by our tax money, puts its content behind DRM now. 😡 |
|
nicolaasjan: (Google and) Rights Holders can exert considerable leverage these days 😉 😠 ... But we're probably drifting OT here... |
|
Lots of things are being discussed here. Some of them will be fixed with this PR #31976 once it's merged. It doesn't fix the DRM issue though. |
Checklist
Question
WRITE QUESTION HERE
Do you support npostart.nl?
When I use this link:
https://www.npostart.nl/jouw-schuld-dat-ik-dik-ben/21-10-2020/VPWON_1320428
I get this error on youtube-dl:
[npo] VPWON_1320428: Downloading token
[npo] VPWON_1320428: Downloading player JSON
[npo] VPWON_1320428: Downloading hls profile JSON
[npo] VPWON_1320428: Downloading dash-widevine profile JSON
[npo] VPWON_1320428: Downloading dash-playready profile JSON
[npo] VPWON_1320428: Downloading smooth profile JSON
[npo] VPWON_1320428: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 410: Gone (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
The text was updated successfully, but these errors were encountered: