[SchoolTV/NPO] Downloading token: Failed to parse JSON

[stoofvlees]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2021.12.17
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

stoofvlees@stoofvlees:~$ youtube-dl --version
2021.12.17
stoofvlees@stoofvlees:~$ youtube-dl -v https://schooltv.nl/video/de-oosterscheldekering-bescherming-tegen-het-water/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://schooltv.nl/video/de-oosterscheldekering-bescherming-tegen-het-water/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.10.12 (CPython) - Linux-6.2.0-37-generic-x86_64-with-glibc2.35
[debug] exe versions: ffmpeg 4.4.2, ffprobe 4.4.2, rtmpdump 2.4
[debug] Proxy map: {}
[schooltv] de-oosterscheldekering-bescherming-tegen-het-water: Downloading webpage
[npo] WO_NTR_16170538: Downloading token
ERROR: WO_NTR_16170538: Failed to parse JSON  (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 906, in _parse_json
    return json.loads(json_string)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 906, in _parse_json
    return json.loads(json_string)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/npo.py", line 184, in _real_extract
    return self._get_info(url, video_id) or self._get_old_info(video_id)
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/npo.py", line 187, in _get_info
    token = self._download_json(
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 895, in _download_json
    res = self._download_json_handle(
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 881, in _download_json_handle
    return self._parse_json(
  File "/usr/lib/python3/dist-packages/youtube_dl/extractor/common.py", line 910, in _parse_json
    raise ExtractorError(errmsg, cause=ve)
youtube_dl.utils.ExtractorError: WO_NTR_16170538: Failed to parse JSON  (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

Doesn't download using the regular URL (see output above). However, I was able to download it using the following steps:

• use Firefox developer tools, tab "Network"
• look for a file named "stream.mpd"
• choose "copy value -> copy URL" (in my case: https://npo-nl-ams-p26-am3.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE3MDE1MjAxNzUsInVyaSI6Ilwvdm9kXC9ucG9cL3VzcFwvbnBvXC9kYXNoX3VuZW5jcnlwdGVkXC9XT19OVFJfMTYxNzA1MzhcL1dPX05UUl8xNjE3MDUzOF92MTU5MDY3NTE0Ny5pc20iLCJjbGllbnRfaXAiOiI2Mi4xNjYuMTY4LjE2Iiwidmlld2VyIjoidmlld2VyIiwicmlkIjoiMDJiNTYxYSJ9.ltMsKupBI0RTu8S91Q9pFpyTjMQYsjZVa-h_Zk4q4io/vod/npo/usp/npo/dash_unencrypted/WO_NTR_16170538/WO_NTR_16170538_v1590675147.ism/stream.mpd)
• use that URL to feed youtube-dl
• done. I had to fix the file name but that's okay

stoofvlees@stoofvlees:~$ youtube-dl https://npo-nl-ams-p26-am3.cdn.streamgate.nl/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE3MDE1MjAxNzUsInVyaSI6Ilwvdm9kXC9ucG9cL3VzcFwvbnBvXC9kYXNoX3VuZW5jcnlwdGVkXC9XT19OVFJfMTYxNzA1MzhcL1dPX05UUl8xNjE3MDUzOF92MTU5MDY3NTE0Ny5pc20iLCJjbGllbnRfaXAiOiI2Mi4xNjYuMTY4LjE2Iiwidmlld2VyIjoidmlld2VyIiwicmlkIjoiMDJiNTYxYSJ9.ltMsKupBI0RTu8S91Q9pFpyTjMQYsjZVa-h_Zk4q4io/vod/npo/usp/npo/dash_unencrypted/WO_NTR_16170538/WO_NTR_16170538_v1590675147.ism/stream.mpd
[generic] stream: Requesting header
WARNING: Falling back on generic information extractor.
[generic] stream: Downloading webpage
[generic] stream: Extracting information
[dashsegments] Total fragments: 40
[download] Destination: stream-stream.fvideo=1949000.mp4
[download] 100% of 71.89MiB in 00:12
[dashsegments] Total fragments: 41
[download] Destination: stream-stream.faudio_eng=128000.m4a
[download] 100% of 4.77MiB in 00:03
[ffmpeg] Merging formats into "stream-stream.mp4"
Deleting original file stream-stream.fvideo=1949000.mp4 (pass -k to keep)
Deleting original file stream-stream.faudio_eng=128000.m4a (pass -k to keep)

[dirkf]

It's useful to know that a non-DRM video is available for the page, but this isn't exactly a "fix".

This site used to use the NPO API but a lot of the NPO stuff seems suspect now.

I ran the page and found that it used an entirely different way of fetching the video, resulting in (for a UK session) 451 error with error reason (translated roughly) "video unavailable in your location".

Any comments regarding other sites supported by the NPO extractor would be welcome.

[dirkf]

Also, the error is in code that was already known to be obsolete (yt-dlp/yt-dlp#6398, #28381).