Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

motherless.com HTTP Error 403: Forbidden #32724

Open
5 tasks done
Iridium-Lo opened this issue Feb 19, 2024 · 16 comments
Open
5 tasks done

motherless.com HTTP Error 403: Forbidden #32724

Iridium-Lo opened this issue Feb 19, 2024 · 16 comments
Labels
broken-IE problem with existing site extraction

Comments

@Iridium-Lo
Copy link

Iridium-Lo commented Feb 19, 2024

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dl version 2021.12.17
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones
youtube-dl --version
2021.12.17

Verbose log

youtube-dl -vF --no-check-certificate https://motherless.com/02FB80C
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-vF', u'--no-check-certificate', u'https://motherless.com/02FB80C']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python 2.7.18 (CPython x86_64 64bit) - Darwin-22.5.0-x86_64-i386-64bit - OpenSSL 1.0.2u  20 Dec 2019
[debug] exe versions: ffmpeg 6.0, ffprobe 6.0, rtmpdump 2.4
[debug] Proxy map: {}
[Motherless] 02FB80C: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2021.12.17-py2.7.egg/youtube_dl/extractor/common.py", line 678, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2021.12.17-py2.7.egg/youtube_dl/YoutubeDL.py", line 2461, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

Description

  • get this error: HTTP Error 403: Forbidden
  • this was working yesterday so I suspect something changed on the site
  • able to play videos on browser
@dirkf
Copy link
Contributor

dirkf commented Feb 20, 2024

See yt-dlp/yt-dlp#9248 and the linked advice, except that yt-dl doesn't have --cookies-from-browser ....

@Iridium-Lo
Copy link
Author

Iridium-Lo commented Feb 20, 2024

I usually use --user-agent Mozilla/5.0

I'll try refreshing the cookies in firefox while on the site then do:

youtube-dl site --user-agent Mozilla/5.0 --cookies-from-browser firefox

Is my understanding of the steps correct?

Oh damn so this is for yt-dlp only? Nothing for YouTube-dl?

@dirkf
Copy link
Contributor

dirkf commented Feb 20, 2024

Follow the FAQ instructions for exporting cookies to a file and then use --cookies ... with the name of the cookie file as argument.

@Iridium-Lo
Copy link
Author

Iridium-Lo commented Feb 20, 2024

when I go to the site page It checks if I'm human.
I cleared the cookies then got them in netscape (yt-download asked for this) format, but the site still knows that it's being downloaded by a non human:

youtube-dl --user-agent Mozilla/5.0 --cookies ~/cookies.txt --no-check-certificate -v https://motherless.com/043D161
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--user-agent', u'Mozilla/5.0', u'--cookies', u'/Users/iridium/cookies.txt', u'--no-check-certificate', u'-v', u'https://motherless.com/043D161']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python 2.7.18 (CPython x86_64 64bit) - Darwin-22.5.0-x86_64-i386-64bit - OpenSSL 1.0.2u  20 Dec 2019
[debug] exe versions: ffmpeg present, ffprobe present, rtmpdump 2.4
[debug] Proxy map: {}
[Motherless] 043D161: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2021.12.17-py2.7.egg/youtube_dl/extractor/common.py", line 678, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2021.12.17-py2.7.egg/youtube_dl/YoutubeDL.py", line 2461, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

Same thing with yt-dlp (after I uninstalled. YouTube-dl):

 yt-dlp --user-agent Mozilla/5.0 --cookies-from-browser firefox --no-check-certificate -v https://motherless.com/043D161
[debug] Command-line config: ['--user-agent', 'Mozilla/5.0', '--cookies-from-browser', 'firefox', '--no-check-certificate', '-v', 'https://motherless.com/043D161']
Extracting cookies from firefox
[debug] Extracting cookies from: "/Users/iridium/Library/Application Support/Firefox/Profiles/a00swf8e.default-release/cookies.sqlite"
Extracted 10 cookies from firefox
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.12.30 from yt-dlp/yt-dlp [f10589e34] (pip)
[debug] Python 3.12.2 (CPython x86_64 64bit) - macOS-13.4-x86_64-i386-64bit (OpenSSL 3.2.1 30 Jan 2024)
[debug] exe versions: rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1798 extractors
[Motherless] Extracting URL: https://motherless.com/043D161
[Motherless] 043D161: Downloading webpage
ERROR: [Motherless] 043D161: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: Forbidden>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 718, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/motherless.py", line 113, in _real_extract
    webpage = self._download_webpage(url, video_id)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 1121, in _download_webpage
    return self.__download_webpage(url_or_request, video_id, note, errnote, None, fatal, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 1072, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 906, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 863, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 4082, in urlopen
    return self._request_director.send(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/networking/common.py", line 114, in send
    response = handler.send(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/networking/_helper.py", line 204, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/networking/common.py", line 325, in send
    return self._send(request)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/networking/_requests.py", line 343, in _send
    raise HTTPError(res, redirect_loop=max_redirects_exceeded)
yt_dlp.networking.exceptions.HTTPError: HTTP Error 403: Forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 850, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 4114, in urlopen
    raise _CompatHTTPError(e) from e
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 403: Forbidden

any ideas?

@dirkf
Copy link
Contributor

dirkf commented Feb 20, 2024

Try using the same UA that acquired the cookies, as per the yt-dlp instructions (or making your browser use Mozilla/5.0).

@Iridium-Lo
Copy link
Author

Iridium-Lo commented Feb 21, 2024

that's what I done with --user-agent Mozilla/5.0 do I need the full string; the stuff in brackets and other stuff?

firefox is using Mozilla/5.0

sorry ignore that, the guide says to use the whole string, trying now.

@Iridium-Lo
Copy link
Author

Iridium-Lo commented Feb 21, 2024

I cleared cookies, then got the cookies in netscape format, but this still didn't work:

youtube-dl:

youtube-dl -v --cookies ~/cookies.txt --user-agent "Mozilla/5.0'(Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0" --no-check-certificate https://motherless.com/03E0E46
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--cookies', u'/Users/iridium/cookies.txt', u'--user-agent', u"Mozilla/5.0'(Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0", u'--no-check-certificate', u'https://motherless.com/03E0E46']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python 2.7.18 (CPython x86_64 64bit) - Darwin-22.5.0-x86_64-i386-64bit - OpenSSL 1.0.2u  20 Dec 2019
[debug] exe versions: ffmpeg present, ffprobe present, rtmpdump 2.4
[debug] Proxy map: {}
[Motherless] 03E0E46: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2021.12.17-py2.7.egg/youtube_dl/extractor/common.py", line 678, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/youtube_dl-2021.12.17-py2.7.egg/youtube_dl/YoutubeDL.py", line 2461, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

yt-dlp (uninstalled YouTube-dl, installed yt-dlp):

yt-dlp --user-agent "Mozilla/5.0'(Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0" --cookies-from-browser firefox --no-check-certificate -v https://motherless.com/043D161
[debug] Command-line config: ['--user-agent', "Mozilla/5.0'(Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0", '--cookies-from-browser', 'firefox', '--no-check-certificate', '-v', 'https://motherless.com/043D161']
Extracting cookies from firefox
[debug] Extracting cookies from: "/Users/iridium/Library/Application Support/Firefox/Profiles/a00swf8e.default-release/cookies.sqlite"
Extracted 24 cookies from firefox
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.12.30 from yt-dlp/yt-dlp [f10589e34] (pip)
[debug] Python 3.12.2 (CPython x86_64 64bit) - macOS-13.4-x86_64-i386-64bit (OpenSSL 3.2.1 30 Jan 2024)
[debug] exe versions: rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1798 extractors
[Motherless] Extracting URL: https://motherless.com/043D161
[Motherless] 043D161: Downloading webpage
ERROR: [Motherless] 043D161: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: Forbidden>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 718, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/motherless.py", line 113, in _real_extract
    webpage = self._download_webpage(url, video_id)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 1121, in _download_webpage
    return self.__download_webpage(url_or_request, video_id, note, errnote, None, fatal, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 1072, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 906, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 863, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 4082, in urlopen
    return self._request_director.send(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/networking/common.py", line 114, in send
    response = handler.send(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/networking/_helper.py", line 204, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/networking/common.py", line 325, in send
    return self._send(request)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/networking/_requests.py", line 343, in _send
    raise HTTPError(res, redirect_loop=max_redirects_exceeded)
yt_dlp.networking.exceptions.HTTPError: HTTP Error 403: Forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 850, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2023.12.30/libexec/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 4114, in urlopen
    raise _CompatHTTPError(e) from e
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 403: Forbidden

@dirkf
Copy link
Contributor

dirkf commented Feb 21, 2024

Unmatched quotes:

---user-agent "Mozilla/5.0'(Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0"
+--user-agent 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0'

Although it's not the problem here, definitely 'quote' your URLs for macOS.

@Iridium-Lo
Copy link
Author

I have never quoted urls (or anything in terminal) from various sites with YouTube-dl as I have IFS=$'\n' (split on newline not space) set in ~/.bash_profile.

Thanks, the stupid typo caused it to fail

yt-dlp:

yt-dlp https://motherless.com/03E0E46 --user-agent 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0' --cookies-from-browser firefox --no-check-certificate -v 
[debug] Command-line config: ['https://motherless.com/03E0E46', '--user-agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0', '--cookies-from-browser', 'firefox', '--no-check-certificate', '-v']
Extracting cookies from firefox
[debug] Extracting cookies from: "/Users/iridium/Library/Application Support/Firefox/Profiles/a00swf8e.default-release/cookies.sqlite"
Extracted 24 cookies from firefox
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.12.30 from yt-dlp/yt-dlp [f10589e34] (pip)
[debug] Python 3.12.2 (CPython x86_64 64bit) - macOS-13.4-x86_64-i386-64bit (OpenSSL 3.2.1 30 Jan 2024)
[debug] exe versions: rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1798 extractors
[Motherless] Extracting URL: https://motherless.com/03E0E46
[Motherless] 03E0E46: Downloading webpage
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: best/bestvideo+bestaudio
[info] 03E0E46: Downloading 1 format(s): 0
[debug] Invoking http downloader on "https://cdn5-videos.motherlessmedia.com/videos/03E0E46.mp4"
[download] Resuming download at byte 1047552
[download] Destination: Sexy Hot Cam.mp4 [03E0E46].mp4
[download]   2.7% of   41.57MiB at    1.33MiB/s ETA 00:30^C
ERROR: Interrupted by user

YouTube-dl:

youtube-dl https://motherless.com/03E0E46 --user-agent 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:123.0) Gecko/20100101 Firefox/123.0' --cookies ~/cookies.txt --no-check-certificate
[Motherless] 03E0E46: Downloading webpage
[download] Destination: Sexy Hot Cam.mp4-03E0E46.mp4
[download]   2.2% of 41.57MiB at  1.15MiB/s ETA 00:35

Is yt-dlp more advanced than YouTube-dl?
Any advantages/disadvantages between the two.

@Iridium-Lo
Copy link
Author

workaround

@Iridium-Lo
Copy link
Author

Iridium-Lo commented Feb 21, 2024

note: you will have to go to the site and let it verify you are human before using this workaround

@dirkf
Copy link
Contributor

dirkf commented Feb 21, 2024

Regarding yt-dl vs yt-dlp, the differences are well documented at the yt-dlp project GitHub site. tldr; yt-dlp gets more development but requires a new-ish Python, while people are able to and do run yt-dl with Py2.6; new stuff in yt-dl is normally pulled into yt-dlp, whereas yt-dl has a backlog of not-yet-back-ported changes and also won't incorporate certain too-difficult yt-dlp features.

While your shell is bash, quoting is mainly to prevent the shell from seeing the & that separates URL query parameters; in newer macOS people get zsh as shell, for which ? is significant too. My advice: save thought and effort by always quoting for 'POSIX' or "Windows cmd". You can find many examples in the tracker of people who have fallen foul of unquoted URLs, often without realising.

@dirkf dirkf added the broken-IE problem with existing site extraction label Feb 22, 2024
@Iridium-Lo
Copy link
Author

Iridium-Lo commented Feb 22, 2024

you need to look into using IFS=$'\n' & and other special characters are not affected when using no quotes on the terminal and in scripts, and when using youtube-dl.

If you are really worried about globbing then do set noglob I've been using this variable globally for years and have never had any issues with not quoting.

It's only a security risk if using bash for API stuff.

yt-dlp seems more convenient with it's cookies option it also gets the best quality without it being specified and does so in a special way. But I think it doesn't have as many devs on it than this, unless you are also maintaining it (you seem to be a big part of this).

Thank you for always helping, others would ignore the request because of the nature of the site, I see you are a big contributor to this, much appreciated.

@dirkf
Copy link
Contributor

dirkf commented Feb 23, 2024

Maintainers don't have to watch the content that's supported by the programme, at least not unless it's very compelling.

As to quoting, a non-default IFS would break my decades-old intuition of how the shell splits words and wouldn't be helpful for less experienced users.

@Iridium-Lo
Copy link
Author

true, about the less experienced users, whatever your comfortable with that's understandable, I've got a decade of experience to have IFS split on new line rather than space.

btw why is it necessary to get things like view count and other data like that? The code would be so much easier to work with without all that

@Iridium-Lo
Copy link
Author

Iridium-Lo commented Feb 25, 2024

Maintainers don't have to watch the content that's supported by the programme, at least not unless it's very compelling.

As to quoting, a non-default IFS would break my decades-old intuition of how the shell splits words and wouldn't be helpful for less experienced users.

@dirkf
err, that's not what I meant.
I mean most people would think it beneath them or just not want to help simply because of the nature of the site.
But there's no need to point out that you don't to watch the content, I didn't mention that, this seems a little defensive.
It's all good if you do I can recommend you some stuff :)

btw I have a solution for this in my bash script using curl to get the cookies, so I don't need to open the browser and run the Firefox extension. also have a python solution, there was a feature request you linked I think, I'll post in there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
broken-IE problem with existing site extraction
Projects
None yet
Development

No branches or pull requests

2 participants