`yt-dlp -x $URL` not working: 404 not found error #9965

jaanli · 2024-05-19T15:44:48Z

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

I'm reporting a bug unrelated to a specific site
I've verified that I have updated yt-dlp to nightly or master (update instructions)
I've checked that all provided URLs are playable in a browser with the same IP and same login details
I've checked that all URLs and arguments with special characters are properly quoted or escaped
I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood

Tried to run this to extract audio but it didn't work :(

yt-dlp -x "https://www.youtube.com/watch\?v\=nJKuaASTx1g"
[generic] Extracting URL: https://www.youtube.com/watch\?v\=nJKuaASTx1g
[generic] watch\?v\=nJKuaASTx1g: Downloading webpage
ERROR: [generic] Unable to download webpage: HTTP Error 404: Not Found (caused by <HTTPError 404: Not Found>)

Any advice on how to debug?

P.S. I tried to update because this checklist requires this preflight, to the nightly, but got these errors:

# To update to nightly from stable executable/binary:
yt-dlp --update-to nightly

# To install nightly with pip:
python3 -m pip install -U --pre yt-dlp[default]

Errors:

Provide verbose output that clearly demonstrates the problem

Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
If using API, add 'verbose': True to YoutubeDL params instead
Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

me@milaptop   ~  yt-dlp -vU -x "https://www.youtube.com/watch\?v\=nJKuaASTx1g"                                                                                 1 ↵  10000  11:42:53 
[debug] Command-line config: ['-vU', '-x', 'https://www.youtube.com/watch\\?v\\=nJKuaASTx1g']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.04.09 from yt-dlp/yt-dlp [ff0779267] (pip)
[debug] Python 3.12.3 (CPython x86_64 64bit) - macOS-14.4.1-x86_64-i386-64bit (OpenSSL 3.3.0 9 Apr 2024)
[debug] exe versions: ffmpeg 7.0 (setts), ffprobe 7.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.3, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.04.09 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.04.09 from yt-dlp/yt-dlp)
[generic] Extracting URL: https://www.youtube.com/watch\?v\=nJKuaASTx1g
[generic] watch\?v\=nJKuaASTx1g: Downloading webpage
ERROR: [generic] Unable to download webpage: HTTP Error 404: Not Found (caused by <HTTPError 404: Not Found>)
  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 734, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/extractor/generic.py", line 2376, in _real_extract
    full_response = self._request_webpage(url, video_id, headers=filter_dict({
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 903, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 890, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query, extensions))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 4131, in urlopen
    return self._request_director.send(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/networking/common.py", line 115, in send
    response = handler.send(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/networking/_helper.py", line 208, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/networking/common.py", line 335, in send
    return self._send(request)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/yt-dlp/2024.4.9_1/libexec/lib/python3.12/site-packages/yt_dlp/networking/_requests.py", line 350, in _send
    raise HTTPError(res, redirect_loop=max_redirects_exceeded)
yt_dlp.networking.exceptions.HTTPError: HTTP Error 404: Not Found

The text was updated successfully, but these errors were encountered:

bashonly · 2024-05-19T15:53:31Z

yt-dlp -x "https://www.youtube.com/watch\?v\=nJKuaASTx1g"

Your terminal is auto-escaping special characters (adding \ before ? and =). If it does this, then you can't quote the URL, because the literal \ will be passed to yt-dlp as part of the URL.

You can only do one or the other -- either quote the URL or escape the special characters with \

jaanli · 2024-05-19T15:54:29Z

Hmm interesting, I didn't see this mentioned in the docs. Let me try it, thank you!

bashonly · 2024-05-19T15:56:28Z

It's not mentioned in the docs because it's the behavior of your terminal/shell and doesn't have anything to do with yt-dlp specifically

jaanli · 2024-05-19T16:01:12Z

Ah okay maybe I'm confused then thank you @bashonly , I just tried escaping the special character but got this:

❯ yt-dlp -x 'https://www.youtube.com/watch\?v\=nJKuaASTx1g'
[generic] Extracting URL: https://www.youtube.com/watch\?v\=nJKuaASTx1g
[generic] watch\?v\=nJKuaASTx1g: Downloading webpage
ERROR: [generic] Unable to download webpage: HTTP Error 404: Not Found (caused by <HTTPError 404: Not Found>)

Not sure what I'm doing wrong.

bashonly · 2024-05-19T16:05:15Z

you are still quoting and escaping

remove the quotes

or

remove the backslashes

DavidPesticcio · 2024-05-19T16:55:14Z

If you're doing this on Linux, MacOS, or WSL you don't need to escape or quote anything in that particular URL.

This may be of help to you: https://www.gnu.org/software/bash/manual/html_node/Quoting.html

Works fine for me... 🤷 😄

$ yt-dlp -x https://www.youtube.com/watch?v=nJKuaASTx1g
[youtube] Extracting URL: https://www.youtube.com/watch?v=nJKuaASTx1g
[youtube] nJKuaASTx1g: Downloading webpage
[youtube] nJKuaASTx1g: Downloading ios player API JSON
[youtube] nJKuaASTx1g: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "nJKuaASTx1g")
[youtube] nJKuaASTx1g: Downloading m3u8 information
[info] nJKuaASTx1g: Downloading 1 format(s): 251
[download] Destination: Nite Jewel - Puisque bele dame m'eime [nJKuaASTx1g].webm
[download] 100% of    2.67MiB in 00:00:01 at 2.48MiB/s
[ExtractAudio] Destination: Nite Jewel - Puisque bele dame m'eime [nJKuaASTx1g].opus
Deleting original file Nite Jewel - Puisque bele dame m'eime [nJKuaASTx1g].webm (pass -k to keep)

bashonly · 2024-05-19T16:58:09Z

If you're doing this on Linux, MacOS, or WSL you don't need to escape or quote anything in that particular URL.

Depends on what shell is being used. E.g. if it's zsh, the ? does need to be escaped or quoted

DavidPesticcio · 2024-05-19T17:01:08Z

If you're doing this on Linux, MacOS, or WSL you don't need to escape or quote anything in that particular URL.

Depends on what shell is being used. E.g. if it's zsh, the ? does need to be escaped or quoted

So in this instance, it's a case of using single quotes around the original URL - not double quotes @jaanli 😉

seproDev · 2024-05-19T17:04:51Z

No. Both single and double quotes work in zsh.

yt-dlp -x 'https://www.youtube.com/watch?v=nJKuaASTx1g'
yt-dlp -x "https://www.youtube.com/watch?v=nJKuaASTx1g"
yt-dlp -x https://www.youtube.com/watch\?v\=nJKuaASTx1g

Any of these work. Just not escaping the characters and quoting.

DavidPesticcio · 2024-05-19T17:32:15Z

@jaanli - FYI:

How to use a special character as a normal one in Unix shells?

https://unix.stackexchange.com/a/296147/131024

jaanli · 2024-05-20T01:47:25Z

Thanks for all the help! Truly surprised it is this hard on a MacBook Pro in 2024 :)

what are the different factors that cause these constellation of issues to be very difficult to articulate in documentation?

Is it really just the escaping? Has anyone seen a UML/flowchart of how escaping algorithms work (or don’t)?

I’ll give these solutions a shot when I’m able - if anyone is able to reproduce please let me know!

bashonly · 2024-05-20T01:49:48Z

There is a lot of noise in this issue that you can ignore.

The TL;DR: your terminal automatically escapes special characters. so just don't add quotes around the URL

If it's still unclear, then see the examples that seproDev gave; any of these should work

jaanli · 2024-05-20T13:30:40Z

Hmm @bashonly I tried that but it didn't work :(

~/projects  🕙 226ms
󰂃 4% ❯ yt-dlp -x https://www.youtube.com/watch?v=nJKuaASTx1g
fish: No matches for wildcard 'https://www.youtube.com/watch?v=nJKuaASTx1g'. See `help wildcards-globbing`.
yt-dlp -x https://www.youtube.com/watch?v=nJKuaASTx1g
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

Any other advice?

seproDev · 2024-05-20T13:35:52Z

@jaanli Now you did neither. Either quote the URL or escape the special characters. You need to do one or the other, otherwise your shell will interpret the special characters. If you do both then it also won't work since then you are passing the literal string containing \? to yt-dlp.

jaanli · 2024-05-20T13:40:09Z

Ah that was it, thank you @seproDev !

yt-dlp -x https://www.youtube.com/watch\?v=nJKuaASTx1g

pukkandan · 2024-05-21T16:17:17Z

Double quoting seems to be a pretty common mistake. We should see if we can catch it in commonmistakes.py

jaanli added bug Bug that is not site-specific triage Untriaged issue labels May 19, 2024

bashonly added question Question and removed bug Bug that is not site-specific triage Untriaged issue labels May 19, 2024

jaanli closed this as completed May 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`yt-dlp -x $URL` not working: 404 not found error #9965

`yt-dlp -x $URL` not working: 404 not found error #9965

jaanli commented May 19, 2024

bashonly commented May 19, 2024

jaanli commented May 19, 2024

bashonly commented May 19, 2024

jaanli commented May 19, 2024

bashonly commented May 19, 2024

DavidPesticcio commented May 19, 2024 •

edited

bashonly commented May 19, 2024

DavidPesticcio commented May 19, 2024

seproDev commented May 19, 2024 •

edited

DavidPesticcio commented May 19, 2024

How to use a special character as a normal one in Unix shells?

jaanli commented May 20, 2024

bashonly commented May 20, 2024 •

edited

jaanli commented May 20, 2024

seproDev commented May 20, 2024

jaanli commented May 20, 2024

pukkandan commented May 21, 2024

yt-dlp -x $URL not working: 404 not found error #9965

yt-dlp -x $URL not working: 404 not found error #9965

Comments

jaanli commented May 19, 2024

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

Checklist

Provide a description that is worded well enough to be understood

Provide verbose output that clearly demonstrates the problem

Complete Verbose Output

bashonly commented May 19, 2024

jaanli commented May 19, 2024

bashonly commented May 19, 2024

jaanli commented May 19, 2024

bashonly commented May 19, 2024

DavidPesticcio commented May 19, 2024 • edited

bashonly commented May 19, 2024

DavidPesticcio commented May 19, 2024

seproDev commented May 19, 2024 • edited

DavidPesticcio commented May 19, 2024

How to use a special character as a normal one in Unix shells?

jaanli commented May 20, 2024

bashonly commented May 20, 2024 • edited

jaanli commented May 20, 2024

seproDev commented May 20, 2024

jaanli commented May 20, 2024

pukkandan commented May 21, 2024

`yt-dlp -x $URL` not working: 404 not found error #9965

`yt-dlp -x $URL` not working: 404 not found error #9965

DavidPesticcio commented May 19, 2024 •

edited

seproDev commented May 19, 2024 •

edited

bashonly commented May 20, 2024 •

edited