[Bug] Can't load the video on this Invidious instance. YouTube is currently trying to block Invidious instances.

[TheEdgeOfRage]

EDIT: Switched title from Constant 403 errors while trying to load videoplayback to Can't load the video on this Invidious instance. YouTube is currently trying to block Invidious instances. in order to reflect the current error message.

---

EDIT by @unixfox: It appears that youtube is trying to block the popular invidious instances by returning 403 if the video stream is loaded through the invidious instance (with proxy or dash enabled).

Alternative solution is to switch off proxy or use another instance that is unblocked.

To public instances maintainers: #4045 (comment)

---

Describe the bug
I host my own invidious instance and some videos just won't load on it. The thumbnail loads, but the video itself does not. There's just a spinner and the timeline says 0:00/0:00. This does not happen with all videos, but I haven't spotted a pattern yet.

When looking at the network tab, I can see that the browser is constantly sending requests to /videoplayback and getting a 403 status with no response body. It sends about 10 requests per second and doesn't stop until I close the tab. For reference, I use DASH on my instance.

Steps to Reproduce

1. Go to https://vid.theedgeofrage.com/watch?v=RBo1Mq1JMh8
2. Try to play video
3. See error

Logs
Logs don't show anything besides the failing request log itself:

invidious-invidious-1     | 2023-08-10 09:50:50 UTC [info] 403 GET /videoplayback?expire=1691682024&ei=iLDUZJnwFNCOx_APssW3qAM&ip=94.130.141.40&id=o-ALEojr5Ke8EW5dc7Xuq-3wksylUWHvc81JCGu0Jw5dkp&itag=136&source=youtube&requiressl=yes&mh=Co&mm=31%2C26&mn=sn-4g5e6nzz%2Csn-f5f7kn7z&ms=au%2Conr&mv=m&mvi=4&pl=22&initcwndbps=445000&spc=UWF9f3uYf0Q_K1VEYIgDmPBkGM6dxrZqXRGhYOsRhw&vprv=1&svpuc=1&mime=video%2Fmp4&gir=yes&clen=438730931&dur=3145.775&lmt=1691291183341453&mt=1691660003&fvip=2&keepalive=yes&fexp=24007246&beids=24350018&c=ANDROID&txp=3309224&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgLzQ6H_s28Scvwg6naQLHHEGK_-68OgYfih7J5xXWEdwCIQDdIlh8g6r6ZjTwl-kV2p4qAlFpOS0K6wzm0WBKq4wJsA%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIhAOBrXV03okr0vNFfd6sxZ-7xvwhPg1UNe209RUtr9O_wAiBaZWVNy_C4QhxWpOxZ1IP67fbfVGc03skcndJdI_ef2Q%3D%3D&host=rr4---sn-4g5e6nzz.googlevideo.com 50.29ms

Screenshots

Additional context

• Browser: Firefox 116
• Desktop OS: Linux
• Invidious deployed through docker, latest version from 2023-08-07

docker-compose.yml file:

version: "3"
services:
 invidious:
   image: quay.io/invidious/invidious:latest
   restart: unless-stopped
   ports:
     - "127.0.0.1:8030:3000"
   environment:
     # Please read the following file for a comprehensive list of all available
     # configuration options and their associated syntax:
     # https://github.com/iv-org/invidious/blob/master/config/config.example.yml
     INVIDIOUS_CONFIG: |
       db:
         dbname: invidious
         user: kemal
         password: REDACTED
         host: invidious-db
         port: 5432
       check_tables: true
       hmac_key: REDACTED
       external_port: 443
       domain: vid.theedgeofrage.com
       https_only: true
       statistics_enabled: false
       use_quic: true
       popular_enabled: false
       captcha_enabled: false
       admins: [TheEdgeOfRage]
       channel_refresh_interval: 10m
       dark_mode: dark
       feed_menu: ["Subscriptions", "Playlists"]
       default_home: Subscriptions
       comments: ["youtube", "reddit"]
       quality: dash
       quality_dash: 1440p
       unseen_only: true
   healthcheck:
     test: wget -nv --tries=1 --spider http://127.0.0.1:3000/api/v1/comments/jNQXAC9IVRw || exit 1
     interval: 30s
     timeout: 5s
     retries: 2
   logging:
     options:
       max-size: "1G"
       max-file: "4"
   depends_on:
     - invidious-db

 invidious-db:
   image: docker.io/library/postgres:14
   restart: unless-stopped
   volumes:
     - postgresdata:/var/lib/postgresql/data
     - ./config/sql:/config/sql
     - ./docker/init-invidious-db.sh:/docker-entrypoint-initdb.d/init-invidious-db.sh
   environment:
     POSTGRES_DB: invidious
     POSTGRES_USER: kemal
     POSTGRES_PASSWORD: REDACTED
   healthcheck:
     test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]

volumes:
 postgresdata:

[unixfox]

Please post the invidious version (bottom of the page) and check if you are up to date.

[TheEdgeOfRage]

I'm running the latest image, from 3 days ago

[unixfox]

Please post the version anyway thank you.

And the video ID where you have these errors.

[swolford]

I'm chiming in with +1 on this issue - 403 on videoplayback? route. I'm having the same, except not as constantly.

Container logs say the same:

2023-08-11 06:44:55 UTC [info] 403 GET /videoplayback?expire=1691757657&ei=-dfVZM_TAvC9x_APvce1wA4&ip=REDACTED&id=o-AIiUCaC5r4PPv9NR8x2HCsd90e5z0F7lKnoyglgAp4Jx&itag=398&source=youtube&requiressl=yes&mh=jt&mm=31%2C29&mn=sn-32o-1the%2Csn-5hnekn7d&ms=au%2Crdu&mv=m&mvi=2&pl=26&initcwndbps=3022500&spc=UWF9f8N6p4VTO2oChYGdB3gnM1LBkKYXrWmuZCCZBw&vprv=1&svpuc=1&mime=video%2Fmp4&gir=yes&clen=46003166&dur=857.880&lmt=1691441992636539&mt=1691735604&fvip=5&keepalive=yes&fexp=24007246&c=ANDROID&txp=5537434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgKG7B8BX1QxXs7KG6F3AP5EBDGY9ZLpJYLMc4EWBsi5cCIQC_FtCjYUiBmORbDPliDnDvBgX8Ydim7OPtZj2Tjq2Vbg%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgekKYcP9okaaUSpSym8cuxU-QEXgRj8hdYhUZEeTOGi4CIQCz6Yk5ChMi5Nke81ZAv6TyOhdakvNoMf71sRSVAAUksw%3D%3D&host=rr2---sn-32o-1the.googlevideo.com 30.69ms

Version:
2023.08.07-3450896 @ master

Affected IDs:
umpuS0jJs-s
nQcPuFmA7XU
ZPSqhb4KKNc
Yi0WRF5WPFw

[TheEdgeOfRage]

@swolford funnily enough, all of those videos play on my instance. And whenever I encounter the 403 error, I just hit the "switch invidious instance" and they pretty much always work on another one.

Also, I just tried the video that was failing on my instance yesterday and it's working now. So it's definitely just an intermittent issue. But once it starts failing on an instance, it keeps happening even after hard reloading, restarting the instance and even updating it. I think this might be related to the communication to youtube itself

[syeopite]

I'm guessing it might be a cache issue.

YouTube recently implemented a change that broke playback as various streaming urls returned will just yield a 403 instead of actual video content. See #4027

A workaround was added in #4037 which was merged on August 7th.

However, due to the video cache, streaming urls from before the fix are still being used. The aforementioned video working now is probably because the cache got cleared.

[TheEdgeOfRage]

I guess I'll leave this open for a couple days then and see if the error resurfaces. If not, I'll close this issue

[blackandcold]

I have it with almost all videos on my personal instance. Latest image also.
Will report if it does not vanish with cache empty.

Edit: all new videos do not work - very strange!
403 all the way

[blackandcold]

Rebuilding the Dockerimage and running docker-compose again made it work for now.
The patch was integrated 3 days ago.

[unixfox]

I'm guessing it might be a cache issue.

I don't think so the cache is only kept for 6 hours max and refresh by invidious after that.

[foodornt]

I can affirm that this issue occurred in previous versions, rebuilding (through docker-compose build) and restarting brought it back to normal

[TheEdgeOfRage]

I haven't encountered the issue at all in the past few days, so I'll close this as resolved.

[mortezadadgar]

I'm getting this error on https://iv.nboeck.de, https://vid.puffyan.us, https://invidious.io.lol, https://invidious.no-logs.com instances
youtube URL: https://www.youtube.com/watch?v=KtQZRAkgLqo

[unixfox]

Reopening as youtube appears to straight block the IP addresses of the popular public instances.

[unixfox]

The issue #3915 still applies for discussing about how to circumvent youtube blockage.

Please see how to circumvent the blockage: #4045 (comment)

[gi-yt]

The issue #3915 still applies for discussing about how to circumvent youtube blockage.

This comment still apply for solutions on how to circumvent the youtube blockage: #3822 (comment)

Youtube might be blocking subnets, I get a 403 on IPv6es in both the 16 IPv6 and routed subnets of BuyVM Luxembourg.
I also have the same issue on Racknerd Los Angeles, but I think that is because of the v6 being in the AAAA record, I'll update here after getting a secondary v6 from them.

I also will try (re)setting up the ipv6 thing on my home network and trying with that, to see if that will help.

The interesting part in all this is that Piped continues to work, with all the IPv6es I threw at it after this wave of blockages..

[unixfox]

Hello to all public instances owners or people that are running a private Invidious instance that is blocked!

We have officially released a tool that allow to automatically rotate the IPv6 address of your Invidious server: https://github.com/iv-org/smart-ipv6-rotator and thus escape from the YouTube block!

You can follow the detailed tutorial here: https://docs.invidious.io/ipv6-rotator/

Please give your feedback, it is very important to us! For any help related to this tool, please comment here: #3915 or on matrix or on IRC.

[gptlang]

I followed all the instructions here: https://docs.invidious.io/ipv6-rotator/

root@invidious ~/invidious [1]# docker-compose exec invidious ping -c 1 ipv6.icanhazip.com
PING ipv6.icanhazip.com (2606:4700::6812:7361): 56 data bytes
64 bytes from 2606:4700::6812:7361: seq=0 ttl=58 time=1.799 ms

But

root@invidious ~/invidious# sudo lsof -i
COMMAND     PID            USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
nginx       660        www-data   27u  IPv4 178042      0t0  TCP invidious:http->172.71.250.19:11990 (ESTABLISHED)
nginx       660        www-data   44u  IPv4 178047      0t0  TCP invidious:http->172.71.250.19:11978 (ESTABLISHED)
nginx       660        www-data   58u  IPv4 176403      0t0  TCP invidious:http->172.71.250.19:62074 (ESTABLISHED)
docker-pr 16742            root    4u  IPv4 180586      0t0  TCP localhost:3000 (LISTEN)

I'm still connected over IPv4

root@invidious ~/invidious# docker-compose exec invidious getent ahostsv6 google.com
2a00:1450:400e:811::200e STREAM google.com
2a00:1450:400e:811::200e DGRAM  google.com

IPv6 is the default though

[unixfox]

We have fixed an issue where invidious would not send all the traffic in ipv6: #4225
Please add this line in your config.yml to be sure that invidious send the traffic in ipv6:

force_resolve: ipv6

If you are using the rotator (https://docs.invidious.io/ipv6-rotator/), please make sure to have the line above and have updated Invidious!
We have reflected the documentation for this new change.