Add support for using Invidious through a HTTP Proxy

[syeopite]

Partially addresses #301

To use just set at least the host and port configuration here in the Invidious configs.

##
## Configuration for using a HTTP proxy
##
## If unset, then no HTTP proxy will be used.
## 
http_proxy:
  user:
  password:
  host:
  port:

[syeopite]

Now that there's "proper" support for HTTP proxies, I suppose Invidious should also be removing its own proxy implementation at HTTPClient and HTTPProxy

[SamantazFox]

Won't that collide with the code in src/invidious/yt_backend/proxy.cr?
It might be a good idea to remove all of that proxy code (the IP list hasn't been updated in ages anyway) and let the instance owner use their own.

[ms-jpq]

I think it might be helpful to add the http proxy for pulling in reddit comments as well, provided it s not alot more work on top of this.

Reddit just recently IP banned alot of VPS IP ranges, so currently the easiest work around is to tunnel things either via TOR or cloudflare warp

[syeopite]

Reddit comments should already be fetched through the proxy in this PR

[mk-pmb]

Is this patch meant to also address the install step that says

Checking player dependencies, this may take more than 20 minutes... If it is stuck, check your internet connection.

?

Edit: It might. At least this time the error is about npm, and I can probably figure out how to configure npm's proxy settings inside.
Edit 2: Seems we don't use proper npm but instead download directly in fetch-player-dependencies.cr line 86.
Edit 3: With exotic network setups we shoud just use proper npm for downloading: #4361

[syeopite]

The part that fetches the videojs dependencies is technically not a part of the main Invidious program so requests in that don't go through the configured proxy.

[unixfox]

The part that fetches the videojs dependencies is technically not a part of the main Invidious program so requests in that don't go through the configured proxy.

Maybe we should? If one configure a proxy he expect all the requests to go through the proxy.

[unixfox]

Please use the edit button, don't triple post...

Please don't go offtopic, create a separate issue for your npm discussion.

[mk-pmb]

I researched a bit more about how to best pass proxy settings to the build process and found that doing it securely (not revealing proxy hostname and credentials in the built image) is more complicated than I had hoped. So build-time proxy stuff should best be left for a 3rd-party project.

@syeopite, I found a few more things that I'd prefer switched around in git history, and some whitespace that seems accidential. Please allow me to propose this alternative history for the same effect as your current PR. The last commit is just so you can easily verify that files in both branches are otherwise identical. If you like, feel free to hard-reset your branch to my proposal's 2nd commit.

Does Crystal have an elegant way to use a common function to de-duplicate the conn.proxy = … assignments?
A project-wide function to set default config for all HTTP(S) connections might also turn out to be useful if one day we want to configure more aspects than just proxy.

[SamantazFox]

Could you please rebase your changes?

[syeopite]

It seems like the upstream library no longer works on Crystal 1.12

[ChunkyProgrammer]

@syeopite https://github.com/mamantoha/http_proxy/blob/master/CHANGELOG.md#0102 I think you might be able to update the library to a newer version for the crystal 1.12 support

[syeopite]

Oops! I didn't see the newest update 😅

[SamantazFox]

Does it work with other crystal versions? (at least with the version matrix used in our CI pipeline)

[syeopite]

Yep the initialize function wasn't touched