Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incomplete proxy support #285

Open
3hhh opened this issue Apr 1, 2021 · 0 comments
Open

Incomplete proxy support #285

3hhh opened this issue Apr 1, 2021 · 0 comments

Comments

@3hhh
Copy link

3hhh commented Apr 1, 2021

This plugin only supports proxies directly via splash [1].

Using the native scrapy proxy support [2] leads to an attempt to contact splash via the configured proxy.

It would be more optimal to transparently pass the scrapy proxy arguments to splash as it's currently done with headers & cookies. Currently users have to write their own middleware for that.

[1] https://splash.readthedocs.io/en/stable/scripting-request-object.html?highlight=proxy#request-set-proxy
[2] https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants