v2.3.4

Use a different stat prefix for Zyte API proxy mode.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

2.3.3

Fix response handling for Zyte API proxy mode.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

2.3.2

Detect scenarios where the proxy Request.meta key has probably been accidentally copied from an earlier response, warn about it, and fix the value.

The Zyte-Client header is again sent when using Zyte API proxy mode, now that Zyte API supports it.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

2.3.1

Fixed support for the proxy mode of Zyte API.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

v2.3.0

Support for the upcoming proxy API of Zyte API.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

v2.2.0

Support Scrapy 2.6.2. Scrapy 1.4+ is now required.

v2.1.0

Use a custom logger instead of the root one

v2.0.0

Following the upstream rebranding of Crawlera as Zyte Smart Proxy Manager, scrapy-crawlera has been renamed as scrapy-zyte-smartproxy, with the following backward-incompatible changes:

• The repository name and Python Package Index (PyPI) name are now scrapy-zyte-smartproxy.
• Setting prefixes have switched from CRAWLERA_ to ZYTE_SMARTPROXY_.
• Spider attribute prefixes and request meta key prefixes have switched from crawlera_ to zyte_smartproxy_.
• scrapy_crawlera is now scrapy_zyte_smartproxy.
• CrawleraMiddleware is now ZyteSmartProxyMiddleware, and its default url is now http://proxy.zyte.com:8011.
• Stat prefixes have switched from crawlera/ to zyte_smartproxy/.
• The online documentation is moving to https://scrapy-zyte-smartproxy.readthedocs.io/

Note: Zyte Smart Proxy Manager headers continue to use the X-Crawlera- prefix.

• In addition to that, the X-Crawlera-Client header is now automatically included in all requests.

v1.7.0

• Added more stats to better understanding the internal states.
• Log warning when using https:// protocol.
• Add default http:// protocol in case of none provided, and log warning about it.
• Fix duplicated request when the response is not from crawlera, this was causing an infinite loop of retries when dont_filter=True.

v1.6.0

• Enable crawlera on demand by setting CRAWLERA_FORCE_ENABLE_ON_HTTP_CODES