A Scrapy middleware to bypass the CloudFlare's anti-bot protection, based on cloudflare-scrape.
pip install scrapy_cloudflare_middleware
Add the middleware to your DOWNLOADER_MIDDLEWARES settings
DOWNLOADER_MIDDLEWARES = {
# The priority of 560 is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.
'scrapy_cloudflare_middleware.middlewares.CloudFlareMiddleware': 560
}Done. Happy scraping !