This is a curated list of awesome webscraping technical blogs and vidoes from research to deployment. Feel free to add more on this topic
- Scrapy Course – Python Web Scraping for Beginners - freeCodeCamp.org
- Scaling Web Scraping: Concurrency vs Parallelism
- Scrapy Redis: Scaling Your Scrapy Spiders Using Distributed Worker
- Accelerating Data-On-Demand Services With Async Python and AWS - OxyCon
- Scrapy Crawl Spider - A Complete Guide
- Talk - Josh Weissbock: Distributed Web Scraping in Python - PyCon US 2022
- Josh Weissbock: Distributed Web Scraping in Python - PyCascades 2019
- Django + Celery + Selenium to Scrape Anything with Python - CodingEntrepreneurs YT
- Building a scalable focused web crawler with Flink - Ken Krugler - Flink Forward San Francisco, April 2018
- Building a Web Crawler from Scratch in Node.js - Youtube (Boot dev)
- ScrapeCon 2024-Youtube Playlists | ScrapeCon | BrightData
- Effective Ways to Scale-Up and Maintain Your Web Crawling Project - Kevin Lloyd Bernal|PyCon APAC 2022