Home
Paul Tremberth edited this page Apr 27, 2017
·
107 revisions
See also: Scrapy homepage, Official documentation, Scrapy snippets on Snipplr
If you're new to Scrapy, start by reading Scrapy at a glance.
Check https://scrapy.org/resources/.
Check https://scrapy.org/resources/.
Check https://scrapy.org/resources/.
- Python Scrapy: Scrape web data using Python
- Scrapy: it GETs the web - PyCon US 2013 talk
- Installing Scrapy on Windows (video tutorial)
- Recursively scraping Craigslist (includes video) - Nov 8, 2012
- Scraping the Web with Scrapy
- Karthik Ananth: Scrapy Workshop
- Scrapy / Python playlist on Youtube channel
English slides:
- Scrapy - a flexible crawler to power your search - give by Shane Evans in Feb 2013 Cambridge Search Meetup
- Web Crawling & Metadata Extraction in Python
- Crawling the web for fun and profit
- Scrapy for dummies
- Web scraping 1 2-3 with python + scrapy (Summer BarCampHK 2012 version)
- Collecting web information with open source tools
- When big data meet python @ COSCUP 2012
- How to scrape any website's content using Scrapy
Spanish slides:
Chinese slides:
Portuguese Slides:
- Django Dynamic Scraper - a web application (written in django) for runnning and controlling Scrapy spiders
- Slybot - A supervised learning crawler based on Scrapely
- scrapy-sentry - Logs Scrapy exceptions into Sentry
- ScrapyGraphite - Output scrapy statistics to carbon/graphite
- scrapy-mongo - A pipeline to store scrapy items in a MongoDB database
- scrapy-boilerplate - small set of utilities to simplify writing low-complexity spiders
- scrapy-inline-requests - provides a decorator to write spider callbacks which performs multiple requests without the need to write multiple callbacks for each request
- scrapy-redis - providesRedis-backed components for Scrapy
- scrapyz - Create simple spiders easily.
- Scrapy-related libraries on PyPI
- Scrapy_cn - provided a demo to solve encoding problems(utf-8).
- elite-proxies-scrapy-middleware - get new proxies from your EliteProxies account
- scrapydo - Crochet-based blocking API for Scrapy.
See http://scrapy.org/companies/
- see Release notes in the official documentation
- SEPs are available in scrapy/sep.