scrapers is a collection of free/libre open-source software written by Aarón Montoya-Moraga.
scrapers is both a tool for building databases and and educational resource for learning scraping.
scrapers is educational because it tries to be heavily documented, clean, and easy to follow.
scrapers performs the scraping in an explicit way, it shows you the browser going through the data, instead of running in the background, thus being very open in the way it works, which can be used for both documentation and live performance.
All of these scrapers were written using Python, Selenium, and ChromeDriver.
- bing-images
- captcha
- google-images
- mugshots
- Install Python2 and Python3
- Install Homebrew if in Mac
- Install Chromedriver
- Thanks to Sam Lavigne and their class Detourning the web.
- Thanks to my classmates at School of Machines that helped me test and prototype some of these scrapers.
MIT