Scrapy, a fast high-level web crawling & scraping framework for Python.

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Fill HTML login forms automatically

CSS Selectors for Python

Python library of web-related functions

Common interface for data container classes

Library to populate items using XPath and CSS with a convenient API

Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

A pure-Python robots.txt parser with support for modern conventions.

The scrapy.org website