Toastie is a basic web spider written in Python that scans webpages' content and enters it into a database. The front-end of the website allows the user to search through scanned pages.
Python, Flask, Werkzeug, Jinja, Requests, urlparse, re, robotparser, pymongo, MongoDB, Twitter Bootstrap
Install python2.7, pip, mongodb, and the python-mongodb connector
sudo pip install Virtualenv
git clone http://www.github.com/gmiller2007/Py-Webscraper
source bin/activate
bin/pip install flask
bin/pip install pymongo
bin/pip install BeautifulSoup
deactivate
source bin/activate
bin/python2.7 application.py
To safely close the virtual environment run the command 'deactivate'