Contains scripts used to learn, test and explore web spiders or crawlers, which are tools used to index / explore various web sites and content.
-
goodreads_quotes_spider.py - Search for quotes using keywords. For eg. harry potter swear results in 'I solemnly swear I am upto no good.' Uses content hosted on goodreads.com
-
verge_newsfeed_spider.py - Generates text news feed from the title page of The Verge Uses content hosted on theverge.com
-
vtech_ece_proflist.py - Generates a List of professors and their reserach interests. Uses content hosted on ece.vt.edu