Welcome to SPYDER a web crawler - Version 1.0.0
- This is a UNIQUE web crawler that only uses pure python!!
- ZERO Dependencies!!! Just a one time download and go security tool.
- Uses only modules from the python standard library.
USAGE:
- Execute the web crawler security tool by: py spyder.py [target-url]
- It should crawl most sites quickly within reason, or CTRL-C.
- Comes with a help menu type: py crawl.py -h.
NOTES: This is a command-line tool built for windows OS, when you crawl a website it returns the results in your command line. This web crawler can currently only crawl unencrypted web sites. Updates to come soon.