Webcrawler for incoming links

Requirements

NPM, Node, MySQL

Clone this repo
Run npm install
Create mysql table and modify it in ./db.js
Run node crawler.js --install
Add a first website (e.g. http://news.ycombinator.com) with empty status to Table sites

node crawler.js

Name		Name	Last commit message	Last commit date
Latest commit History 55 Commits
config		config
dashboard		dashboard
data		data
exports		exports
.dockerignore		.dockerignore
.gitignore		.gitignore
Dockerfile		Dockerfile
counter.txt		counter.txt
crawler.js		crawler.js
db.js		db.js
docker-compose.yml		docker-compose.yml
package-lock.json		package-lock.json
package.json		package.json
readme.md		readme.md
watchdog.js		watchdog.js