New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Block IP from crawlers #107

Open

gtalarico opened this issue Mar 15, 2017 · 1 comment

Labels

Owner

gtalarico commented Mar 15, 2017

Server is getting slammed with crawlers trying to find stuff + 404 errors
Gunicorn Server Hooks
http://stackoverflow.com/questions/40951861/how-to-use-variables-created-in-gunicorns-server-hooks

Gunicorn Docs
http://docs.gunicorn.org/en/stable/settings.html#server-hooks

Crawler Blacklist:
https://myip.ms/browse/blacklist/Blacklist_IP_Blacklist_IP_Addresses_Live_Database_Real-time

Add custom rule ?if crawler to hit /blog/ block ip

gtalarico added the performance label

Owner Author

gtalarico commented May 4, 2017

http://esd.io/blog/flask-apps-heroku-real-ip-spoofing.html

gtalarico mentioned this issue

More errors caused by invalid urls #111

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment