pyrobotstxt package can be used to (systematically) generate robots.txt files. Moreover, this package comes in handy for creating and including ASCII images in robots.txt files.
In future releases, it would be possible to parse and analyze robots.txt file generated using any software (not limited to pyrobotstxt)
We believe in monolithic software development and created this tiny package that does its job without any bloat. It is useful for
- Createing robots.txt File
- Parsing a robots.txt File [in progress]
- Analyzing robots.txt File [in progress]
You can find detailed tutorial on pyrobotstxt tutorial website. You can consult pyrobotstxt documentation website for API reference.
Pull Requests, Feature Suggestions, and collaborations are welcome.
This work is a collaborative effort of seowings, and serpwings.