Documentation for Isoxya web crawler.
Isoxya is an extensible data processing system for crawling and scraping the internet. It is written in a compiled, statically typed language for speed and reliability. Isoxya is free, open-source, and packaged as a container. The API can be used by CLI scripts or called by another project. Various plugins are available, many also open-source.
https://www.isoxya.com/
https://hub.docker.com/r/tiredpixel/isoxya-api
https://github.com/tiredpixel/isoxya-api
Isoxya Pro adds high availability, error recovery, and horizontal scaling. It is available on-premises for installation on your servers, or in the cloud via SaaS subscription. Isoxya Pro is available via commercial licence. The API is able to scale crawlers, processors, and streamers. Support or custom development is available from Isoxya's creator.
To get started, follow the tutorial.
- / GET
- /list/:list_id DELETE (Pro)
- /list/:list_id GET (Pro)
- /list/:list_id/list_page POST (Pro)
- /processor GET
- /processor POST
- /processor/:processor_id DELETE
- /processor/:processor_id GET
- /site POST
- /site/:site_id GET
- /site/:site_id/crawl GET
- /site/:site_id/crawl POST
- /site/:site_id/crawl/:site_v GET
- /site/:site_id/crawl/:site_v PATCH (Pro)
- /site/:site_id/crawl/:site_v/crawl GET (Pro)
- /site/:site_id/crawl/:site_v/list GET (Pro)
- /site/:site_id/list GET (Pro)
- /site/:site_id/list POST (Pro)
- /streamer GET
- /streamer POST
- /streamer/:streamer_id DELETE
- /streamer/:streamer_id GET
Copyright © Nic Williams. It is free software, released under the BSD 3-Clause licence, and may be redistributed under the terms specified in LICENSE.