Warning
This is a prototype repository undergoing rapid changes. It's not intended for production use yet.
This repository hosts the AI worker and runner for processing inference requests on the Livepeer AI subnet.
The AI worker repository includes:
-
Runner: The AI runner, a containerized Python application, processes inference requests on Livepeer AI's Pipelines and models, providing a REST API for model interaction.
-
Worker: The AI worker allows the ai-video branch of go-livepeer to interact with the AI runner. It includes golang API bindings, a worker for routing inference requests, and a Docker manager for AI runner containers.
The AI runner's code is in the runner directory. For more details, see the AI runner README.
The AI worker's code is in the worker directory. It includes:
- Golang API Bindings: Generated from the AI runner's OpenAPI spec using
make codegen. - Worker: Listens for inference requests from the Livepeer AI subnet and routes them to the AI runner.
- Docker Manager: Manages AI runner containers.
The AI worker and runner are designed to work with the ai-video branch of go-livepeer. You can run both independently for testing. To build the AI worker locally and run examples, follow these steps:
- Follow the README instructions in the runner directory to download model checkpoints and build the runner image.
- Generate Go bindings for the runner OpenAPI spec with
make codegen. - Run any examples in the
cmd/examplesdirectory, e.g.,go run cmd/examples/text-to-image/main.go <RUNS> <PROMPT>.
For more on developing and debugging the AI runner, see the development documentation.