pi-genai-stack

• The main objective is to run 🦙 Ollama and 🐬 TinyDolphin on a Raspberry Pi 5 with 🐳 Docker Compose.
• The stack provides development environments to experiment with Ollama and 🦜🔗 Lanchain without installing anything:
  • Python dev environment (available)
  • JavaScript dev environment (available)

How to install the stack on the Pi

• The compose file use the include feature, so you need at least the 2.21.0 version

git clone https://github.com/bots-garden/pi-genai-stack.git
cd pi-genai-stack

Models (LLM)

At start, Pi GenAI Stack will download 7 models:

• TinyDolphin
• TinyLlama + TinyLlama Chat
• DeepSeek Coder + DeepSeek Coder Instruct
• Gemma + Gemma Instruct

Then all the samples of the demos will use one of these models.

Run the samples of the python-demo

start the stack with the demo

docker compose --profile demo up

Use the python with the interactive mode:

docker exec --workdir /python-demo -it python-demo /bin/bash

Run the python files:

python3 1-give-me-a-dockerfile.py
# or
python3 2-tell-me-more-about-docker-and-wasm.py

Use the Python dev environment

start the stack with the Python dev environment

docker compose --profile python up

The Python dev environment is a Web IDE based on Coder Server with the Python runtime and tools. The environment is starded as a Docker Compose service and you can open the IDE with this URL: http://localhost:3000.

To use it remotely (to connect to the WebIDE running on the Pi from your workstation), use the DNS name of your Pi or its IP address. For example http://hal.local:3000 (where hal.local is the DNS name of my Pi).

Use the JavaScript dev environment

start the stack with the JavaScript dev environment

docker compose --profile javascript up

The JavaScript dev environment is a Web IDE based on Coder Server with the Node.js runtime and tools. The environment is starded as a Docker Compose service and you can open the IDE with this URL: http://localhost:3001.

To use it remotely (to connect to the WebIDE running on the Pi from your workstation), use the DNS name of your Pi or its IP address. For example http://hal.local:3001 (where hal.local is the DNS name of my Pi).

Test your stack remotely

First test with a simple curl request

• The answer time is long because we do not use streaming
• Where hal.local is the DNS name of my Pi

curl http://hal.local:11434/api/generate -d '{
  "model": "tinydolphin",
  "prompt": "Explain simply what is WebAssembly",
  "stream": false
}'

Or try this one:

curl http://hal.local:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
    "model": "tinydolphin",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Hello!"
        }
    ]
}'

Get the list of the models

curl http://hal.local:11434/api/tags

Load another model

curl http://hal.local:11434/api/pull -d '{
  "name": "phi"
}'

✋ llama2 is too big for a Pi

Delete a model

curl -X DELETE http://hal.local:11434/api/delete -d '{
  "name": "llama2"
}'

Blog posts

• Host Ollama and TinyDolphin LLM on a Pi5 with Docker Compose: Run Ollama on a Pi5
• First Steps with LangChain and the Python toolkit: Ollama on my Pi5: The Python dev environment
• Prompts and Chains with Ollama and LangChain
• Make a GenAI Web app in less than 40 lines of code
• Make a GenAI Conversational Chatbot with memory
• Create a GenAI Rust Teacher
• Let's talk with a GenAI French cook