👟 Runtime, LLM-powered NPCs

Interactive_NPCs.mp4

---

pip install gigax

Features

• 🕹️ NPCs that <speak>, <jump>, <attack> and perform any other action you've defined
• ⚡ <1 second GPU inference on most machines
• 🤗 Open-weights models available, fined-tuned from: Llama-3, Phi-3, Mistral, etc.
• 🔒 Structured generation with Outlines 〰️ means the output format is always respected
• 🗄️ Coming soon: Local server mode, with language-agnostic API
• 📜 Available on API: Runtime quest generation, for players and NPCs
• 😶‍🌫️ Available on API: Memory creation, storage and retrieval with a Vector DB

Gigax has new releases and features on the way. Make sure to ⭐ star and 👀 watch this repository!

Usage

Model instantiation

• We provide various models on the 🤗 Huggingface hub:

  • NPC-LLM-7B (our Mistral-7B fine-tune)
  • NPC-LLM-3_8B (our Phi-3 fine-tune)

• Start by instantiating one of them using outlines:

from outlines import models
from gigax.step import NPCStepper
from transformers import AutoTokenizer, AutoModelForCausalLM

# Download model from the Hub
model_name = "Gigax/NPC-LLM-7B"
llm = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Our stepper takes in a Outlines model to enable guided generation
# This forces the model to follow our output format
model = models.Transformers(llm, tokenizer)

# Instantiate a stepper: handles prompting + output parsing
stepper = NPCStepper(model=model)

Stepping an NPC

• From there, stepping an NPC is a one-liner:

action = stepper.get_action(
    context=context,
    locations=locations,
    NPCs=NPCs,
    protagonist=protagonist,
    items=items,
    events=events,
)

• We provide classes to instantiate Locations, NPCs, etc. :

from gigax.parse import CharacterAction
from gigax.scene import (
    Character,
    Item,
    Location,
    ProtagonistCharacter,
    Skill,
    ParameterType,
)
# Use sample data
current_location = Location(name="Old Town", description="A quiet and peaceful town.")
NPCs = [
    Character(
    name="John the Brave",
    description="A fearless warrior",
    current_location=current_location,
    )
]
protagonist = ProtagonistCharacter(
    name="Aldren",
    description="Brave and curious",
    current_location=current_location,
    memories=["Saved the village", "Lost a friend"],
    quests=["Find the ancient artifact", "Defeat the evil warlock"],
    skills=[
        Skill(
            name="Attack",
            description="Deliver a powerful blow",
            parameter_types=[ParameterType.character],
        )
    ],
    psychological_profile="Determined and compassionate",
)
items = [Item(name="Sword", description="A sharp blade")]
events = [
    CharacterAction(
        command="Say",
        protagonist=protagonist,
        parameters=[items[0], "What a fine sword!"],
    )
]

API

Contact us to give our NPC API a try - we'll take care of model serving, NPC memory, and more!