Imagine a game where AI-powered Non-Player Characters (NPCs) can not only respond to your questions but also perform actions based on your requests! 😄 Cohere's Command model generates character responses, including dialogue and actions, using templates that define their talking style, action list, and bio. 💬 Unreal Engine, even though it can't run Python directly, communicates with it through text files, enabling real-time interaction. 🎮 AssemblyAI transcribes player dialogue, while ElevenLabs generates voice dialogue, which is then imported into the game using the Runtime Audio Importer. 🗣️ The MetaHuman SDK brings these characters to life with facial animations. 😮 Actions requested by players are parsed and executed, ranging from animations to item spawns and even AI-generated music creation. 🕹️
The Repository contains the python script and the demo.umap which you can import in a level in unreal to look at the level blueprint and see the logic. 📂 There are a couple of other files as well that can help you understand better. 📚 If you find this repository helpful, consider giving it a star! ⭐
Discord Server: https://discord.gg/CfK7DCWKwy
- 🤖 Cohere's Language Models and Embedding Models
- 🗣️ ElevenLabs
- 📝 AssemblyAI
- 🗃️ ChromaDB
- 🔗 Langchain
- 🎮 Unreal Engine
- 🌲 Quixel Bridge
- 🧔 Metahuman SDK
- 🙆 Mixamo
- 🏃 Runtime Audio Importer
- "Warlock's End - Metahuman Armor" (https://skfb.ly/oHInI) by pdravecky.fx is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).(https://sketchfab.com/3d-models/warlocks-end-metahuman-armor-e3352bc5d74f4c38afe7bdf700420947)
- "Sci-fi suit for Metahuman" (https://skfb.ly/oFuBz) by Alexander Kurmanin is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/). (https://sketchfab.com/3d-models/sci-fi-suit-for-metahuman-0e938256aa4f4af3b3a23c811da585fa#download)
- "Magic Ring - Blue" (https://skfb.ly/6Uq7O) by Noob Model :D is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).(https://sketchfab.com/3d-models/magic-ring-blue-5c81d8a7610543779d0c7e85f9175fae)
- "Magic Ring - Yellow" (https://skfb.ly/6WMOx) by Noob Model :D is licensed under Creative Commons Attribution (http://creativecommons.org/licenses/by/4.0/).(https://sketchfab.com/3d-models/magic-ring-yellow-757ff3fbfb2045c68165c06494917faa#download)
- Quixel Bridge (https://quixel.com/bridge)
- Metahuman Creator (https://metahuman.unrealengine.com/)
If you found this interesting check out Alystria AI for more fun projects
https://www.linkedin.com/company/alystria-ai- 🌐 Github: https://github.com/AkshitIreddy
- 🐤 Twitter: https://twitter.com/Akshit2089
- 💡 LinkedIn: https://www.linkedin.com/in/akshit-ireddy
- ✍️ Medium: https://medium.com/@akshit.r.ireddy