Docker-compose for LocalAI CUDA 12 autoinstall WITHOUT MODELS

[antoniostasi1987]

Hi all,
someone can help mi to build a docker-compose.yaml file for build a LocalAI version that support CUDA 12 WITHOUT pre-installed models? Thanks a lot to all!

[mudler]

Pre installed models are only present in the All-in-One images. You can use the standard images, that does not have any pre-installed model, see https://localai.io/docs/reference/container-images/ for a list depending on your HW.

[antoniostasi1987]

Hi, where I can recover configuration files for pre-installed models in AIO images? My intentions is to compile a new version with a subset of these models, renamed and reconfigured.
Wait for a response, thanks a lot

[mudler]

The AIO models configurations files are in: https://github.com/mudler/LocalAI/tree/master/aio - you can also just drop the files in your models folder

[antoniostasi1987]

Clear. thanks.
Now I placed my YAML files inside the models folder, and I expected it to download automatically, but that doesn't happen. What am I doing wrong?
Thanks and sorry to bother you, but I'm new to this topic

[BradKML]

Seconding this for @mudler but for adding SOTA models like LLaMA3 or maybe CodeQwen + DeepSeekCoder, directly from HuggingFace?