How can we make model calls faster

[userandpass]

What is the issue?

I used docker to load multiple ollama images and distribute them using nginx, which was much slower than calling the deployed model directly

OS

Linux

GPU

Nvidia

CPU

No response

Ollama version

0.1.34

[userandpass]

After I added the "keep_alive": "24h" parameter, after a while I executed the nvidia-smi command, there was no ollama on the card, so I needed to call the interface to display it