[Request] llama对话过程能否保活？

[gaye746560359]

🥰 需求描述

我是docker部署的ollama，机器是i-3600k+rtx4090
在用lobechat使用llama模型对话时有两处需要优化：
1.启动对话要很久的时间，一直加载中，应该是执行ollama run llama很费时间
2.对话过程暂停一段时间后，再次对话又需要等待很久

🧐 解决方案

能否在启动对话后保活模型？或者有其他方案？

📝 补充信息

No response

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

🥰 Description of requirements

I deployed ollama with docker, and the machine is i-3600k+rtx4090
There are two things that need to be optimized when using llama model dialogue with lobechat:

1. It takes a long time to start the conversation and it keeps loading. It should be that executing ollama run llama is very time-consuming.
2. After the dialogue process is paused for a period of time, it will take a long time to talk again.

🧐 Solution

Is it possible to keep a model alive after starting a conversation? Or are there other options?

📝 Supplementary information

No response

[arvinxx]

这可能要问下 ollama 的社区了

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

You might have to ask the ollama community about this.

[MapleEve]

看Ollama 部署的文档，我这边 PR 过对应的环境变量可以让模型保活时间变长

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Looking at the Ollama deployment documentation, I PRed the corresponding environment variables to make the model stay alive longer.