[Request] llama对话过程能否保活？

[gaye746560359]

🥰 需求描述

我是docker部署的ollama，机器是i-3600k+rtx4090
在用lobechat使用llama模型对话时有两处需要优化：
1.启动对话要很久的时间，一直加载中，应该是执行ollama run llama很费时间
2.对话过程暂停一段时间后，再次对话又需要等待很久

🧐 解决方案

能否在启动对话后保活模型？或者有其他方案？

📝 补充信息

No response

[lobehubbot]

👀 @gaye746560359

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

🥰 Description of requirements

I deployed ollama with docker, and the machine is i-3600k+rtx4090
There are two things that need to be optimized when using llama model dialogue with lobechat:

1. It takes a long time to start the conversation and it keeps loading. It should be that executing ollama run llama is very time-consuming.
2. After the dialogue process is paused for a period of time, it will take a long time to talk again.

🧐 Solution

Is it possible to keep a model alive after starting a conversation? Or are there other options?

📝 Supplementary information

No response

[arvinxx]

这可能要问下 ollama 的社区了

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

You might have to ask the ollama community about this.

[MapleEve]

看Ollama 部署的文档，我这边 PR 过对应的环境变量可以让模型保活时间变长

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Looking at the Ollama deployment documentation, I PRed the corresponding environment variables to make the model stay alive longer.