[Feature] Plans to add model provider support

[fred-bf]

There have been many discussions in the community regarding support for multiple models.

• Ollama integration #3484
• [Feature] 希望加入智谱清言的API调用 #3923
• [Feature] 整合类似chatpdf的功能 #960
• [Bug] chatglm整合后，网页版正常运行， 而桌面版一直出现failed to fetch #3431
• [Feature] Multi-Backend API Configuration #3125

Here, we will gather NextChat's current support plans for different models and provide dynamic updates on the overall progress.

Firstly, we expect to separate the model-related logic from the frontend and may consider creating a separate JavaScript component to standardize it (this could be managed as an independent package). Afterwards, we will develop adapters for each model based on this component/package. We anticipate that each adapter will have at least the following basic capabilities: multimodality (text, images), token billing, and customizable model parameters (temperature, max_tokens, etc.).

We have roughly divided the work into the following parts:

NextChat UI Separation

• Separation of UI components
• Allow to register model providers in Settings page
• Standardization of configuration, statistics, sharing, and other functionalities
• Standardization model features, including function calling, agents loader, etc

Implementation of Multi-Model Providers

• Basic multi-model package (using OpenAI GPT as an example)
• Ollama
  • Support localhost ollama deployment( https://docs.nextchat.dev/models/ollama )
  • Support manage local models
• Derivatives of GPT
  • Azure
  • ?OneAPI (to be determined)
• Open-source models (considering support for a local App Model Manager)
  • Llama
  • Mistral
• Closed-source models
  • Claude
  • AWS Lex
  • Google Gemini
• Other models
  • Wenxinyiyan
  • Zhipu
• Hosting Platforms
  • Poe.com
  • together.ai
  • Cloudflare AI

Local Model Manager

• Support for local model downloading and running

Server-Side Multi-Model Service

• Support for independently deploying a multi-model service as NextChat's API proxy

Current implementation:

• [WIP] feat: #1550 Azure, Bard, Claude, ChatGLM, Qwen, Spark #2603

[WBinBin001]

是否有计划支持 mistral.ai

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Are there any plans to support mistral.ai?

[fred-bf]

@WBinBin001 you can use mistral through Ollama https://docs.nextchat.dev/models/ollama

[WBinBin001]

@WBinBin001 you can use mistral through Ollama https://docs.nextchat.dev/models/ollama

是否会支持 mistral.ai 平台的 api 和 key

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

@WBinBin001 you can use mistral through Ollama https://docs.nextchat.dev/models/ollama

Will it support the api and key of the mistral.ai platform?

[PPoooMM]

The Mistral AI API models, which include Mistral-small-latest, Mistral-medium-latest, and Mistral-large-latest, particularly the Mistral Large model, have been ranked second, only behind GPT-4, which has an MMLU score of 86.4%. The Mistral Large model achieved a score of 81.2%, surpassing even GPT-4 Turbo, which scored 80.48%. This makes the model particularly interesting, and I support its inclusion in the most popular cross-platform chatbots, like ChatGPTNextWeb.

[EarlyBedEarlyUp]

Looking forward to supporting Claude 3

[snowords]

Vote for moonshot

[GrayXu]

I think that some functions should not be implemented in this repo. Different LLM backends can standardize the API through xusenlinzy/api-for-open-llm or BerriAI/litellm. ChatGPTNextWeb only needs to focus on the functionality of setting URLs and models for different conversations.

[Genuifx]

Kimi is awesome. Support it!

[0x5c0f]

希望支持 AWS Bedrock
Looking forward to supporting AWS Bedrock

[fred-bf]

claude is supported in PR: #4457

[qiqitom]

I'd really like to see Gemini Pro 1.5 added.
The Gemini Pro 1.0 model performs surprisingly well! as well as, the video processing capabilities of the Gemini Pro 1.5 are impressive. Thank you.