feat: Add support for ZhipuAI's GLM LLMs

[ifsheldon]

Please describe the purpose of this pull request.
This intends to add support for LLMs from ZhipuAI. The new glm-4 model is comparable to GPT-4-Turbo and better at Chinese. The APIs for calling this model is almost the same as the latest OpenAI APIs, so we think it's fairly straightforward and easy to add the support.

How to test
Not yet. This is a draft PR

Have you tested this PR?
Not yet. This is a draft PR

Related issues or PRs
We want to add this for better Chinese support. See #799

Is your PR over 500 lines of code?
No

[ifsheldon]

I have a few questions:

• MemGPT/memgpt/llm_api_tools.py

  Line 379 in 68cc8a7

  def create(

  
  Here in this function, why didn't you use the official openai library to make requests?
  Because authentication of ZhipuAI is a bit complicated, I want to use the official zhipuai to make requests, which introduces an optional dependency.
• Also related to openai, why did you choose to create your own types instead of using OpenAI's? I think their types for messages are more complete.
• Why didn't you type messages of this function? If you don't have time for this, we can help with that in separate PRs. This will also make it easier to add more LLM service providers.

Please don't get me wrong, I'm not trying to be rude. I just want to hear from you if you had other concerns so I can adjust my PR.

[cpacker]

@ifsheldon for both the openai questions - we removed openai as a dependency because we were having issues with clashing versions with some of our other dependencies (llamaindex) and optional dependencies (eg pyautogen) that were also using the openai library. The removal + migration to HTTP requests instead of the client library was around the same time that there was a breaking v0.x -> v1.0 change. Our Message type is a little different since it's a little specific to how we're storing data under the hood, but you're right in that it's very close to the official OAI types. Our ChatCompletionResponse type should be effectively identical.

For the last question - we're working on improving the typing in the repo, happy to merge any typing PRs especially if it's just adding hints to existing functions. There's a gradual effort here as well: #855

[cpacker]

For Zhipu specifically, it would be great if we could incorporate it on top of the existing HTTP ChatCompletion request style instead of adding any client library dependencies. Assuming the API roughly follows OpenAI (the REST API, not the python client), it should be possible to do this via the existing credentials style + local LLM route #835 (it supports POST requests with bearer tokens or api-key headers atm, but we can add more if needed).

[ifsheldon]

Assuming the API roughly follows OpenAI (the REST API, not the python client), it should be possible to do this via the existing credentials style + local LLM route #835 (it supports POST requests with bearer tokens or api-key headers atm, but we can add more if needed).

I would say the APIs are a subset of OpenAI's. For example, messages don't have a name field. So, if we use existing code to send requests, it may work but it's really flaky and it kind of feels like "always watch your back". For authentication, the authentication token is in Authentication header, but the code is customized with JWT.

See the reference code from ZhipuAI to calculate the auth code

import time
import jwt

def generate_token(apikey: str, exp_seconds: int):
    try:
        id, secret = apikey.split(".")
    except Exception as e:
        raise Exception("invalid apikey", e)

    payload = {
        "api_key": id,
        "exp": int(round(time.time() * 1000)) + exp_seconds * 1000,
        "timestamp": int(round(time.time() * 1000)),
    }

    return jwt.encode(
        payload,
        secret,
        algorithm="HS256",
        headers={"alg": "HS256", "sign_type": "SIGN"},
    )

As for dependency management, yeah, managing dependencies is hell in Python.

I took a closer look at the code of zhipuailibrary and the dependency tree of my env where latest memgpt from source and zhipuai live together. The conclusions are:

• The library is a thin wrapper around httpx.Client, with type information.
• 3 of 4 dependencies of zhipuai are reused in memgpt's dependency tree, except pyjwt which is required for auth.
  • That means if we use requests to send requests ourselves, we still need to introduce one additional dependency PyJWT and we also lose types offered in zhipuai.

See the dependency trees of `memgpt` and `zhipuai`

I deleted irrelevant dependency branches. You can see the only addition is PyJWT which we need anyways.

zhipuai==2.0.1
├── cachetools [required: >=4.2.2, installed: 5.3.2]
├── httpx [required: >=0.23.0, installed: 0.25.2]
├── pydantic [required: >=2.5.2, installed: 2.5.2]
└── PyJWT [required: ~=2.8.0, installed: 2.8.0]

pymemgpt==0.2.11
├── chromadb [required: >=0.4.18,<0.5.0, installed: 0.4.21]
│   ├── fastapi [required: >=0.95.2, installed: 0.105.0]
│   │   ├── pydantic [required: >=1.7.4,<3.0.0,!=2.1.0,!=2.0.1,!=2.0.0,!=1.8.1,!=1.8, installed: 2.5.2] 
│   ├── kubernetes [required: >=28.1.0, installed: 28.1.0]
│   │   ├── google-auth [required: >=1.0.1, installed: 2.25.2]
│   │   │   ├── cachetools [required: >=2.0.0,<6.0, installed: 5.3.2]  
│   ├── pydantic [required: >=1.9, installed: 2.5.2]  
├── httpx [required: >=0.25.2,<0.26.0, installed: 0.25.2]  
│   ├── cachetools [required: Any, installed: 5.3.2]  
│   ├── pydantic [required: >=1.10, installed: 2.5.2]  
├── llama-index [required: >=0.9.13,<0.10.0, installed: 0.9.24]
│   ├── httpx [required: Any, installed: 0.25.2]  
│   ├── openai [required: >=1.1.0, installed: 1.7.2]
│   │   ├── httpx [required: >=0.23.0,<1, installed: 0.25.2]  
│   │   ├── pydantic [required: >=1.9.0,<3, installed: 2.5.2]  
├── pydantic [required: >=2.5.2,<3.0.0, installed: 2.5.2]  

So, I think it's fine to introduce zhipuai as an optional dependency, since the conflicts it might make is minimum as we inspect the dep trees. What do you think? @cpacker