Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

使用 Higress + NextChat 快速搭建私人GPT助理 #938

Open
johnlanni opened this issue May 14, 2024 · 27 comments
Open

使用 Higress + NextChat 快速搭建私人GPT助理 #938

johnlanni opened this issue May 14, 2024 · 27 comments
Labels
area/ai

Comments

@johnlanni
Copy link
Collaborator

johnlanni commented May 14, 2024 •
edited

第一步：创建一个名为 docker-compose.yml 的文件，并填入以下内容：

注意：

  1. YOUR_DASHSCOPE_API_KEY 需要替换为你自己的通义千问的 API Key；
  2. /path-to-local-config-folder 需要替换为一个本地可以用来保存配置文件的目录路径。
  3. CODE=访问密码 这里换成你自己的密码
version: '3.9'

networks:
  higress-net:
    external: false

services:
  higress:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0
    environment:
      - CONFIG_TEMPLATE=ai-proxy
      - DEFAULT_AI_SERVICE=qwen
      - DASHSCOPE_API_KEY=YOUR_DASHSCOPE_API_KEY
    networks:
      - higress-net
    ports:
      - "8080:8080/tcp"
      - "8001:8001/tcp"
    volumes:
      - /path-to-local-config-folder:/data
    restart: always
  nextchat:
    image: yidadaa/chatgpt-next-web
    environment:
      - CODE=访问密码
      - OPENAI_API_KEY=unused
      - CUSTOM_MODELS=-all,+gpt-4o
      - BASE_URL=http://higress:8080
    networks:
      - higress-net
    ports:
      - "3000:3000/tcp"
    restart: always

第二步：在命令行中运行以下命令，启动 docker compose 项目：

docker compose -p higress-ai up -d

第三步：在浏览器里访问 http://localhost:3000/，打开 NextChat 页面；

image

第四步：点击对话输入框工具栏最右侧的模型设置按钮，将模型切换为 gpt-4-turbo 并关闭窗口。

因为 Higress 的 AI Proxy 插件（可以访问 http://localhost:8001 查看插件配置）配置了 gpt-4-turbo 到 qwen-max 的模型映射，所以实际上这里提供的模型服务是 qwen-max

image

完成！现在你就可以与 AI 进行对话了。

按照上面文件进行配置，默认和你对话的将是通义千问。如果想切换到OpenAI ChatGPT，只需要将文件中的 DEFAULT_AI_SERVICE=qwen 修改为 DEFAULT_AI_SERVICE=openai，并添加 OpenAI 的 API key 配置，然后重新启动 docker compose 项目即可。

# 以上配置省略
  higress:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.3.6
    environment:
      - CONFIG_TEMPLATE=ai-proxy
      - DEFAULT_AI_SERVICE=openai
      - OPENAI_API_KEY=YOUR_OPENAI_API_KEY
# 以下配置省略
docker compose -p higress-ai down
docker compose -p higress-ai up -d
@johnlanni johnlanni changed the title 使用 Higress + NextChat 搭建私人GPT助理 使用 Higress + NextChat 快速搭建私人GPT助理 May 14, 2024
@johnlanni johnlanni pinned this issue May 14, 2024
@CH3CHO CH3CHO mentioned this issue May 15, 2024
Open
22 tasks
@johnlanni johnlanni unpinned this issue May 15, 2024
@Dwayneten
Copy link

一直返回

{
  "error": true,
  "message": "empty response from server"
}

因为 Higress 的 AI Proxy 插件（可以访问 http://localhost:8001/ 查看插件配置）配置了 gpt-4-turbo 到 qwen-max 的模型映射，所以实际上这里提供的模型服务是 qwen-max

这个 AI Proxy 插件需要自己配置吗

@CH3CHO
Copy link
Collaborator

CH3CHO commented May 17, 2024 via email

@Dwayneten
Copy link

token 配的对吗？

DashScope 的 API-KEY 我用 HTTP 调用接口测试过可以正常返回

@CH3CHO
Copy link
Collaborator

CH3CHO commented May 18, 2024

DashScope 的 API-KEY 我用 HTTP 调用接口测试过可以正常返回

你可以在 docker-compose.yml 里给 higress 加一个 GATEWAY_COMPONENT_LOG_LEVEL=misc:error,wasm:debug 的环境变量，然后 exec 到 higress 容器里面，查看 /var/log/higress/gateway.log 文件，看看 DashScope 实际返回了什么。

@Dwayneten
Copy link

DashScope 的 API-KEY 我用 HTTP 调用接口测试过可以正常返回

你可以在 docker-compose.yml 里给 higress 加一个 GATEWAY_COMPONENT_LOG_LEVEL=misc:error,wasm:debug 的环境变量，然后 exec 到 higress 容器里面，查看 /var/log/higress/gateway.log 文件，看看 DashScope 实际返回了什么。

有个这个错误

2024-05-18T01:57:11.528002Z     error   wasm    cannot fetch Wasm module oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-proxy:1.0.0: could not fetch Wasm OCI image: could not fetch image: Get "https://dockerauth.cn-hangzhou.aliyuncs.com/auth?scope=repository%3Aplugins%2Fai-proxy%3Apull&service=registry.aliyuncs.com%3Acn-hangzhou%3Achina%3Acri-r0xfyoxudmtseqq7": dial tcp [2408:4005:1000:10::2]:443: connect: network is unreachable

@CH3CHO
Copy link
Collaborator

CH3CHO commented May 18, 2024

DashScope 的 API-KEY 我用 HTTP 调用接口测试过可以正常返回

你可以在 docker-compose.yml 里给 higress 加一个 GATEWAY_COMPONENT_LOG_LEVEL=misc:error,wasm:debug 的环境变量，然后 exec 到 higress 容器里面，查看 /var/log/higress/gateway.log 文件，看看 DashScope 实际返回了什么。

有个这个错误

2024-05-18T01:57:11.528002Z     error   wasm    cannot fetch Wasm module oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-proxy:1.0.0: could not fetch Wasm OCI image: could not fetch image: Get "https://dockerauth.cn-hangzhou.aliyuncs.com/auth?scope=repository%3Aplugins%2Fai-proxy%3Apull&service=registry.aliyuncs.com%3Acn-hangzhou%3Achina%3Acri-r0xfyoxudmtseqq7": dial tcp [2408:4005:1000:10::2]:443: connect: network is unreachable

拉不到插件镜像。你的机器能访问外网吗？

@johnlanni
Copy link
Collaborator Author

johnlanni commented May 18, 2024 •
edited

貌似是访问阿里云acr镜像走了ipv6导致的

image

看了下这个域名 higress-registry.cn-hangzhou.cr.aliyuncs.com 并没有 ipv6 的解析，是不是本地有什么网络特殊配置

@Dwayneten
Copy link

是不是本地有什么网络特殊配置

在docker-compose.yml中给 Higress 指定 dns 为 223.5.5.5 后正常了
感谢回复 🥰

@dpy013
Copy link

dpy013 commented May 31, 2024

各位，按照上面的说明都部署好了，打开提示API not found: /这问题怎么破？
2024-05-31 00:52:13,142 INFO Set uid to user 0 succeeded
2024-05-31 00:52:13,144 INFO supervisord started with pid 1
2024-05-31 00:52:14,147 INFO spawned: 'apiserver' with pid 8
2024-05-31 00:52:14,150 INFO spawned: 'controller' with pid 9
2024-05-31 00:52:14,153 INFO spawned: 'pilot' with pid 10
2024-05-31 00:52:14,155 INFO spawned: 'gateway' with pid 11
2024-05-31 00:52:14,162 INFO spawned: 'console' with pid 12
2024-05-31 00:52:15,186 INFO success: apiserver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-05-31 00:52:15,186 INFO success: controller entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-05-31 00:52:15,186 INFO success: pilot entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-05-31 00:52:15,186 INFO success: gateway entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-05-31 00:52:15,186 INFO success: console entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

@johnlanni
Copy link
Collaborator Author

@dpy013 是说直接访问 127.0.0.1:8080吗，得访问127.0.0.1:3000

@dpy013
Copy link

dpy013 commented May 31, 2024

是的，访问的确实是127.0.0.1:8080
用的是腾讯云的服务器搭的。
操作系统是ubuntu24.04 lts

@dpy013
Copy link

dpy013 commented May 31, 2024

删除了原来的重新部署，并且访问了127.0.0.1:3000发送问题出现以下错误：

访问密码不正确或为空，请前往登录页输入正确的访问密码，或者在设置页填入你自己的 OpenAI API Key。

{
  "error": true,
  "msg": "wrong access code"
}

@johnlanni
Copy link
Collaborator Author

@dpy013 注意docker-compose.yaml里nextchat容器的CODE环境变量配置

@dpy013
Copy link

dpy013 commented May 31, 2024

把通译千问的api 密钥修改到了 - OPENAI_API_KEY=unused这个参数里了，还是不能用。

@johnlanni
Copy link
Collaborator Author

@dpy013 跟这个没关系

image

注意下这个

@dpy013
Copy link

dpy013 commented May 31, 2024

看来不是服务端问题，是我自己的key出问题了。
我阿里云打开了nfa，会不会是nfa影响到了通译的key？

@johnlanni
Copy link
Collaborator Author

应该不会的

@dpy013
Copy link

dpy013 commented May 31, 2024

刚重新生成了一个key在去看看。

@dpy013
Copy link

dpy013 commented May 31, 2024

这错误越来越奇妙了！
{
"error": true,
"msg": "empty access code"
}

@dpy013
Copy link

dpy013 commented May 31, 2024

下面是日志：
error-logs.zip

@johnlanni
Copy link
Collaborator Author

@dpy013 你的docker-compose脱敏发一下吧

@dpy013
Copy link

dpy013 commented May 31, 2024 •
edited by johnlanni

以下是docker-compose.yaml文件：

version: '3.9'

networks:
  higress-net:
    external: false

services:
  higress:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0-rc.1
    environment:
      - CONFIG_TEMPLATE=ai-proxy
      - DEFAULT_AI_SERVICE=qwen
      - DASHSCOPE_API_KEY=xx-xxxxxxxxxaaaaaessddd19sa4c97d10909
    networks:
      - higress-net
    ports:
      - "8080:8080/tcp"
      - "8001:8001/tcp"
    volumes:
      - /home/ubuntu/AI:/data
    restart: always
  nextchat:
    image: yidadaa/chatgpt-next-web
    environment:
      - CODE=xxxx1234???
      - OPENAI_API_KEY=unused
      - CUSTOM_MODELS=-all,+gpt-4o
      - BASE_URL=http://higress:8080
    networks:
      - higress-net
    ports:
      - "3000:3000/tcp"
    restart: always

@dpy013
Copy link

dpy013 commented May 31, 2024

文件名称维
docker-compose.yaml
启动命令用的是
docker-compose up -d

@johnlanni
Copy link
Collaborator Author

CODE环境变量去掉试试，可能是 chatgpt-next-web 的新镜像的问题

@johnlanni
Copy link
Collaborator Author

#1023

@dpy013 可以试试lobechat

@generic-code-learn
Copy link

你好报错了：
{
"cause": {
"errno": -3001,
"code": "EAI_AGAIN",
"syscall": "getaddrinfo",
"hostname": "higress"
}
}

@johnlanni
Copy link
Collaborator Author

你好报错了： { "cause": { "errno": -3001, "code": "EAI_AGAIN", "syscall": "getaddrinfo", "hostname": "higress" } }

定位到是新版本镜像的配置问题，你重新执行一下 docker pull higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:1.4.0 ，再启动试试

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/ai
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@CH3CHO @johnlanni @Dwayneten @dpy013 @generic-code-learn