Skip to content

Latest commit

 

History

History
49 lines (33 loc) · 1.01 KB

local_readme.md

File metadata and controls

49 lines (33 loc) · 1.01 KB
Raw

TODO

  • fastchat for train_internlm
  • fastchat for internlm2composer [conversation not offical]
  • multi process Controller with redis and Gunicorn

Setup

step 1 install fastchat

git clone https://github.com/lm-sys/FastChat.git
cd FastChat
pip install -e .

step 1.5 install Gunicorn

https://fastapi.tiangolo.com/deployment/server-workers/

pip install "uvicorn[standard]" gunicorn

step 2 install redis

https://redis.io/docs/install/install-redis/

curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list

sudo apt-get update
sudo apt-get install redis

test your redis

$ redis-cli ping
PONG

step 3 run !

gunicorn --workers 20 --worker-class uvicorn.workers.UvicornWorker --bind 172.28.0.23:21001 fastchat.serve.controller_app:app