Fail to connect to the model

[geokfong]

Last login: Fri Mar 22 20:40:29 on ttys002
cd /Users/geokf/rmkv/RWKV-Runner.app/Contents/MacOS/../../../ && /Library/Frameworks/Python.framework/Versions/3.10/bin/python3 ./backend-python/main.py --port 8000 --host 127.0.0.1
geokf@MacBook-Air-3 ~ % cd /Users/geokf/rmkv/RWKV-Runner.app/Contents/MacOS/../../../ && /Library/Frameworks/Python.framework/Versions/3.10/bin/python3 ./backend-python/main.py --port 8000 --host 127.0.0.1
--- 0.7619640827178955 seconds ---
INFO: Started server process [4246]
INFO: Waiting for application startup.
cyac not found
torch found: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/torch/lib
torch set
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: 127.0.0.1:49503 - "GET / HTTP/1.1" 200 OK
INFO: 127.0.0.1:49503 - "GET /status HTTP/1.1" 200 OK
Updated Model Config: max_tokens=4100 temperature=1.0 top_p=0.3 presence_penalty=0.0 frequency_penalty=1.0 penalty_decay=None top_k=None global_penalty=None
INFO: 127.0.0.1:49503 - "POST /update-config HTTP/1.1" 200 OK
Strategy Devices: {'cpu'}
state cache disabled
RWKV_JIT_ON 1 RWKV_CUDA_ON 0 RESCALE_LAYER 0

Loading /Users/geokf/rmkv/models/RWKV-x060-World-3B-v2-20240228-ctx4096.pth ...
Model detected: v6.0
Strategy: (total 32+1=33 layers)

• cpu [float32, float32], store 33 layers
  0-cpu-float32-float32 1-cpu-float32-float32 2-cpu-float32-float32 3-cpu-float32-float32 4-cpu-float32-float32 5-cpu-float32-float32 6-cpu-float32-float32 7-cpu-float32-float32 8-cpu-float32-float32 9-cpu-float32-float32 10-cpu-float32-float32 11-cpu-float32-float32 12-cpu-float32-float32 13-cpu-float32-float32 14-cpu-float32-float32 15-cpu-float32-float32 16-cpu-float32-float32 17-cpu-float32-float32 18-cpu-float32-float32 19-cpu-float32-float32 20-cpu-float32-float32 21-cpu-float32-float32 22-cpu-float32-float32 23-cpu-float32-float32 24-cpu-float32-float32 25-cpu-float32-float32 26-cpu-float32-float32 27-cpu-float32-float32 28-cpu-float32-float32 29-cpu-float32-float32 30-cpu-float32-float32 31-cpu-float32-float32 32-cpu-float32-float32
  emb.weight f32 cpu 65536 2560
  blocks.0.ln1.weight f32 cpu 2560
  blocks.0.ln1.bias f32 cpu 2560
  blocks.0.ln2.weight f32 cpu 2560
  blocks.0.ln2.bias f32 cpu 2560
  blocks.0.att.time_maa_x f32 cpu 2560
  blocks.0.att.time_maa_w f32 cpu 2560
  blocks.0.att.time_maa_k f32 cpu 2560
  blocks.0.att.time_maa_v f32 cpu 2560
  blocks.0.att.time_maa_r f32 cpu 2560
  blocks.0.att.time_maa_g f32 cpu 2560
  blocks.0.att.time_maa_w1 f32 cpu 2560 160
  blocks.0.att.time_maa_w2 f32 cpu 5 32
  blocks.0.att.time_decay f32 cpu 40 64
  blocks.0.att.time_decay_w1 f32 cpu 2560 64
  blocks.0.att.time_decay_w2 f32 cpu 64 2560
  blocks.0.att.time_first f32 cpu 40 64
  blocks.0.att.receptance.weight f32 cpu 2560 2560
  blocks.0.att.key.weight f32 cpu 2560 2560
  blocks.0.att.value.weight f32 cpu 2560 2560
  blocks.0.att.output.weight f32 cpu 2560 2560
  blocks.0.att.gate.weight f32 cpu 2560 2560
  blocks.0.att.ln_x.weight f32 cpu 2560
  blocks.0.att.ln_x.bias f32 cpu 2560
  blocks.0.ffn.time_maa_k f32 cpu 2560
  blocks.0.ffn.time_maa_r f32 cpu 2560
  blocks.0.ffn.key.weight f32 cpu 2560 8960
  blocks.0.ffn.receptance.weight f32 cpu 2560 2560
  blocks.0.ffn.value.weight f32 cpu 8960 2560
  ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
  blocks.31.ln1.weight f32 cpu 2560
  blocks.31.ln1.bias f32 cpu 2560
  blocks.31.ln2.weight f32 cpu 2560
  blocks.31.ln2.bias f32 cpu 2560
  blocks.31.att.time_maa_x f32 cpu 2560
  blocks.31.att.time_maa_w f32 cpu 2560
  blocks.31.att.time_maa_k f32 cpu 2560
  blocks.31.att.time_maa_v f32 cpu 2560
  blocks.31.att.time_maa_r f32 cpu 2560
  blocks.31.att.time_maa_g f32 cpu 2560
  blocks.31.att.time_maa_w1 f32 cpu 2560 160
  blocks.31.att.time_maa_w2 f32 cpu 5 32
  blocks.31.att.time_decay f32 cpu 40 64
  blocks.31.att.time_decay_w1 f32 cpu 2560 64
  blocks.31.att.time_decay_w2 f32 cpu 64 2560
  blocks.31.att.time_first f32 cpu 40 64
  blocks.31.att.receptance.weight f32 cpu 2560 2560
  blocks.31.att.key.weight f32 cpu 2560 2560
  blocks.31.att.value.weight f32 cpu 2560 2560
  blocks.31.att.output.weight f32 cpu 2560 2560
  blocks.31.att.gate.weight f32 cpu 2560 2560
  blocks.31.att.ln_x.weight f32 cpu 2560
  blocks.31.att.ln_x.bias f32 cpu 2560
  blocks.31.ffn.time_maa_k f32 cpu 2560
  blocks.31.ffn.time_maa_r f32 cpu 2560
  blocks.31.ffn.key.weight f32 cpu 2560 8960
  blocks.31.ffn.receptance.weight f32 cpu 2560 2560
  blocks.31.ffn.value.weight f32 cpu 8960 2560
  ln_out.weight f32 cpu 2560
  ln_out.bias f32 cpu 2560
  head.weight f32 cpu 2560 65536
  Updated Model Config: max_tokens=4100 temperature=1.0 top_p=0.3 presence_penalty=0.0 frequency_penalty=1.0 penalty_decay=None top_k=None global_penalty=None
  INFO: 127.0.0.1:49521 - "POST /update-config HTTP/1.1" 200 OK
  zsh: killed /Library/Frameworks/Python.framework/Versions/3.10/bin/python3 --port 8000
  geokf@MacBook-Air-3 rmkv %

Hi, I'm trying to connect to the model, but failed. Anyone able to help here? Thanks

[josStorer]

What method are you using to connect to the model? If you are sending a request over a LAN, you need to pass --host 0.0.0.0, which you can enable in the client's settings.