Tabby not starting webserver

[NeuroJulian]

Tabby isn't starting webserver. No logs. Model is downloaded (with progress bar).

2024-04-18T18:54:32.272097Z INFO tabby::serve: crates/tabby/src/serve.rs:118: Starting server, this might take a few minutes...

I am using this command:

docker run -it --gpus all \
  -p 8080:8080 -v $HOME/.tabby:/data \
  tabbyml/tabby serve --model StarCoder-1B --device cuda

OS and GPU:
RTX 3060 12GB, Driver Version: 535.161.07 CUDA Version: 12.2, Ubuntu 20.04

Regards,
Julian

[wsxiaoys]

Can you provide the image tag you're using? and maybe try with tabbyml/tabby:0.9.1 to explicitly set the image tag?

[NeuroJulian]

I've got the same error, using 0.9.1 and latest tag.

[NeuroJulian]

Maybe docker and docker compose version can help:
Docker Compose version v2.26.1

Client: Docker Engine - Community
 Version:           26.0.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        d260a54
 Built:             Thu Apr 11 10:53:25 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.0.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       60b9add
  Built:            Thu Apr 11 10:53:25 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[scottrbaxter]

Try passing the --webserver arg, e.g.

docker run -it --gpus all \
  -p 8080:8080 -v $HOME/.tabby:/data \
  tabbyml/tabby serve --model StarCoder-1B --webserver --device cuda

[NeuroJulian]

No response... How can I read the logs?

[andar1an]

I can recreate this with a systemd service on version 0.11.1, and logs don't indicate anything. Systemd indicates service is running and there are no errors.

I also tried to vary between different arguments like model and webserver

Reverting to older version.

[andar1an]

It appears that there was a Breaking change in 0.11 where --webserver should be removed. Is that still in the docker container exec command?