Running on CPU /fedora fails with permissions denied

[jsyk]

Hi, I am trying to setup the server on fedora 39. I want to run model on the CPU.
However, I get these errors in the terminal.

jara@fedora:~$ docker run --entrypoint /opt/tabby/bin/tabby-cpu -it -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model TabbyML/StarCoder-1B
Unable to find image 'tabbyml/tabby:latest' locally
latest: Pulling from tabbyml/tabby
aece8493d397: Pull complete 
976b13f9ef1a: Pull complete 
b63c65d3acaa: Pull complete 
4dda4739d007: Pull complete 
197843862cd5: Pull complete 
cf87cabd72e2: Pull complete 
7908cc89184b: Pull complete 
125997f82033: Pull complete 
675adc119131: Pull complete 
ac621710435d: Pull complete 
7c972629d3c4: Pull complete 
5703d8ad1703: Pull complete 
e9686d8d952e: Pull complete 
Digest: sha256:12e2c361b496941712b6a8380c734aa65dd2021f0a47ee85c4304457ad036b42
Status: Downloaded newer image for tabbyml/tabby:latest
thread 'main' panicked at /root/workspace/crates/tabby-common/src/registry.rs:61:21:
Failed to fetch model organization <TabbyML>: Failed to create dir "/data/models/TabbyML"

Caused by:
    Permission denied (os error 13)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The error comes from inside of the container, right?
What is wrong here?

Thanks, Jaroslav.

[boxbeam]

Hi @jsyk, we've been unable to reproduce this issue. There are a few things to look at:

Could you use the following command: docker run --entrypoint /opt/tabby/bin/tabby-cpu -it -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.9.1 serve --model TabbyML/StarCoder-1B

This will use version 0.9.1, which might fix the issue you're seeing. If that doesn't, then please check the permission settings for your .tabby folder - please stat it and show us what the output is.

[jsyk]

Hello, unfortunately the problem persists:

jara@fedora:~$ docker run --entrypoint /opt/tabby/bin/tabby-cpu -it -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.9.1 serve --model TabbyML/StarCoder-1B
Unable to find image 'tabbyml/tabby:0.9.1' locally
0.9.1: Pulling from tabbyml/tabby
aece8493d397: Already exists 
976b13f9ef1a: Already exists 
b63c65d3acaa: Already exists 
4dda4739d007: Already exists 
197843862cd5: Already exists 
cf87cabd72e2: Already exists 
7908cc89184b: Already exists 
125997f82033: Already exists 
675adc119131: Already exists 
c7b3b2c7c37e: Pull complete 
c9d060db8b81: Pull complete 
a657e8bdb729: Pull complete 
d9f661e951cd: Pull complete 
Digest: sha256:d008d53722048765293c0d52d729b14d980adabc08fa93e62248143b3beeca24
Status: Downloaded newer image for tabbyml/tabby:0.9.1
thread 'main' panicked at /root/workspace/crates/tabby-common/src/registry.rs:61:21:
Failed to fetch model organization <TabbyML>: Failed to create dir "/data/models/TabbyML"

Caused by:
    Permission denied (os error 13)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
jara@fedora:~$ 

jara@fedora:~$ docker run --entrypoint /opt/tabby/bin/tabby-cpu -it -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.9.1 serve --model TabbyML/StarCoder-1B
thread 'main' panicked at /root/workspace/crates/tabby-common/src/registry.rs:61:21:
Failed to fetch model organization <TabbyML>: Failed to create dir "/data/models/TabbyML"

Caused by:
    Permission denied (os error 13)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
jara@fedora:~$ stat .tabby
  File: .tabby
  Size: 0               Blocks: 0          IO Block: 4096   directory
Device: 0,38    Inode: 118036      Links: 1
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:user_home_t:s0
Access: 2024-03-22 19:22:22.452530014 +0100
Modify: 2024-03-22 19:22:22.452530014 +0100
Change: 2024-03-22 19:22:22.452530014 +0100
 Birth: 2024-03-22 19:22:22.452530014 +0100

jara@fedora:~$ rmdir .tabby

jara@fedora:~$ stat .tabby
stat: cannot statx '.tabby': No such file or directory

jara@fedora:~$ docker run --entrypoint /opt/tabby/bin/tabby-cpu -it -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.9.1 serve --model TabbyML/StarCoder-1B
thread 'main' panicked at /root/workspace/crates/tabby-common/src/registry.rs:61:21:
Failed to fetch model organization <TabbyML>: Failed to create dir "/data/models/TabbyML"

Caused by:
    Permission denied (os error 13)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

jara@fedora:~$ stat .tabby
  File: .tabby
  Size: 0               Blocks: 0          IO Block: 4096   directory
Device: 0,38    Inode: 118043      Links: 1
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:user_home_t:s0
Access: 2024-03-22 19:23:28.392694092 +0100
Modify: 2024-03-22 19:23:28.392694092 +0100
Change: 2024-03-22 19:23:28.392694092 +0100
 Birth: 2024-03-22 19:23:28.392694092 +0100

Naturaly I tried as root, but with identical result:

root@fedora:~# docker run --entrypoint /opt/tabby/bin/tabby-cpu -it -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby:0.9.1 serve --model TabbyML/StarCoder-1B~
thread 'main' panicked at /root/workspace/crates/tabby-common/src/registry.rs:61:21:
Failed to fetch model organization <TabbyML>: Failed to create dir "/data/models/TabbyML"

Caused by:
    Permission denied (os error 13)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

root@fedora:~# stat .tabby
  File: .tabby
  Size: 0               Blocks: 0          IO Block: 4096   directory
Device: 0,34    Inode: 1089828     Links: 1
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:admin_home_t:s0
Access: 2024-03-22 19:19:40.075232087 +0100
Modify: 2024-03-22 19:18:12.824894092 +0100
Change: 2024-03-22 19:18:12.824894092 +0100
 Birth: 2024-03-22 19:18:12.824894092 +0100

root@fedora:~# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

[jsyk]

The problem was caused by selinux. Fedora ships with selinux enabled (enforcing) by default, and I forgot to turn it off on this notebook. With a disabled (perimissive) selinux, all works as expected!

[boxbeam]

Great, thank you for the update! We may want to include information about this in our documentation.