Encounter install error when install vllm package.

[for-just-we]

System Info

Target: x86_64-unknown-linux-gnu
Cargo version: 1.75.0
Commit sha: N/A
Docker label: N/A
nvidia-smi:

  +---------------------------------------------------------------------------------------+  
   | NVIDIA-SMI 535.171.04             Driver Version: 535.171.04   CUDA Version: 12.2     |  
   |-----------------------------------------+----------------------+----------------------+  
   | GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |  
   | Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |  
   |                                         |                      |               MIG M. |  
   |=========================================+======================+======================|  
   |   0  NVIDIA RTX A6000               Off | 00000000:01:00.0 Off |                  Off |  
   | 52%   77C    P2             291W / 300W |  47471MiB / 49140MiB |     96%      Default |  
   |                                         |                      |                  N/A |  
   +-----------------------------------------+----------------------+----------------------+  
   |   1  NVIDIA RTX A6000               Off | 00000000:25:00.0 Off |                  Off |  
   | 38%   67C    P2             266W / 300W |  47527MiB / 49140MiB |     97%      Default |  
   |                                         |                      |                  N/A |  
   +-----------------------------------------+----------------------+----------------------+  
   |   2  NVIDIA RTX A6000               Off | 00000000:41:00.0 Off |                  Off |  
   | 48%   75C    P2             285W / 300W |  47483MiB / 49140MiB |     95%      Default |  
   |                                         |                      |                  N/A |  
   +-----------------------------------------+----------------------+----------------------+  
   |   3  NVIDIA RTX A6000               Off | 00000000:61:00.0 Off |                  Off |  
   | 41%   69C    P2             285W / 300W |  47454MiB / 49140MiB |     98%      Default |  
   |                                         |                      |                  N/A |  
   +-----------------------------------------+----------------------+----------------------+  
   |   4  NVIDIA RTX A6000               Off | 00000000:81:00.0 Off |                  Off |  
   | 49%   75C    P2             297W / 300W |  41417MiB / 49140MiB |    100%      Default |  
   |                                         |                      |                  N/A |  
   +-----------------------------------------+----------------------+----------------------+  
   |   5  NVIDIA RTX A6000               Off | 00000000:A1:00.0 Off |                  Off |  
   | 39%   66C    P2             295W / 300W |  39745MiB / 49140MiB |    100%      Default |  
   |                                         |                      |                  N/A |  
   +-----------------------------------------+----------------------+----------------------+  
|   6  NVIDIA RTX A6000               Off | 00000000:C1:00.0 Off |                  Off |  
   | 47%   73C    P2             293W / 300W |  23995MiB / 49140MiB |    100%      Default |  
   |                                         |                      |                  N/A |  
   +-----------------------------------------+----------------------+----------------------+  
   |   7  NVIDIA RTX A6000               Off | 00000000:E1:00.0 Off |                  Off |  
   | 39%   66C    P2             293W / 300W |  35729MiB / 49140MiB |    100%      Default |  
   |                                         |                      |                  N/A |  
   +-----------------------------------------+----------------------+----------------------+  

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

TGI version: 2.0.2

create an new conda environment, running:

• 1.BUILD_EXTENSIONS=True make install

• 2.cd server

• 3.make install-causal-conv1d

• 4.make install-flash-attention

• 5.make install-flash-attention-v2-cuda

• 6.make install-vllm-cuda

The commands 1-5 are executed normally. But command 6 ends with below error message, in which vllm is installed successfully but vllm-nccl-cu12 failed:

...
creating 'dist/vllm-0.4.1+cu122-py3.11-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing vllm-0.4.1+cu122-py3.11-linux-x86_64.egg
removing '/server9/cbj/programming/anaconda3/envs/tgi_server/lib/python3.11/site-packages/vllm-0.4.1+cu122-py3.11-linux-x86_64.egg' (and everything under it)
creating /server9/cbj/programming/anaconda3/envs/tgi_server/lib/python3.11/site-packages/vllm-0.4.1+cu122-py3.11-linux-x86_64.egg
Extracting vllm-0.4.1+cu122-py3.11-linux-x86_64.egg to /server9/cbj/programming/anaconda3/envs/tgi_server/lib/python3.11/site-packages
Adding vllm 0.4.1+cu122 to easy-install.pth file

Installed /server9/cbj/programming/anaconda3/envs/tgi_server/lib/python3.11/site-packages/vllm-0.4.1+cu122-py3.11-linux-x86_64.egg
Processing dependencies for vllm==0.4.1+cu122
Searching for vllm-nccl-cu12<2.19,>=2.18
Reading https://pypi.org/simple/vllm-nccl-cu12/
Downloading https://files.pythonhosted.org/packages/41/07/c1be8f4ffdc257646dda26470b803487150c732aa5c9f532dd789f186a54/vllm_nccl_cu12-2.18.1.0.4.0.tar.gz#sha256
=d56535da1b893ac49c1f40be9245f999e543c3fc95b4839642b70dd1d72760c0
Best match: vllm-nccl-cu12 2.18.1.0.4.0
Processing vllm_nccl_cu12-2.18.1.0.4.0.tar.gz
Writing /tmp/easy_install-c0u46qco/vllm_nccl_cu12-2.18.1.0.4.0/setup.cfg
Running vllm_nccl_cu12-2.18.1.0.4.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-c0u46qco/vllm_nccl_cu12-2.18.1.0.4.0/egg-dist-tmp-r7fzlcup
error: SandboxViolation: mkdir('/server9/cbj/.config/vllm/nccl/cu12', 511) {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.

After checking, I found the error command is python setup.py install. But I don't understand why vllm_nccl_cu12 will install to /server9/cbj/.config dir instead of my conda environment.

Note, I can successfully execute commands 1-6 in TGI 2.0.1 following Faster-LLM-Survey, installing flash-attention, vllm, ...

Expected behavior

vllm-nccl-cu12 could be installed normally