Python version?

[jamesheatonrdm]

It is not clear what Python version is suitable for this project. I am unable to install the required dependencies when using versions 3.6, 3.8, 3.9, 3.10 and 3.11

python3.11:

Collecting cuml-cu11 (from -r requirements.txt (line 10))
  Downloading cuml-cu11-24.2.0.tar.gz (6.9 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/home/jheaton/ML/Automatic Labelling/dinov2/venvs/dinov2/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/jheaton/ML/Automatic Labelling/dinov2/venvs/dinov2/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/jheaton/ML/Automatic Labelling/dinov2/venvs/dinov2/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-ac3u0_zd/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-ac3u0_zd/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-ac3u0_zd/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-ac3u0_zd/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 90, in <module>
      RuntimeError: Bad params
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

python3.10:

Collecting ptxcompiler-cu11
  Using cached ptxcompiler-cu11-0.8.1.tar.gz (6.9 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-4pgp33xy/ptxcompiler-cu11_3b0336377e224ee8b4d722d27a01fc8b/setup.py", line 90, in <module>
          raise RuntimeError("Bad params")
      RuntimeError: Bad params
      [end of output]

python3.9:

Collecting ptxcompiler-cu11
  Using cached ptxcompiler-cu11-0.8.1.tar.gz (6.9 kB)
    ERROR: Command errored out with exit status 1:
     command: '/home/jheaton/ML/Automatic Labelling/dinov2/venvs/dinov2/bin/python3.9' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yua3d_3u/ptxcompiler-cu11/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yua3d_3u/ptxcompiler-cu11/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-yua3d_3u/ptxcompiler-cu11/pip-egg-info
         cwd: /tmp/pip-install-yua3d_3u/ptxcompiler-cu11/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-yua3d_3u/ptxcompiler-cu11/setup.py", line 90, in <module>
        raise RuntimeError("Bad params")
    RuntimeError: Bad params
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

python3.8:

Collecting cuml-cu11
  Using cached cuml-cu11-23.12.0.tar.gz (6.8 kB)
    ERROR: Command errored out with exit status 1:
     command: '/home/jheaton/ML/Automatic Labelling/dinov2/dinov2/bin/python3.8' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-0lp6ogs6/cuml-cu11/setup.py'"'"'; __file__='"'"'/tmp/pip-install-0lp6ogs6/cuml-cu11/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-0lp6ogs6/cuml-cu11/pip-egg-info
         cwd: /tmp/pip-install-0lp6ogs6/cuml-cu11/
    Complete output (15 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-0lp6ogs6/cuml-cu11/setup.py", line 137, in <module>
        raise RuntimeError(open("ERROR.txt", "r").read())
    RuntimeError:
    ###########################################################################################
    The package you are trying to install is only a placeholder project on PyPI.org repository.
    This package is hosted on NVIDIA Python Package Index.
    
    This package can be installed as:
    ```
    $ pip install --no-cache-dir --extra-index-url https://pypi.nvidia.com cuml-cu11
    ```
    ###########################################################################################
    
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

The pip install supposed fix in the above issue does not work

python3.6:

Collecting torch==2.0.0 (from -r requirements.txt (line 2))
  Could not find a version that satisfies the requirement torch==2.0.0 (from -r requirements.txt (line 2)) (from versions: 1.0.0, 1.0.1, 1.0.1.post2, 1.1.0, 1.2.0, 1.3.0, 1.3.1, 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.8.1, 1.9.0, 1.9.1, 1.10.0, 1.10.1, 1.10.2)
No matching distribution found for torch==2.0.0 (from -r requirements.txt (line 2))

Does anyone have any advice as to which python version I should be using? I have CUDA 12.1 installed on my Ubuntu 20.04 system.

I have attempted to create the conda environment but this takes a VERY long time - it is still solving the environment after I have tested all of these different python versions with pip. Hopefully using conda will work but I would also like to know why seemingly no python versions appear to work.

[chokevin8]

I have the exact same problem here. Exact same error message with no versions of python working. Tried upgrading pip and setuptools with no success. Conda environment solving takes forever too. Any help from any of the contributors would be appreciated, as it seems like this is a recent issue (I don't see any other similar issues in the Issues tab). Thank you!

[jamesheatonrdm]

@chokevin8

I have just come back to the conda install and it has failed saying that my installed CUDA version (12.1) is incompatible

[jamesheatonrdm]

The issue seems to be with ptxcompiler-cu11. There doesn't appear to be much documentation for this python package, and the error when installing is not useful at all so I am unsure where to go from here.

[jamesheatonrdm]

@chokevin8
I manage to run the segmentation notebook without installing the cuml-cu11 package. Here is how I did it:

Create conda environment with python 3.10.13

conda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install xformers::xformers
conda install conda-forge::omegaconf
conda install fvcore
conda install iopath
conda install pip
pip install git+https://github.com/facebookincubator/submitit
pip install ftfy
pip install regex
pip install -U openmim
mim install mmcv==1.5.0
mim install mmsegmentation==0.27.0

Don't ask my why this works because I have no idea. But these steps can get you to run segmentation inference, which is what I was wanting from this repo

[chokevin8]

@jamesheatonrdm can confirm that this also works for training. Thank you so much! I also have no idea why this works.

[Al3cL1]

@chokevin8 Did you ever run into a "Failed building wheel for mmcv-full":

Building wheels for collected packages: mmcv-full
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Building wheel for mmcv-full (setup.py) ... error
  ERROR: Failed building wheel for mmcv-full
  Running setup.py clean for mmcv-full
Failed to build mmcv-full
ERROR: Could not build wheels for mmcv-full, which is required to install pyproject.toml-based projects

Seems to be a recurring issue with my code, and i dont seem to be able to find much of a solution online

[chokevin8]

@Al3cL1 Hmm, no, do you get this when running what @jamesheatonrdm suggested (mentioned below)? Maybe deactivate and remove your environment and create a new environment with python 3.10.13 if you already haven't. I remember I did have to separately install some other required packages for training like fvcore afterwards, but I didn't get an error like yours.

@chokevin8 I manage to run the segmentation notebook without installing the cuml-cu11 package. Here is how I did it:

Create conda environment with python 3.10.13

conda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install xformers::xformers
conda install conda-forge::omegaconf
conda install fvcore
conda install iopath
conda install pip
pip install git+https://github.com/facebookincubator/submitit
pip install ftfy
pip install regex
pip install -U openmim
mim install mmcv==1.5.0
mim install mmsegmentation==0.27.0

Don't ask my why this works because I have no idea. But these steps can get you to run segmentation inference, which is what I was wanting from this repo

[Al3cL1]

@Al3cL1 Hmm, no, do you get this when running what @jamesheatonrdm suggested (mentioned below)? Maybe deactivate and remove your environment and create a new environment with python 3.10.13 if you already haven't. I remember I did have to separately install some other required packages for training like fvcore afterwards, but I didn't get an error like yours.

@chokevin8 I manage to run the segmentation notebook without installing the cuml-cu11 package. Here is how I did it:
Create conda environment with python 3.10.13

conda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install xformers::xformers
conda install conda-forge::omegaconf
conda install fvcore
conda install iopath
conda install pip
pip install git+https://github.com/facebookincubator/submitit
pip install ftfy
pip install regex
pip install -U openmim
mim install mmcv==1.5.0
mim install mmsegmentation==0.27.0

Don't ask my why this works because I have no idea. But these steps can get you to run segmentation inference, which is what I was wanting from this repo

Yeah no ok I think its an issue with my cuda/cudatoolkit version, which might be affecting dependencies. I think I got it thought.

[zshn25]

conda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install xformers::xformers
conda install conda-forge::omegaconf
conda install fvcore
conda install iopath
conda install pip
pip install git+https://github.com/facebookincubator/submitit
pip install ftfy
pip install regex
pip install -U openmim
mim install mmcv==1.5.0
mim install mmsegmentation==0.27.0

A conda.yml file would be as follows

name: dinov2
channels:
  - defaults
  - pytorch
  - nvidia
  - xformers
  - conda-forge
dependencies:
  - python=3.10
  - pytorch::pytorch=2.0.0
  - pytorch::pytorch-cuda=11.7.0
  - pytorch::torchvision=0.15.0
  - omegaconf
  - torchmetrics=0.10.3
  - fvcore
  - iopath
  - xformers::xformers
  - tensorboard
  - pip
  - pip:
    - git+https://github.com/facebookincubator/submitit
    - --extra-index-url https://pypi.nvidia.com
    - cuml-cu11
    - openmim

you can install it with conda env update --file conda.yaml