Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Torch Dependency Resolution problem in Pip #443

Open
arky opened this issue Mar 24, 2024 · 10 comments
Open

Torch Dependency Resolution problem in Pip #443

arky opened this issue Mar 24, 2024 · 10 comments
Labels
Waiting for more info

Comments

@arky
Copy link

arky commented Mar 24, 2024

Currently PytorchWildlife has dependency conflict on linux.

$ python -m pip install PytorchWildlife
Defaulting to user installation because normal site-packages is not writeable
Collecting PytorchWildlife
  Using cached PytorchWildlife-1.0.2.2-py3-none-any.whl.metadata (19 kB)
Requirement already satisfied: numpy in /home/arky/.local/lib/python3.10/site-packages (from PytorchWildlife) (1.26.3)
INFO: pip is looking at multiple versions of pytorchwildlife to determine which version is compatible with other requirements. This could take a while.
  Using cached PytorchWildlife-1.0.1.1-py3-none-any.whl.metadata (18 kB)
  Using cached PytorchWildlife-1.0.1-py3-none-any.whl.metadata (18 kB)
  Using cached PytorchWildlife-1.0.0-py3-none-any.whl.metadata (4.2 kB)
  Using cached PytorchWildlife-0.0.1.1.2-py3-none-any.whl.metadata (4.2 kB)
  Using cached PytorchWildlife-0.0.1.1.1-py3-none-any.whl.metadata (2.5 kB)
  Using cached PytorchWildlife-0.0.1-py3-none-any.whl.metadata (2.5 kB)
  Using cached PytorchWildlife-0.0.0-py3-none-any.whl.metadata (2.5 kB)
INFO: pip is still looking at multiple versions of pytorchwildlife to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install pytorchwildlife==0.0.0, pytorchwildlife==0.0.1, pytorchwildlife==0.0.1.1.1, pytorchwildlife==0.0.1.1.2, pytorchwildlife==1.0.0, pytorchwildlife==1.0.1, pytorchwildlife==1.0.1.1 and pytorchwildlife==1.0.2.2 because these package versions have conflicting dependencies.

The conflict is caused by:
    pytorchwildlife 1.0.2.2 depends on torch==1.10.1
    pytorchwildlife 1.0.1.1 depends on torch==1.10.1
    pytorchwildlife 1.0.1 depends on torch==1.10.1
    pytorchwildlife 1.0.0 depends on torch==1.10.1
    pytorchwildlife 0.0.1.1.2 depends on torch==1.10.1
    pytorchwildlife 0.0.1.1.1 depends on torch==1.10.1
    pytorchwildlife 0.0.1 depends on torch==1.10.1
    pytorchwildlife 0.0.0 depends on torch==1.10.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
@zhmiao
Copy link
Collaborator

zhmiao commented Mar 24, 2024

Hello @arky, thanks for reporting this. Is this on a fresh python environment or are you trying to install PytorchWildlife on an existing environment? We wonder if this is caused by some existing packages.

@arky
Copy link
Author

arky commented Mar 25, 2024

@zhmiao Apologies, I have replicated this issue with 1) default system packages on ubuntu LTS 2) Under a fresh venv virtual envirnoment with Python (3.10.12) and Pip (24.0).

@arky
Copy link
Author

arky commented Mar 25, 2024

Providing a conda based testcase for this issue

$ conda  -V 
conda 23.11.0

# Test case 
$ conda create -n pytwildlife 
Retrieving notices: ...working... done
Channels:
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): | /-|/\|/done
Solving environment: done
## Package Plan ##

  environment location: /home/arky/anaconda3/envs/pytwildlife



Proceed ([y]/n)? y
Invalid choice: source acy
Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate pytwildlife
#
# To deactivate an active environment, use
#
#     $ conda deactivate

$ conda activate pytwildlife
$ pip install pytorchwildlife
Defaulting to user installation because normal site-packages is not writeable
Collecting pytorchwildlife
  Downloading PytorchWildlife-1.0.2.4.tar.gz (30 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in ./.local/lib/python3.10/site-packages (from pytorchwildlife) (1.26.3)
INFO: pip is looking at multiple versions of pytorchwildlife to determine which version is compatible with other requirements. This could take a while.
  Downloading PytorchWildlife-1.0.2.3.tar.gz (30 kB)
  Preparing metadata (setup.py) ... done
  Using cached PytorchWildlife-1.0.2.2-py3-none-any.whl.metadata (19 kB)
  Using cached PytorchWildlife-1.0.1.1-py3-none-any.whl.metadata (18 kB)
  Using cached PytorchWildlife-1.0.1-py3-none-any.whl.metadata (18 kB)
  Using cached PytorchWildlife-1.0.0-py3-none-any.whl.metadata (4.2 kB)
  Using cached PytorchWildlife-0.0.1.1.2-py3-none-any.whl.metadata (4.2 kB)
  Using cached PytorchWildlife-0.0.1.1.1-py3-none-any.whl.metadata (2.5 kB)
INFO: pip is still looking at multiple versions of pytorchwildlife to determine which version is compatible with other requirements. This could take a while.
  Using cached PytorchWildlife-0.0.1-py3-none-any.whl.metadata (2.5 kB)
  Using cached PytorchWildlife-0.0.0-py3-none-any.whl.metadata (2.5 kB)
ERROR: Cannot install pytorchwildlife==0.0.0, pytorchwildlife==0.0.1, pytorchwildlife==0.0.1.1.1, pytorchwildlife==0.0.1.1.2, pytorchwildlife==1.0.0, pytorchwildlife==1.0.1, pytorchwildlife==1.0.1.1, pytorchwildlife==1.0.2.2, pytorchwildlife==1.0.2.3 and pytorchwildlife==1.0.2.4 because these package versions have conflicting dependencies.

The conflict is caused by:
    pytorchwildlife 1.0.2.4 depends on torch==1.10.1
    pytorchwildlife 1.0.2.3 depends on torch==1.10.1
    pytorchwildlife 1.0.2.2 depends on torch==1.10.1
    pytorchwildlife 1.0.1.1 depends on torch==1.10.1
    pytorchwildlife 1.0.1 depends on torch==1.10.1
    pytorchwildlife 1.0.0 depends on torch==1.10.1
    pytorchwildlife 0.0.1.1.2 depends on torch==1.10.1
    pytorchwildlife 0.0.1.1.1 depends on torch==1.10.1
    pytorchwildlife 0.0.1 depends on torch==1.10.1
    pytorchwildlife 0.0.0 depends on torch==1.10.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

@aa-hernandez
Copy link
Collaborator

Hi @arky!

Thank you so much for sharing these details. Currently, PytorchWildlife requires Python 3.8 to work. Could you please run this command to create a Python 3.8 environment? It should solve the issue!

conda create -n pytorch-wildlife python=3.8 -y

You can find a step-by-step guide in our installation instructions, please let us know if it works and if you have further questions. Thank you :)

@u17057842
Copy link

Hello, @aa-hernandez, I experienced the same dependancy conflict, using a fresh env with python=3.8.

@aa-hernandez
Copy link
Collaborator

Hi @u17057842!

Thank you for reporting this error with us. Could you please share the error message so we can find the source of the issue? Thank you :)

@zhmiao
Copy link
Collaborator

zhmiao commented Mar 27, 2024

Also @arky, I see the error is caused by torch==1.10.1. I wonder if you have a torch preinstalled with a different version?

@arky
Copy link
Author

arky commented Mar 28, 2024 •
edited

@zhmiao There is no pre-existing torch installation as this new virtual environment. I think @aa-hernandez might be right that the is relation to python version.

$ python -V
Python 3.10.12

$ python -m venv ~/.venv/pytorch-wildlife

arky@arky-thinkpad:~$ source ~/.venv/pytorch-wildlife/

arky@arky-thinkpad:~$ source ~/.venv/pytorch-wildlife/bin/activate

(pytorch-wildlife) arky@arky-thinkpad:~$ python -m pip install PytorchWildlife
Collecting PytorchWildlife
  Downloading PytorchWildlife-1.0.2.5-py3-none-any.whl (29 kB)
Collecting supervision==0.16.0
  Downloading supervision-0.16.0-py3-none-any.whl (72 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.2/72.2 KB 1.3 MB/s eta 0:00:00
Collecting Pillow==10.1.0
  Downloading Pillow-10.1.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 6.3 MB/s eta 0:00:00
Collecting PytorchWildlife
  Downloading PytorchWildlife-1.0.2.4.tar.gz (30 kB)
  Preparing metadata (setup.py) ... done
Collecting chardet
  Downloading chardet-5.2.0-py3-none-any.whl (199 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.4/199.4 KB 10.0 MB/s eta 0:00:00
Collecting gradio==4.8.0
  Downloading gradio-4.8.0-py3-none-any.whl (16.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.5/16.5 MB 3.8 MB/s eta 0:00:00
Collecting numpy
  Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 2.2 MB/s eta 0:00:00
Collecting PytorchWildlife
  Downloading PytorchWildlife-1.0.2.3.tar.gz (30 kB)
  Preparing metadata (setup.py) ... done
  Downloading PytorchWildlife-1.0.2.2-py3-none-any.whl (28 kB)
  Downloading PytorchWildlife-1.0.1.1-py3-none-any.whl (24 kB)
  Downloading PytorchWildlife-1.0.1-py3-none-any.whl (25 kB)
  Downloading PytorchWildlife-1.0.0-py3-none-any.whl (18 kB)
  Downloading PytorchWildlife-0.0.1.1.2-py3-none-any.whl (18 kB)
  Downloading PytorchWildlife-0.0.1.1.1-py3-none-any.whl (17 kB)
  Downloading PytorchWildlife-0.0.1-py3-none-any.whl (17 kB)
  Downloading PytorchWildlife-0.0.0-py3-none-any.whl (8.5 kB)
ERROR: Cannot install pytorchwildlife==0.0.0, pytorchwildlife==0.0.1, pytorchwildlife==0.0.1.1.1, pytorchwildlife==0.0.1.1.2, pytorchwildlife==1.0.0, pytorchwildlife==1.0.1, pytorchwildlife==1.0.1.1, pytorchwildlife==1.0.2.2, pytorchwildlife==1.0.2.3, pytorchwildlife==1.0.2.4 and pytorchwildlife==1.0.2.5 because these package versions have conflicting dependencies.

The conflict is caused by:
    pytorchwildlife 1.0.2.5 depends on torch==1.10.1
    pytorchwildlife 1.0.2.4 depends on torch==1.10.1
    pytorchwildlife 1.0.2.3 depends on torch==1.10.1
    pytorchwildlife 1.0.2.2 depends on torch==1.10.1
    pytorchwildlife 1.0.1.1 depends on torch==1.10.1
    pytorchwildlife 1.0.1 depends on torch==1.10.1
    pytorchwildlife 1.0.0 depends on torch==1.10.1
    pytorchwildlife 0.0.1.1.2 depends on torch==1.10.1
    pytorchwildlife 0.0.1.1.1 depends on torch==1.10.1
    pytorchwildlife 0.0.1 depends on torch==1.10.1
    pytorchwildlife 0.0.0 depends on torch==1.10.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

@arky
Copy link
Author

arky commented Mar 28, 2024

@aa-hernandez Conda seems to work now. But pip doesn't work

$ conda create -n pytorch-wildlife python=3.8 -y
Retrieving notices: ...working... done
Channels:
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/arky/anaconda3/envs/pytorch-wildlife

  added / updated specs:
    - python=3.8


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    _libgcc_mutex-0.1          |             main           3 KB
    _openmp_mutex-5.1          |            1_gnu          21 KB
    ca-certificates-2024.3.11  |       h06a4308_0         127 KB
    ld_impl_linux-64-2.38      |       h1181459_1         654 KB
    libffi-3.4.4               |       h6a678d5_0         142 KB
    libgcc-ng-11.2.0           |       h1234567_1         5.3 MB
    libgomp-11.2.0             |       h1234567_1         474 KB
    libstdcxx-ng-11.2.0        |       h1234567_1         4.7 MB
    ncurses-6.4                |       h6a678d5_0         914 KB
    openssl-3.0.13             |       h7f8727e_0         5.2 MB
    pip-23.3.1                 |   py38h06a4308_0         2.6 MB
    python-3.8.19              |       h955ad1f_0        23.8 MB
    readline-8.2               |       h5eee18b_0         357 KB
    setuptools-68.2.2          |   py38h06a4308_0         948 KB
    sqlite-3.41.2              |       h5eee18b_0         1.2 MB
    tk-8.6.12                  |       h1ccaba5_0         3.0 MB
    wheel-0.41.2               |   py38h06a4308_0         108 KB
    xz-5.4.6                   |       h5eee18b_0         651 KB
    zlib-1.2.13                |       h5eee18b_0         103 KB
    ------------------------------------------------------------
                                           Total:        50.2 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main 
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu 
  ca-certificates    pkgs/main/linux-64::ca-certificates-2024.3.11-h06a4308_0 
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1 
  libffi             pkgs/main/linux-64::libffi-3.4.4-h6a678d5_0 
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1 
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1 
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1 
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0 
  openssl            pkgs/main/linux-64::openssl-3.0.13-h7f8727e_0 
  pip                pkgs/main/linux-64::pip-23.3.1-py38h06a4308_0 
  python             pkgs/main/linux-64::python-3.8.19-h955ad1f_0 
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0 
  setuptools         pkgs/main/linux-64::setuptools-68.2.2-py38h06a4308_0 
  sqlite             pkgs/main/linux-64::sqlite-3.41.2-h5eee18b_0 
  tk                 pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0 
  wheel              pkgs/main/linux-64::wheel-0.41.2-py38h06a4308_0 
  xz                 pkgs/main/linux-64::xz-5.4.6-h5eee18b_0 
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0 



Downloading and Extracting Packages:
                                                                                                         
Preparing transaction: done                                                                              
Verifying transaction: done                                                                              
Executing transaction: done                                                                              
#                                                                                                        
# To activate this environment, use                                                                      
#                                                                                                        
#     $ conda activate pytorch-wildlife                                                                  
#                                                                                                        
# To deactivate an active environment, use                                                               
#                                                                                                        
#     $ conda deactivate                         

~$ conda create -n pytorch-wildlife-3.10 python=3.10 -y                              
Channels:                                                                                                
 - defaults                                                                                              
Platform: linux-64                                                                                       
Collecting package metadata (repodata.json): done                                                        
Solving environment: done

## Package Plan ##

  environment location: /home/arky/anaconda3/envs/pytorch-wildlife-3.10

  added / updated specs:
    - python=3.10


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    bzip2-1.0.8                |       h5eee18b_5         262 KB
    libuuid-1.41.5             |       h5eee18b_0          27 KB
    pip-23.3.1                 |  py310h06a4308_0         2.7 MB
    python-3.10.14             |       h955ad1f_0        26.8 MB
    setuptools-68.2.2          |  py310h06a4308_0         957 KB
    tzdata-2024a               |       h04d1e81_0         116 KB
    wheel-0.41.2               |  py310h06a4308_0         109 KB
    ------------------------------------------------------------
                                           Total:        30.9 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main 
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu 
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_5 
  ca-certificates    pkgs/main/linux-64::ca-certificates-2024.3.11-h06a4308_0 
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1 
  libffi             pkgs/main/linux-64::libffi-3.4.4-h6a678d5_0 
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1 
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1 
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1 
  libuuid            pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0 
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0 
  openssl            pkgs/main/linux-64::openssl-3.0.13-h7f8727e_0 
  pip                pkgs/main/linux-64::pip-23.3.1-py310h06a4308_0 
  python             pkgs/main/linux-64::python-3.10.14-h955ad1f_0 
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0 
  setuptools         pkgs/main/linux-64::setuptools-68.2.2-py310h06a4308_0 
  sqlite             pkgs/main/linux-64::sqlite-3.41.2-h5eee18b_0 
  tk                 pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0 
  tzdata             pkgs/main/noarch::tzdata-2024a-h04d1e81_0 
  wheel              pkgs/main/linux-64::wheel-0.41.2-py310h06a4308_0 
  xz                 pkgs/main/linux-64::xz-5.4.6-h5eee18b_0 
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0 



Downloading and Extracting Packages:
                                                                                                         
Preparing transaction: done                                                                              
Verifying transaction: done                                                                              
Executing transaction: done                                                                              
#                                                                                                        
# To activate this environment, use                                                                      
#                                                                                                        
#     $ conda activate pytorch-wildlife-3.10
#
# To deactivate an active environment, use
#
#     $ conda deactivate

@arky arky changed the title Torch Dependency Resolution problem Torch Dependency Resolution problem in Pip Mar 28, 2024
@zhmiao
Copy link
Collaborator

zhmiao commented Mar 28, 2024

@arky, does it work on Python 3.10 in your conda environment? Generally it shouldn't because torch 1.10.1 does not support Python 3.10. The reason we use torch 1.10.1 is because YOLOv5 package has some functions that does not support higher versions of Pytorch. We are considering either create a customed YOLOv5 that solves this issue or replace it in future versions of Megadetector. But it is weird pip venv doesn't work. We never used pip venv before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Waiting for more info
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@arky @zhmiao @aa-hernandez @u17057842