Cannot build lock even though dependencies are found.

[jeschkies]

Be sure to check the existing issues, both open and closed.

Describe the issue briefly here.

Describe you environment

1. macOS Sierra
2. Python 3.5.2
3. pipenv, version 8.2.7

Expected result

I want to install shakedown with pipenv --three install dcos-shakedown. It should create a Pipfile and a lock.

Actual result

I get the following output

CRITICAL:pip.index:Could not find a version that satisfies the requirement dcoscli==0.5.5 (from versions: 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.2.0, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14)
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
No matching distribution found for dcoscli==0.5.5

This is odd since pipenv --three install dcoscli can install 0.5.5 and create a lock. There is a dcoscli-0.5.5 on PyPi. I assumes that pipenv tries to create a closure for Python 2 as well but there is no dcoscli 0.5.5 for Python 2.

Steps to replicate

Just run pipenv --three install --verbose dcos-shakedown with Python 3.5.

[jeschkies]

The shakedown package should probably be Python 3 only. However, I think that pipenv should create the lock only for a specific Python version if the Pipefile includes a required Python version.

[jeschkies]

Here is the verbose output of the lock

pipenv --three lock --verbose

Locking [dev-packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done
Locking [packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1
Current constraints:
  dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown

Finding the best candidates:
  found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was <any>)

Finding secondary dependencies:

New dependencies found in this round:
  adding [u'click', '', '[]']
  adding [u'dcos-shakedown', '', '[]']
  adding [u'dcoscli', '==0.5.5', '[]']
  adding [u'paramiko', '', '[]']
  adding [u'pytest', '', '[]']
  adding [u'pytest-timeout', '', '[]']
  adding [u'retrying', '', '[]']
  adding [u'scp', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  click
  dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown
  dcoscli==0.5.5
  paramiko
  pytest
  pytest-timeout
  retrying
  scp

Finding the best candidates:
  found candidate click==6.7 (constraint was <any>)
  found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was <any>)
  found candidate dcoscli==0.5.5 (constraint was ==0.5.5)
  found candidate paramiko==2.3.1 (constraint was <any>)
  found candidate pytest==3.2.3 (constraint was <any>)
  found candidate pytest-timeout==1.2.0 (constraint was <any>)
  found candidate retrying==1.3.3 (constraint was <any>)
  found candidate scp==0.10.2 (constraint was <any>)

Finding secondary dependencies:
  click==6.7                requires click==6.7
  scp==0.10.2 not in cache, need to check index
  scp==0.10.2               requires paramiko, scp==0.10.2
  pytest-timeout==1.2.0     requires pytest-timeout==1.2.0, pytest>=2.8.0
  dcoscli==0.5.5 not in cache, need to check index
CRITICAL:pip.index:Could not find a version that satisfies the requirement dcoscli==0.5.5 (from versions: 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.2.0, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14)
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
No matching distribution found for dcoscli==0.5.5

[jeschkies]

This is odd. When I install dcsocli with pipenv --three install dcoscli I see

Installing dcoscli…
Collecting dcoscli
  Using cached dcoscli-0.5.5-py3-none-any.whl

followed by

                          ROUND 1
Current constraints:
  dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown
  dcoscli

Finding the best candidates:
  found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was <any>)
  found candidate dcoscli==0.4.14 (constraint was <any>)

Finding secondary dependencies:
  dcoscli==0.4.14           requires dcos==0.4.14, dcoscli==0.4.14, docopt<1.0,>=0.6, pkginfo==1.2.1, toml<1.0,>=0.9, virtualenv<16.0,>=13.0

why is it trying to install 0.4.14 of dcoscli? Especially when pipenv --three run dcos --version outputs dcoscli.version=0.5.5.

[techalchemy]

It’s not. This is the locking process. It’s doing dependency resolution based on what it know and what you have cached. Try pipenv lock —clear

…

On Oct 18, 2017, at 6:16 AM, Karsten Jeschkies ***@***.***> wrote: This is odd. When I install dcsocli with pipenv --three install dcoscli I see Installing dcoscli… Collecting dcoscli Using cached dcoscli-0.5.5-py3-none-any.whl followed by ROUND 1 Current constraints: dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown dcoscli Finding the best candidates: found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was <any>) found candidate dcoscli==0.4.14 (constraint was <any>) Finding secondary dependencies: dcoscli==0.4.14 requires dcos==0.4.14, dcoscli==0.4.14, docopt<1.0,>=0.6, pkginfo==1.2.1, toml<1.0,>=0.9, virtualenv<16.0,>=13.0 why is it trying to install 0.4.14 of dcoscli? Especially when pipenv --three run dcos --version outputs dcoscli.version=0.5.5. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[techalchemy]

You’ll need to provide the output of pipenv lock —verbose and pipenv —version

…

On Oct 18, 2017, at 6:16 AM, Karsten Jeschkies ***@***.***> wrote: This is odd. When I install dcsocli with pipenv --three install dcoscli I see Installing dcoscli… Collecting dcoscli Using cached dcoscli-0.5.5-py3-none-any.whl followed by ROUND 1 Current constraints: dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown dcoscli Finding the best candidates: found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was <any>) found candidate dcoscli==0.4.14 (constraint was <any>) Finding secondary dependencies: dcoscli==0.4.14 requires dcos==0.4.14, dcoscli==0.4.14, docopt<1.0,>=0.6, pkginfo==1.2.1, toml<1.0,>=0.9, virtualenv<16.0,>=13.0 why is it trying to install 0.4.14 of dcoscli? Especially when pipenv --three run dcos --version outputs dcoscli.version=0.5.5. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[jeschkies]

Clearing the lock cache did not help. Here is the full output of pipenv --three lock --verbose

› pipenv --three lock --verbose
Virtualenv already exists!
Removing existing virtualenv…
Warning: PYENV_ROOT is not set. New python paths will probably not be exported properly after installation.
Creating a virtualenv for this project…
Using /Users/kjeschkies/.pyenv/shims/python3 to create virtualenv…
⠋Running virtualenv with interpreter /Users/kjeschkies/.pyenv/shims/python3
Using base prefix '/Users/kjeschkies/.pyenv/versions/3.5.2'
New python executable in /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI/bin/python3
Also creating executable in /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done
Locking [packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1
Current constraints:
  dcos-shakedown

Finding the best candidates:
  found candidate dcos-shakedown==1.4.8 (constraint was <any>)

Finding secondary dependencies:
  dcos-shakedown==1.4.8     requires click, dcos-shakedown==1.4.8, dcoscli==0.5.5, paramiko, pytest, pytest-timeout, retrying, scp

New dependencies found in this round:
  adding [u'click', '', '[]']
  adding [u'dcos-shakedown', '==1.4.8', '[]']
  adding [u'dcoscli', '==0.5.5', '[]']
  adding [u'paramiko', '', '[]']
  adding [u'pytest', '', '[]']
  adding [u'pytest-timeout', '', '[]']
  adding [u'retrying', '', '[]']
  adding [u'scp', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  click
  dcos-shakedown==1.4.8
  dcoscli==0.5.5
  paramiko
  pytest
  pytest-timeout
  retrying
  scp

Finding the best candidates:
  found candidate click==6.7 (constraint was <any>)
  found candidate dcos-shakedown==1.4.8 (constraint was ==1.4.8)
  found candidate dcoscli==0.5.5 (constraint was ==0.5.5)
  found candidate paramiko==2.3.1 (constraint was <any>)
  found candidate pytest==3.2.3 (constraint was <any>)
  found candidate pytest-timeout==1.2.0 (constraint was <any>)
  found candidate retrying==1.3.3 (constraint was <any>)
  found candidate scp==0.10.2 (constraint was <any>)

Finding secondary dependencies:
  pytest-timeout==1.2.0 not in cache, need to check index
  pytest-timeout==1.2.0     requires pytest-timeout==1.2.0, pytest>=2.8.0
  paramiko==2.3.1           requires bcrypt>=3.1.3, cryptography>=1.5, paramiko==2.3.1, pyasn1>=0.1.7, pynacl>=1.0.1
  click==6.7 not in cache, need to check index
  click==6.7                requires click==6.7
  pytest==3.2.3             requires py>=1.4.33, pytest==3.2.3, setuptools
  dcos-shakedown==1.4.8     requires click, dcos-shakedown==1.4.8, dcoscli==0.5.5, paramiko, pytest, pytest-timeout, retrying, scp
  retrying==1.3.3           requires retrying==1.3.3, six>=1.7.0
  dcoscli==0.5.5 not in cache, need to check index
CRITICAL:pip.index:Could not find a version that satisfies the requirement dcoscli==0.5.5 (from versions: 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.2.0, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14)
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
No matching distribution found for dcoscli==0.5.5

[techalchemy]

FYI every time you pass --three you are telling pipenv to destroy your existing virtualenv, not sure if that was intended but just FYI. I will look into this a bit more but at a glance since you have pyenv installed you should set the environment variable PYENV_ROOT per the error message also.

[jeschkies]

@techalchemy, thanks. I did not know that --three would create a new env each time. I thought it would just tell pipenv to use only Python 3. I did set PYENV_ROOT once but then pipenv created a virtuelenv with Python 3.6 instead of 3.5 as defined in my local pyenv.

[jeschkies]

I still think that the issue is that pipenv tries to create a closure for Python 2 and that shakedown is not limited to Python 3.

[vphilippon]

Throwing in my observation on the dependency resolution behavior:
It really looks like the Python 2 or pip2 executable is used for the dependency resolution here.
I have the feeling I've seen similar issues lately, but haven't had the time to look into it.

Taking a guess here on the environment here, I'd say pipenv was installed globally (which is ok), and that the global or root python is Python 2. @jeschkies Can you confirm if that's right? Note, I'm not talking about the python version of the virtualenv created by pipenv, I'm talking about the python executable used to run pipenv.
I've also noted that you seem to be using pyenv, I'm not sure if there's any side effect here on the python executable being used by pipenv itself..

I'm not an expert (yet) on how pipenv determines which python/pip version to use for itself, but if it just blindly picks the root python of the environment in which it's installed for itself, then we know why we get python 2 results here.

[jeschkies]

Thanks for the pointers @vphilippon. I've ripped out my old pyenv version and installed the latest and Python 3.5.2. I then updated pip to 9.0.1 outside of the virtualenv. I also patched shakedown to require a specific Python version and used git dependencies. See
• https://github.com/jeschkies/shakedown/blob/master/setup.py
• https://github.com/mesosphere/marathon/blob/karsten/use-pipenv/Pipfile

I wonder how the lock works since it does not mention the commit hash as far as I can tell. Also, the lock states the dcos cli version as 0.4.14 but pipenv run dcos --version outputs 0.5.5.

Overall I wonder how pipenv is handling different Python versions and what should be expected from pipenv in that regard.

[jeschkies]

Hm the lock does not work for the dependencies:

system in marathon/ on karsten/use-pipenv
› pipenv --rm
Removing virtualenv (/Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI)…

system in marathon/ on karsten/use-pipenv
› pipenv install
Creating a virtualenv for this project…
Using /Users/kjeschkies/.pyenv/versions/3.5.2/bin/python3.5m to create virtualenv…
⠋Running virtualenv with interpreter /Users/kjeschkies/.pyenv/versions/3.5.2/bin/python3.5m
Using base prefix '/Users/kjeschkies/.pyenv/versions/3.5.2'
New python executable in /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI/bin/python3.5m
Also creating executable in /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI
Installing dependencies from Pipfile.lock (0d5d00)…
Ignoring ipaddress: markers 'python_version < "3"' don't match your environment
Ignoring enum34: markers 'python_version < "3"' don't match your environment
Ignoring futures: markers 'python_version == "2.7"' don't match your environment
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 34/34 — 00:00:08
To activate this project's virtualenv, run the following:
 $ pipenv shell

system in marathon/ on karsten/use-pipenv
› pipenv run dcos --version
dcoscli.version=0.4.14
dcos.version=1.9.4
dcos.commit=f843772e2740889cc4bba263db44f1403e670037
dcos.bootstrap-id=79f6d6e1c406bffd3fab28b36815220575ae5995

So basically installing with pipenv install -e git+https://github.com/jeschkies/shakedown#egg=dcos-shakedown creating a Pipefile.lock that is not quite accurate.

[techalchemy]

Well the command you ran just reused the lockfile you had present already, I believe pipenv first needs to be told which version of python it is supposed to use if you need a different version from the default for your use case

[vphilippon]

Looking at that, the default, the venv seems to use Python 3.5.2, and evaluates markers as such. That looks like a step forward.

@jeschkies I would delete the Pipfile.lock and run pipenv lock --clear to go on a fersh locking. If the root python issue is no longer in cause, that should do it.

[techalchemy]

I agree with @vphilippon but if you have further issues I would ask if you can provide your Pipfile and Pipfile.lock so that we can assist with the troubleshooting process further

[jeschkies]

I did start fresh without a Pipfile.lock and then used the lock file to recreate the env. I will try it one more time a report back.

[jeschkies]

Alright. I've reproduced the issue in a clean folder. You see all commands out the output in this gist.

Here is what I try to do and what happens:

1. Install shakedown from git for Python 3.5 with pipenv --three -e git+https://github.com/jeschkies/shakedown.git#egg=dcos-shakedown
2. dcos-cli, one dependency of shakedown is install with version 0.5.5. This is as expected.
3. Remove the virtualenv with pipenv --rm.
4. Create a new virtualenv with pipenv --three install.
5. dcos-cli 0.4.14 is installed. This is not expected.

[techalchemy]

Right but I still need you to include your pipfile.lock to help troubleshoot. Can you also provide the output of pipenv lock —verbose after you have seen the wrong version info

[jeschkies]

Here is the output of pipenv lock --verbose and the generated Pipfile.lock

[jeschkies]

The Pipfile.lock includes

"dcos": {
    "hashes": [
                "sha256:3d48d0773b5dd5e82ddb9cfa946f17755789e8c2ff53ecc75b32194b41248611"
            ],
    "version": "==0.4.14"
}

but the pipenv run dcos --version yields dcoscli.version=0.5.5.

[vphilippon]

Allright, this verbose output given is coherent and match the content of the Pipfile.lock.
But, the reported installed version (from dcos --version) doesn't match.

I tried the steps and could not reproduce. I get:

"dcos": {
    "hashes": [
        "sha256:01d722e13296bd38c2b391ffbd5bc111fa7b5ec09bb058651713bce62a2d45a7"
    ],
    "version": "==0.5.5"
},
"dcos-shakedown": {
    "editable": true,
    "git": "https://github.com/jeschkies/shakedown.git",
    "ref": "012841a38a59d00043c15a66400501811715ab86"
},
"dcoscli": {
    "hashes": [
        "sha256:47b442a2823ab3b27cd520f5c3762ab0e630eca473a273e111619bd5c1a16ea6"
    ],
    "version": "==0.5.5"
},

and, from pipenv run dcos --version

dcoscli.version=0.5.5
dcos.version=N/A
dcos.commit=N/A
dcos.bootstrap-id=N/A

both on the first install, and on the re-install from the Pipfile.lock.

@jeschkies In the commands from the gist, I've seen no pipenv lock --clear (note the --clear). Did you run it as I suggested before? If not, please do, as it could still be an issue with the dependency cache. (And there's no harm in redoing it to be a 100% sure.)

Otherwise, if we're 100% sure the dependency cache was cleared, please give the output of pipenv run pip freeze.

[jeschkies]

@vphilippon are you using pyenv? I'm suspecting this is the issue here. I cannot get it to lock dcos-cli 0.5.5 with the following:

• mkdir pipenv-test && cd pipenv-test
• pyenv local 3.5.2
• pipenv --three install -e git+https://github.com/jeschkies/shakedown.git#egg=dcos-shakedown

The Pipfile.lock will have the 0.4.14. What am I missing here? Is pipenv picking up the from Python version or the wrong dcos-cli package?

[vphilippon]

@jeschkies No, I'm not using pyenv. That was the next thing I was going to check, after making sure it was not a dependency cache issue. I also wanted to have a look at the full list of installed package version, which is why I asked for the output of pipenv run pip freeze.

We'll need someone to try using pyenv and try to reproduce, although if all of your python environment are on Py 3+ now, that should not be an issue causing Py 2 dependencies to be computed.

[techalchemy]

I don't believe pipenv respects the local pyenv setting, which will cause problems for you potentially. When I attempted to just run pipenv install... my system attempted to fall back to python 2.7.

Running the command as pipenv --python=3.5.2 install -e git+https://github.com/jeschkies/shakedown.git#egg=dcos-shakedown worked just fine (and is the recommended way to specify python versions to pipenv instead of hoping it picks up on a local file):

        "dcos": {
            "hashes": [
                "sha256:01d722e13296bd38c2b391ffbd5bc111fa7b5ec09bb058651713bce62a2d45a7"
            ],
            "version": "==0.5.5"
        },
        "dcos-shakedown": {
            "editable": true,
            "git": "https://github.com/jeschkies/shakedown.git",
            "ref": "012841a38a59d00043c15a66400501811715ab86"
        },
        "dcoscli": {
            "hashes": [
                "sha256:47b442a2823ab3b27cd520f5c3762ab0e630eca473a273e111619bd5c1a16ea6"
            ],
            "version": "==0.5.5"
        },

[techalchemy]

FYI this is an issue related to how pipenv resolves pyenv python paths, since it doesn't know about pyenv local settings it will use its own guess at which python interpreter to use in the virtualenv as compared with for dependency resolution. These will wind up being different because one will be resolved by the pyenv shims and one will not.

[erinxocon]

@techalchemy is this a bug in pipenv or pyenv or both???

[jeschkies]

Here is the output of pipenv run pip freeze after I ran pipenv --python=3.5.2 install -e git+https://github.com/jeschkies/shakedown.git#egg=dcos-shakedown. Here is the content of the Pipfile.lock

       "dcos": {
            "hashes": [
                "sha256:3d48d0773b5dd5e82ddb9cfa946f17755789e8c2ff53ecc75b32194b41248611"
            ],
            "version": "==0.4.14"
        },
        "dcos-shakedown": {
            "editable": true,
            "git": "https://github.com/jeschkies/shakedown.git",
            "ref": "012841a38a59d00043c15a66400501811715ab86"
        },
        "dcoscli": {
            "hashes": [
                "sha256:1d7506df5e32fc96566169896fb42ba80aba4687d1c65eb2b52b320b5cfad0ca"
            ],
            "version": "==0.4.14"
        },

This is so strange. Even with pyenv global 3.5.2 it does not work.

[techalchemy]

@jeschkies try pipenv lock —clear per @vphillipon above now that you’ve explicitly told pipenv to use 3.5.2 and see if you wind up with the correct resolutions

@erinxocon it’s only a bug if you think pipenv should respect local pyenv settings in general. What I mean is, if you set pyenv local 3.5.2 and then run pipenv install I think it works correctly. However, the —three and —two helper flags search the path separately for the latest installed version no matter which one you have configured with pyenv. So the question is really whether pipenv should instead respect your pyenv selections if they apply. This feels kind of like an edge case especially when you can just be explicit when you don’t want the latest version.

[jeschkies]

@techalchemy even a pipenv lock --clear did not work. I think it's really just pipenv ignoring my pyenv configuration not matter what. What is pipenv's search path for Python with option --three in macOS?

[erinxocon]

@jeschkies It just looks at your $PATH variable to search through.

@techalchemy I'm torn. Kinda seems like a user environment issue, out of the scope of pipenv. However that was originally our thinking when using pew for virtualenv management, which caused more problems due to shells that aren't configured correctly, so while not a pipenv issue, we changed to avoid problems.

[jeschkies]

@erinxocon hm, how would you have different Python versions without pyenv? Should I just installed them and have all of them on $PATH? That does not seem right. I understand that pipenv should not be too smart about it. However, some docs would probably help here.

[techalchemy]

@jeschkies normal python gets put on the path, and if you install 6 versions yes, all go on your path. Pyenv is a helper function but is definitely not necessary, and pipenv doesn’t read configuration files. It just looks on your path for what ‘python’ points at unless you specify a version or a specific executable. Pyenv shims are a bit messy which I believe is why we don’t use them.

[erinxocon]

--three and --two just look up the path for python3 or python2 in your path. As of right now python2 is only allowed to be named python and python3 is python3. You can have all versions of python in your path, they'll just be addressed using a more specific syntax. python26, python27, python34

[jeschkies]

Hm, ok. So is there another way to manage Python versions that work nicer with pipenv?

[techalchemy]

@jeschkies nicer? I personally use pyenv withPIPENV_DEFAULT_PYTHON_VERSION SET TO 3.6.3. With pyenv installed (or any setup really) you can just pass —python=3.6.3 or —python=2.7.14. That’s the thing with guessing (I.e. with —three and —two)— it’s not explicit. If you want a specific version you will need to specify it

[jamieconnolly]

I have the same issue when trying to install Django==2.0b1. It only offers a Python 3 wheel (the same as dcoscli).

I've installed Pipenv globally through Homebrew, and I use pyenv to manage my Python versions. I've also specified python_full_version = "3.6.3" in my Pipfile (along with the prerelease option).

It fails when I run pipenv install through the global Pipenv, but if I install Pipenv (via Pip) in a Python 3 version, it then succeeds. I'm guessing this means Pipenv isn't using the version specified in Pipfile to do the dependency version look up? I get the same error when using pip install in Python 2.7.10.

I've tried using the PIPENV_DEFAULT_PYTHON_VERSION environment variable, but that didn't work either.

[erinxocon]

Hi @jamieconnolly I think this might have something to do with the shims pyenv puts in place to make multiple environments available. Can't be sure on this as I don't use pyenv.

[techalchemy]

@jamieconnolly I am guessing it is related to the same bug we have touchpoints for in various places relating to, as you mentioned, dependency resolution trickery. One question though -- does pipenv install successfully install the specified version in the former case, but fail on the resolution step and advise you to use --skip-lock? Or does it fail to install in the first place?

[jeschkies]

@jamieconnolly, thanks for the hint. I'll try using a pipenv installation through pip of the local pyenv configuration.

@techalchemy, I should be more specific. It seems that pipenv is not picking up the correct Python version from pyenv when it generates the Pipfile.lock. As you've said

So the question is really whether pipenv should instead respect your pyenv selections if they apply.

Since the answer is not necessarily yes, I'm wondering how I can manage different Python version so that pipenv picks up the correct one when it generates the lock file.

I'll give --python=3.6.3 a try as well.

[techalchemy]

@jeschkies what I meant by that is it doesn't respect whatever you set with pyenv shell or pyenv local or whatever. It should respect the pythons that are on your path, and potentially if pyenv is configured properly it will respect whatever is set with pyenv global since thats how the shim for python will be resolved (if you don't specify a version)

[jeschkies]

Ok, but I'm wondering why the pyenv local Python version is not found when the lock file is generated. It's either the configuration of pyenv or an issue with pipenv. I'm not sure it's the configuration since pipenv has no troubles installing the correct package and the Python version in pipenv shell python --version is correct as well. So why is pipenv lock not picking up the correct version?

[techalchemy]

@jeschkies because if you don't specify a version, pipenv run and pipenv shell will default to system python (which pyenv will resolve via shims if you use pyenv local to whatever you set). Resolution happens using the pip-tools resolver and doesn't ever invoke python directly, so the pyenv shims are essentially bypassed for version resolution and pipenv just looks through your PATH and PYENV_ROOT for all of the available pythons. So that's why pyenv local doens't have any impact on the lockfile.

[jeschkies]

@techalchemy, ah, ok, I think I get a sense of the difference now. So pipenv install and pipenv shell do invoke Python directly and that's why I end with the correct versions there?

[techalchemy]

@jeschkies that's mostly right, although I think pipenv shell will depend on whether you are using the PIPENV_SHELL_FANCY environment variable / are on windows

[cduruk]

I am seeing the same issue with Django==2.0rc1. Really not much to add other than what @jamieconnolly said but here's some data, if you'd like:

Pipfile

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]

[packages]

"psycopg2" = "*"
django = "==2.0rc1"

[pipenv]
allow_prereleases = true

➜  metada git:(master) ✗ pip3 freeze
certifi==2017.11.5
chardet==3.0.4
Django==2.0rc1
flake8==3.5.0
idna==2.6
mccabe==0.6.1
pew==1.1.0
pycodestyle==2.3.1
pyflakes==1.6.0
pytz==2017.3
requests==2.18.4
urllib3==1.22
virtualenv==15.1.0
virtualenv-clone==0.2.6

When I try to lock, even with --pre, I get this:

➜  metada git:(master) ✗ pipenv lock --pre                                                         
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
CRITICAL:pip.index:Could not find a version that satisfies the requirement django==2.0rc1 (from versions: 1.1.3, 1.1.4, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.3, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.4, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.7, 1.4.8, 1.4.9, 1.4.10, 1.4.11, 1.4.12, 1.4.13, 1.4.14, 1.4.15, 1.4.16, 1.4.17, 1.4.18, 1.4.19, 1.4.20, 1.4.21, 1.4.22, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 1.5.7, 1.5.8, 1.5.9, 1.5.10, 1.5.11, 1.5.12, 1.6, 1.6.1, 1.6.2, 1.6.3, 1.6.4, 1.6.5, 1.6.6, 1.6.7, 1.6.8, 1.6.9, 1.6.10, 1.6.11, 1.7, 1.7.1, 1.7.2, 1.7.3, 1.7.4, 1.7.5, 1.7.6, 1.7.7, 1.7.8, 1.7.9, 1.7.10, 1.7.11, 1.8a1, 1.8b1, 1.8b2, 1.8rc1, 1.8, 1.8.1, 1.8.2, 1.8.3, 1.8.4, 1.8.5, 1.8.6, 1.8.7, 1.8.8, 1.8.9, 1.8.10, 1.8.11, 1.8.12, 1.8.13, 1.8.14, 1.8.15, 1.8.16, 1.8.17, 1.8.18, 1.9a1, 1.9b1, 1.9rc1, 1.9rc2, 1.9, 1.9.1, 1.9.2, 1.9.3, 1.9.4, 1.9.5, 1.9.6, 1.9.7, 1.9.8, 1.9.9, 1.9.10, 1.9.11, 1.9.12, 1.9.13, 1.10a1, 1.10b1, 1.10rc1, 1.10, 1.10.1, 1.10.2, 1.10.3, 1.10.4, 1.10.5, 1.10.6, 1.10.7, 1.10.8, 1.11a1, 1.11b1, 1.11rc1, 1.11, 1.11.1, 1.11.2, 1.11.3, 1.11.4, 1.11.5, 1.11.6, 1.11.7)
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.

So when I do that, and run the graph:

➜  metada git:(master) ✗ pipenv graph              
Django==2.0rc1
  - pytz [required: Any, installed: 2017.3]
psycopg2==2.7.3.2

Just can't create a lock file if I specify the version like that.

If I drop the ==2.0rc1 in Pipfile and allow pre-relerases (which I already did), it'll go back to a 1.17 something like that in the lock file, which isn't what I want.

And here's a another output:

➜  metada git:(master) ✗ pipenv run pip freeze
Django==2.0rc1
pytz==2017.3

[cduruk]

Sorry to spam everyone, but I also found a "workaround" my issue. I initially had piping installed via brew but decided to switch to installation via pip, or rather pip3 via pip3 install --user pipenv.

This version of pipenv was able to lock my Django==2.0rc1 properly. I am not sure if #965 is really related to this, but I also saw that issue whilst Googlin' so I thought I'd mention. Basically how one install pipenv seems to make a difference as to whether it picks up the correct python wheels while doing install/lock.

[spool]

I seem to have gotten round a similar issue by removing the homebrew pipenv install and using a standard pip3 install pipenv. I think my error messages relate to the python 2 dependencies that https://github.com/Homebrew/homebrew-core/blob/master/Formula/pipenv.rb require, I expect in assumption of a native python2 install rather than taking advantage of python 3 if available. Will try to send an error message when I have time to reproduce.

[kennethreitz]

this is likely because of setup.py constraints, nothing we can do about it

[kennethreitz]

e.g. install pipenv with python3

[erinxocon]

Homebrew pipenv will soon use python3 as it's default to help avoid these things.