Try to build wheels for directories/VCS

[takluyver]

Follow up from gh-4144

To allow build system abstractions, we want installation to go through wheels in more cases. In particular, installing packages from a local directory or a VCS URL currently runs 'setup.py install'. The aim of this PR is to have it build a wheel, which is stored in an ephemeral cache directory, used for installation, and then discarded.

We can't cache it permanently based on the path/URL, because the code there might change, but our cache wouldn't be invalidated.

[takluyver]

Well, that's weird. On Python 3.3, test_install_curdir_usersite finds a .dist-info folder. On Python 3.6, 3.5, 3.4 and 2.7, it finds .egg-info. 😕

[takluyver]

The remaining test failure is because the tests timed out running on PyPy; I don't think that's related to this PR.

[ghost]

@takluyver After pypa/setuptools#1037 is merged, you should be able to run setup.py egg_info --skip-fetch-build-eggs and then read the setup_requires in .egg-info/setup_requires.txt. This would allow pip to fulfill the installation of setup requirements via wheel and would satisfy the following:

This PEP covers step #2. It is fully expected that a future PEP will cover step #3, including how to have the build system dynamically specify more dependencies that the build system requires to perform its job.

[takluyver]

Thanks @xoviat . The future PEP in question will hopefully be PEP 517, which is currently in draft. So that feature will be useful to implement a setuptools-based build backend.

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will eligible for code review and hopefully merging!

[alex]

My understanding is that this is one of the major blockers for pip 10; is there anything I could to help with it?

[pradyunsg]

@alex The only thing I see being unresolved in this PR is the merge conflicts.

The functionality seems fine (took a passing look at it) and tests are passing (basically).

[alex]

@pradyunsg Cool. I went ahead and submitted #4562 with the conflicts resolved.

[pradyunsg]

@takluyver I'm closing this PR and removing it from the 10.0 milestone since #4562 supersedes this PR.

[nshgraph]

May I ask (naively) what the rationale is for making it build a wheel? We are after all specifying pip install

This just landed in pip 10 and broke our builds in quite a hard-to-debug way. I'm genuinely interested in knowing why the change was made so I can better understand the rationales behind pip and avoid such mistakes on my part in the future!

[Carreau]

Hi Nathan, I've only followed that from far away, and I would suggest falling back on the peps and their discussions for more background. By forcing the source->wheel->install path you allow decoupling of the build and install process. That both simplify maintainer burden and allow to add new tools (like flit), that only need to handle one of these step. It decreases the risk of different behavior when installing from tgz and whl. It also allow pip to really know which files have been installed where, and to properly remove those on uninstall / upgrade. I'm pretty sure the maintainer are sorry to have broken your build, and that filling a bug report could at least help us/them to improve documentation on that regard. It may also allow to discover common behavior, and simplify common patterns later on.

…

On Mon, Apr 16, 2018, 07:04 Nathan Holmberg ***@***.***> wrote: May I ask (naively) what the rationale is for making it build a wheel? We are after all specifying pip install This just landed in pip 10 and broke our builds in quite a hard-to-debug way. I'm genuinely interested in knowing why the change was made so I can better understand the rationales behind pip and avoid such mistakes on my part in the future! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUez2cWjSm9jKLGSVhdNkt-vioYhqO7ks5tpKSagaJpZM4Ngtut> .

[takluyver]

Specifically, PEP 517 defined a standard interface for an install tool (like pip) to ask a build tool (like flit), "please make a wheel from this source tree for me". Unfortunately we didn't get support for that interface into pip 10, but the first step was that installing from source should always build a wheel and install that.

This also fits with @dstufft's goal of reducing the variety of pathways from source to installed code. If it's easy for developers to bypass building a distribution, it's easy for stuff to accidentally be left out of that distribution without the developers noticing there's a problem.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.