pip docs for Requirements files -- issues

[gwideman]

Page http://www.pip-installer.org/en/latest/cookbook.html (Title "Cookbook") has some readability issues in the Requirements files section.

1. On that page, the initial section headed 'Requirements Files' starts off with sentences which seem to imply that the topic of requirements files has already been introduced, and this section continues the discussion. So there's probably introductory info missing here. Google returned some 1.0.1 docs , and there's more of an introduction to requirements files on page: http://www.pip-installer.org/en/1.0.1/#requirements-files
2. The sentence "Requirement files are intended to exhaust an environment and to be flat."... doesn't have a meaning, so far as I can tell. What does it mean to "exhaust an environment", or for a file to be "flat" (even italicized "flat"!)?
3. The description of Requirements files format on page http://www.pip-installer.org/en/latest/logic.html (Title "Internal Details") should be linked somewhere in the Cookbook page Requirements section.

[qwcode]

as for #3, the "Requirements File Format" section is linked from the "Requirements Files" cookbook entry at the bottom.

as for #1/#2, I understand both concerns, and I can make another pass on editing to address those.

[qwcode]

how about this version: https://github.com/qwcode/pip/blob/a8468a6974bd45518cf49f5590d0e4929fda515d/docs/cookbook.rst

[gwideman]

Hi Marcus:
Number 3 -- how did I miss that? Sorry.
Number 1, 2. I like the improvement, but still needs more context and specificity, I think.

In order to understand the role(s) of requirements files, we need to know what produces them and what consumes them. (Just pip? Or do other tools read them too?)

I think an introductory statement something like this:

"The 'pip install' command takes as its main input a list of modules or packages [that are required and should be installed if not present? to do other things with?]. The user can provide this list on the 'pip install' command line, but as that is tedious and error-prone, pip can instead accept a requirements file [or files?].

A requirements file is simply a text file that lists the items that are required, along with other constraints such as version range [are there constraints other than version?], each item in the same form as a pip command argument.

There are three main use cases where requirements files are involved:

1. The developer of a package creates a requirements file by hand, simply to make the job of invoking 'pip install' easier and more reliably repeatable.
2. The requirements file may be produced by the 'pip freeze' command [... rest of qwcode's number 2]
3. In a project where there are multiple requirements files for packages that depend upon one another, the developer may consolidate the requirements into a single requirements file. This is useful to work around pip's inability to merge all requirements for the same package that appear in multiple requirements files. In particular, pip doesn't calculate the combined version constraints that would satisfy all requirements. Instead the developer must manually consolidate the version constraints. For example [... rest of qwcode's number 3]

(Subsidiary point: qwcode's example wording for this item describes this as a version 'conflict', which technically it is. However, 'conflict' might suggest that the two requirements specify version ranges that are mutually incompatible, which is a different kettle of fish, and thus a red herring, whereas actually they are overlapping.)

[qwcode]

veteran technical writer, are you? ; )
It's good to have the fresh eyes on this stuff.

developer may consolidate the requirements into a single requirements file [...] multiple requirements files

they are compiling multiple requirement specs into one spec, and then placing that combined spec into the one and only requirements file in this user story. The specs live in the "install_requires" metadata in the "setup.py" specification for each project, not in project requirements files. That's a common confusion. I'll add something about that.

see my latest version.
https://github.com/qwcode/pip/blob/cd40d3e2fd5a5861762db1431730baa20efbc50b/docs/cookbook.rst

the changes:

• be clear on what consumes requirements files (pip install -r)
• "pip install args" --> "items to be installed"
• "resolve dependency conflicts" --> "properly resolve dependencies"
• distinguish requirements files from "install_requires".

[gwideman]

Hi Marcus, Thanks for taking time to look at this, especially since you already have work in progress on this front, not to mention other things to do.

Your revisions are going in a direction which would have helped me on first reading. And they've flushed out a puzzle that I now realize I'd missed before:

'pip freeze' looks like it's an operation that a developer would do during the packaging process, not something that a recipient of a package would do at the install stage? (If true, this contradicts the expectation that pip is for installing.)

I am reluctant to keep pecking away at this detail-by-detail, testing everyone's patience. I think my next step is to try to assemble docs for myself which identify a small number of 'canonical' end-to-end package-deployment-install scenarios which I can then try out, and clearly identify the actual deployable thing(s) in each case, and elaborate how (and with what tools) it's built, how it's shipped, and how it's then installed.

(This comment is pretty parallel to my "needs a coherent big picture" comment on https://bitbucket.org/pypa/python-packaging-user-guide/issue/24/suggestions-for-the-packaging-guide-page.)

I am pretty convinced it all reduces to a fairly simple picture, but at the moment I'm still floundering in the poor signal-to-noise.

[qwcode]

ok, when you're more sorted, please come back and help pip and the user guide get it's docs in order. help appreciated.

as for pip freeze, it's not for packaging (where "packaging" is about packaging one project). It's a tool for installation. Once you feel like you have a set of things installed exactly like you want, you can "freeze" it with pip freeze so you can exactly repeat the installation later in another environment (i.e. another install of python, or another virtualenv enviroment).

[gwideman]

please come back

Will try to do just that.

as for pip freeze, it's not for packaging. It's a tool for installation.[...]

Ah-hah! Thanks for that explanation. So the use case for 'freeze' is that sometimes (often?) the current best deployment practices (setuptools ---> PyPI ---> pip) doesn't succeed in installing a package, or doesn't install it optimally for the user's python arrangement. Evidently, this calls for the user to revise or redo the installation changing something or other. Once that's optimized, then pip freeze can capture that installation configuration so it can be reapplied to either another machine, or to a different python installation (or virtual installation) on the same machine.

Unless I overlooked something, that context of such a motivating use case doesn't appear either in the pip doc, nor in the pip discussions at Python Packaging User Guide.

Anyhow, thanks for clearing that up.

[qwcode]

closing per #1247