Include file locations - Windows doesn't have a standard place

[pfmoore]

In "Other Issues", in the section about include files, you say

There are standard places for thison a system

(btw, there's a small typo there). On Windows, with MSVC, there isn't (as far as I am aware) a standard location for include files. Apart from the standard includes supplied by the compiler and/or the Windows SDK, all other headers are located via the environment variables and/or project settings. So installing a 3rd party library like zlib, still requires project configuration to make the build aware of where the zlib headers are. Package managers like vcpkg handle some of this, I believe, but need to be integrated with the build tools.

This doesn't significantly alter your point, but it does mean that the reasons for putting includes in site-packages are not just "due to the limitations that PyPI/wheels impose".

[rgommers]

Thanks for pointing that out @pfmoore. I had a closer look at how this works and what typical advice is. For MSVC, what is searched by default is what's in the PATH and INCLUDE env vars (according to the MSVC docs for /X). What is in PATH is only a few default system locations, like C:\Windows\system32 (may vary).

What I can find for advice is sparse; typically paths are added to a Visual Studio project file, so that they're the same for everyone on a project. That of course does not help at all when you want to use something standard across many projects.

A default system location like C:\Windows\system32 is not a great place to install to. in the absence of other paths, it is the only option that will work unless you add a new agreed-upon standard path - \path\to\env\lib\pythonx.y\site-packages\pkgname\subdir\include\ clearly does not work, as the current content explains. Other options:

• it is already possible to install to a virtual environment include path: https://peps.python.org/pep-0405/#include-files. It is just recommended against (see, e.g., what pybind11-global docs say about it: https://pybind11.readthedocs.io/en/stable/installing.html#include-with-pypi)
• what Conda does is add environments which have env-name\include\, and then add that path to the include paths with an "activation script" which sets CFLAGS, CXXFLAGS, etc. for each installed compiler. (I don't like how it messes with env vars, but it works)
• I can't quite describe what Spack does accurately enough right now, but it does have a way and involves awareness of external compilers I believe (it does not add to CFLAGS et al.)

This doesn't significantly alter your point, but it does mean that the reasons for putting includes in site-packages are not just "due to the limitations that PyPI/wheels impose".

It kind of is though (as the PEP 405 text also acknowledges) - it follows from not having any awareness of compilers, sets of packages that have to work together, having both virtualenvs and "non virtualenv but arbitrary standalone builds of Python", etc. For something like installing header files from one package and having another package find them, you need some sort of coordinated approach, and it's just missing from the design of Python packaging.