Add build-system.requires to lock file

[adisbladis]

Problem

A notable thing missing from pdm.lock are build-systems used to build from sdists.

This is useful in efforts like pdm2nix that uses PDM metadata.
Nix build environments are constructed in a much stricter way where we cannot dynamically install/load build-systems.
Currently users of such tools have to resort to manually adding overrides into Nix code to work around this missing metadata.

Solution

One more possible entries in pdm.lock:

• build-system.requires

We only need to know build-system.requires & up front when constructing the build environment.
There is no need to add build-system.backend or build-system.backend-path.

The PyPi API does not have this metadata. It would need to be extracted from the sdist.

[phaer]

A similar feature has been considered for pex in pex-tool/pex#2100 and highlights an additional argument for locking w.r.t. to reproducible builds.

[adisbladis]

A similar feature has been considered for pex in pex-tool/pex#2100 and highlights an additional argument for locking w.r.t. to reproducible builds.

That's similar indeed, but with one important distinction:
This feature is only about adding an entry to the lock file which build-systems are to be used when building a particular package.

It does not involve also locking all build system dependencies. While that feature would be great as well I don't want to conflate the two.

[frostming]

Pinning build dependencies is a rather big topic. If we are doing so, build dependencies for all sdists also need to be pinned.

As you said, the PyPi API does not have this metadata. It would need to be extracted from the sdist. Which will tremendously affect the locking performance.

It's not something we can tackle at any near future. It requires some PEPs and/or eco-system support.

[whitequark]

Pinning build dependencies is a rather big topic. If we are doing so, build dependencies for all sdists also need to be pinned.

As you said, the PyPi API does not have this metadata. It would need to be extracted from the sdist. Which will tremendously affect the locking performance.

It's not something we can tackle at any near future. It requires some PEPs and/or eco-system support.

Do you think you could implement a limited form of this where this is just done for build-system.requires? I have a pragmatic reason for this. Without this, https://pypi.org/project/pdm-download/ cannot be used to ensure that a package can be built offline, even if it exclusively uses wheels as dependencies.

[frostming]

Without this, https://pypi.org/project/pdm-download/ cannot be used to ensure that a package can be built offline, even if it exclusively uses wheels as dependencies.

If you are not doing cross-platform deployment you can use https://pypi.org/project/pdm-wheel/

[whitequark]

If you are not doing cross-platform deployment you can use https://pypi.org/project/pdm-wheel/

I tried it but it doesn't include the wheel for pdm-backend which is in my build-system.requires in the wheels/ folder, so I think that doesn't solve it?

[frostming]

I tried it but it doesn't include the wheel for pdm-backend which is in my build-system.requires in the wheels/ folder, so I think that doesn't solve it?

Only the current package is missing, you can add pdm-backend to the dependencies