Performance Issue: Too many hashes in RequirementPreparer

[axbycc-mark]

Description

RequirementPreparer hashes more than necessary. This leads to poor performance on large wheels in the GB size range.

A call to prepare_linked_requirement calls down to _checked_download_dir.

pip/src/pip/_internal/operations/prepare.py

Line 501 in f5e4ee1

file_path = _check_download_dir(

If the file exists in the download_dir, hashing is triggered. The file is marked as downloaded.

pip/src/pip/_internal/operations/prepare.py

Line 516 in f5e4ee1

self._downloaded[req.link.url] = file_path

Then we head into _prepare_linked_requirement and eventually hash again.
https://github.com/pypa/pip/blob/main/src/pip/_internal/operations/prepare.py#L612

Potential Fix

Files which have passed the hash check can be marked as such, to prevent rehashing.

Expected behavior

RequirementPreparer hashes each file at most once.

pip version

24.0

Python version

3.11

OS

Windows 10

How to Reproduce

Construct a RequirementPreparer supplied with a download_dir. Run prepare_linked_requirement() for a link available as a wheel in the download_dir.

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.