hatch test --all appears not to work on older x86_64 hardware

[dhdaines]

Hatch version:

$ hatch --version
Hatch, version 1.10.0

Python version:

$ python3 --version
Python 3.10.12

Ran this:

hatch new foo
cd foo
hatch test --all

Hmm! It only runs on Python 3.10, but I thought the default matrix had Python 3.8 through 3.12 in it?

What if I try to set the matrix to 3.11 and 3.12:

[[tool.hatch.envs.hatch-test.matrix]]
python = ["3.12", "3.11"]

Now it tells me:

Skipped 2 incompatible environments:
hatch-test.py3.12 -> cannot locate Python: 3.12
hatch-test.py3.11 -> cannot locate Python: 3.11

Can I try to create those virtual environments? NO, I cannot!

$ hatch -e hatch-test.py3.12 shell
< HORRIFIC STACK TRACE!!! >

Have I failed to grasp something in the documentation? This used to work before hatch test was a thing...
Pardon my language but ... wtf?

[dhdaines]

It seems like this may be related to #1258

I also have an older (but totally functional) CPU. I don't mean to be a jerk but it's really unacceptable that a tool should randomly, incomprehensibly fail when used on older hardware...

[ofek]

Do you know of a way to detect the CPU and correlate that to a specific manufacturing year/line?

For now you can set the environment variable as that issue showed or you could install them on your system somehow outside of using Hatch and ensure that they are on PATH.

[dhdaines]

It's a "v2" CPU, from 2012, a Core i7-860. It has no AVX instructions, which causes lots of problems with things like TensorFlow (another good reason to consign that horrible software to the dustbin of history). But I really like hatch and want to use it.

You can check for the presence of avx in /proc/cpuinfo as a proxy for when v3 builds can be used. Likewise sse3 indicates that you can use v2 builds.

Despite all that I tried to set that environment variable, ran hatch python remove all, reinstalled the python versions, but it still doesn't work:

dhd@dhd-c1:~$ export HATCH_PYTHON_VARIANT_LINUX="v2"
dhd@dhd-c1:~$ hatch new foo
foo
├── src
│   └── foo
│       ├── __about__.py
│       └── __init__.py
├── tests
│   └── __init__.py
├── LICENSE.txt
├── README.md
└── pyproject.toml
dhd@dhd-c1:~$ cd foo
dhd@dhd-c1:~/foo$ hatch test -py 3.12
Skipped 1 incompatible environment:
hatch-test.py3.12 -> cannot locate Python: 3.12
dhd@dhd-c1:~/foo$ hatch python install 3.12
Installed 3.12 @ /home/dhd/.local/share/hatch/pythons/3.12
dhd@dhd-c1:~/foo$ hatch test -py 3.12
Skipped 1 incompatible environment:
hatch-test.py3.12 -> cannot locate Python: 3.12

[dhdaines]

I will try to make a PR...

[dhdaines]

The source of my confusion (the "WTF" above where I couldn't for the life of me figure out why it still wasn't working after setting the environment variable is this):

Hatch has "global" pythons in ~/.local/share/hatch/pythons, but it also has another set of them in ~/.local/share/hatch/env/virtual/.pythons. If you happened to create virtual environments with the wrong "variant" of python for your creaky old CPU at some point in the past, then you must remove everything from ~/.local/share/hatch/env/virtual/.pythons before trying to get the correct version with the HATCH_PYTHON_VARIANT_LINUX environment, or else you suffer eternal confusion and non-working matrix environments.

Not sure if there's a way that Hatch could be more informative when this happens? Perhaps by giving some more detail of why exactly it "couldn't find" a python version that's clearly there but doesn't work on your CPU?

[dhdaines]

Not sure if there's a way that Hatch could be more informative when this happens? Perhaps by giving some more detail of why exactly it "couldn't find" a python version that's clearly there but doesn't work on your CPU?

This is possible to do. The problem is that the logic is mostly wrapped up in virtualenv.detect.builtin and the way hatch currently uses this doesn't allow any exceptions to be detected.

If you can provide a try_first_with argument with a list of hatch's currently installed python binaries when calling get_interpreter here, then you will get an exception if one of those binaries, say, dies with SIGILL.

(edit to add: you do not want all of the installed python binaries, only the one corresponding to the requested version, if it exists)

I am not sure of the best way to do this though.

[dhdaines]

Oh wait, no, it's really easy. Added this to the PR.

[dhdaines]

(in eb024e6)

[ofek]

Fixed by #1484