Installation issue: openmm python wrapper

[zzzoom]

Steps to reproduce the issue

$ spack spec -I openmm %gcc@13.2.0 ~cuda
Input spec
--------------------------------
 -   openmm%gcc@13.2.0~cuda

Concretized
--------------------------------
 -   openmm@8.1.1%gcc@13.2.0~cuda~ipo build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-zen4
[e]      ^cmake@3.28.3%gcc@13.2.0~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-ubuntu24.04-zen4
[+]      ^doxygen@1.9.8%gcc@13.2.0~graphviz~ipo~mscgen build_system=cmake build_type=Release generator=make arch=linux-ubuntu24.04-zen4
[e]          ^bison@3.8.2%gcc@13.2.0~color build_system=autotools arch=linux-ubuntu24.04-zen4
[e]          ^cmake@3.28.3%gcc@13.2.0~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-ubuntu24.04-zen4
[e]          ^flex@2.6.4%gcc@13.2.0+lex~nls build_system=autotools patches=f8b85a0 arch=linux-ubuntu24.04-zen4
[e]          ^gmake@4.3%gcc@13.2.0~guile build_system=generic patches=599f134 arch=linux-ubuntu24.04-zen4
[+]          ^python@3.11.7%gcc@13.2.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=13fa8bf,b0615b2,ebdca64,f2fd060 arch=linux-ubuntu24.04-zen4
[e]              ^gettext@0.21%gcc@13.2.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-ubuntu24.04-zen4
[e]              ^openssl@3.0.13%gcc@13.2.0~docs+shared build_system=generic certs=mozilla arch=linux-ubuntu24.04-zen4
[+]      ^gcc-runtime@13.2.0%gcc@13.2.0 build_system=generic arch=linux-ubuntu24.04-zen4
[e]      ^glibc@2.39%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[e]      ^gmake@4.3%gcc@13.2.0~guile build_system=generic patches=599f134 arch=linux-ubuntu24.04-zen4
[+]      ^py-cython@3.0.8%gcc@13.2.0 build_system=python_pip arch=linux-ubuntu24.04-zen4
[+]          ^py-pip@23.1.2%gcc@13.2.0 build_system=generic arch=linux-ubuntu24.04-zen4
[+]          ^py-setuptools@69.2.0%gcc@13.2.0 build_system=generic arch=linux-ubuntu24.04-zen4
[+]          ^py-wheel@0.41.2%gcc@13.2.0 build_system=generic arch=linux-ubuntu24.04-zen4
[+]      ^py-numpy@1.26.4%gcc@13.2.0 build_system=python_pip patches=873745d arch=linux-ubuntu24.04-zen4
[+]          ^openblas@0.3.26%gcc@13.2.0~bignuma~consistent_fpcsr+dynamic_dispatch+fortran~ilp64+locking+pic+shared build_system=makefile symbol_suffix=none threads=none arch=linux-ubuntu24.04-zen4
[e]              ^perl@5.38.2%gcc@13.2.0~cpanm+opcode+open+shared+threads build_system=generic arch=linux-ubuntu24.04-zen4
[+]          ^py-meson-python@0.15.0%gcc@13.2.0 build_system=python_pip arch=linux-ubuntu24.04-zen4
[e]              ^meson@1.3.2%gcc@13.2.0 build_system=python_pip patches=0f0b1bd arch=linux-ubuntu24.04-zen4
[+]              ^py-pyproject-metadata@0.7.1%gcc@13.2.0 build_system=python_pip arch=linux-ubuntu24.04-zen4
[+]                  ^py-packaging@23.1%gcc@13.2.0 build_system=python_pip arch=linux-ubuntu24.04-zen4
[+]                      ^py-flit-core@3.9.0%gcc@13.2.0 build_system=python_pip arch=linux-ubuntu24.04-zen4
[+]      ^python@3.11.7%gcc@13.2.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=13fa8bf,b0615b2,ebdca64,f2fd060 arch=linux-ubuntu24.04-zen4
[+]          ^bzip2@1.0.8%gcc@13.2.0~debug~pic+shared build_system=generic arch=linux-ubuntu24.04-zen4
[e]              ^diffutils@3.10%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^expat@2.6.2%gcc@13.2.0+libbsd build_system=autotools arch=linux-ubuntu24.04-zen4
[+]              ^libbsd@0.12.1%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[+]                  ^libmd@1.0.4%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^gdbm@1.23%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[e]          ^gettext@0.21%gcc@13.2.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^libffi@3.4.6%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^libxcrypt@4.4.35%gcc@13.2.0~obsolete_api build_system=autotools patches=4885da3 arch=linux-ubuntu24.04-zen4
[+]          ^ncurses@6.5%gcc@13.2.0~symlinks+termlib abi=none build_system=autotools patches=7a351bc arch=linux-ubuntu24.04-zen4
[e]          ^openssl@3.0.13%gcc@13.2.0~docs+shared build_system=generic certs=mozilla arch=linux-ubuntu24.04-zen4
[e]          ^pkgconf@1.8.1%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^readline@8.2%gcc@13.2.0 build_system=autotools patches=bbf97f1 arch=linux-ubuntu24.04-zen4
[+]          ^sqlite@3.43.2%gcc@13.2.0+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^util-linux-uuid@2.38.1%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^xz@5.4.6%gcc@13.2.0~pic build_system=autotools libs=shared,static arch=linux-ubuntu24.04-zen4
[+]          ^zlib-ng@2.1.6%gcc@13.2.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-ubuntu24.04-zen4
[+]      ^python-venv@1.0%gcc@13.2.0 build_system=generic arch=linux-ubuntu24.04-zen4
[+]      ^swig@4.1.1%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4
[+]          ^pcre2@10.43%gcc@13.2.0~jit+multibyte build_system=autotools arch=linux-ubuntu24.04-zen4
[e]          ^pkgconf@1.8.1%gcc@13.2.0 build_system=autotools arch=linux-ubuntu24.04-zen4

Error message

Spack sets openmm's PYTHONPATH properly, but its python wrapper egg is extracted to a subdirectory inside site-packages so importing the module fails.

Error message

$ spack load openmm@8.1.1 ~cuda
$ spack load python@3.11.7
$ echo $PYTHONPATH | tr ':' '\n'
/home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/openmm-8.1.1-kmn2rzqdzp6gspku2f46nkw6q4w4t7sb/lib/python3.11/site-packages
/home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/py-numpy-1.26.4-tpsazw4xuxnboj6ssj5ddcy3bdxdh6cb/lib/python3.11/site-packages
/home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/python-venv-1.0-3ppprpw2bw7evapokawatal2w7m6cmlu/lib/python3.11/site-packages
$ python `spack location -i openmm@8.1.1 %gcc@13.2.0 ~cuda`/examples/benchmark.py
Traceback (most recent call last):
  File "/home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/openmm-8.1.1-kmn2rzqdzp6gspku2f46nkw6q4w4t7sb/examples/benchmark.py", line 2, in 
    import openmm.app as app
ModuleNotFoundError: No module named 'openmm'
$ ls -l /home/bc/repo/spack/opt/spack/linux-ubuntu24.04-zen4/gcc-13.2.0/openmm-8.1.1-kmn2rzqdzp6gspku2f46nkw6q4w4t7sb/lib/python3.11/site-packages
total 4
drwxr-sr-x 5 bc bc 4096 May 10 03:32 OpenMM-8.1.1-py3.11-linux-x86_64.egg
$

Information on your system

• Spack: 0.22.0.dev0 (636d479)
• Python: 3.12.3
• Platform: linux-ubuntu24.04-zen4
• Concretizer: clingo

Additional information

spack-build-out.txt
spack-build-env.txt

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
• I have uploaded the build log and environment files
• I have searched the issues of this repo and believe this is not a duplicate

[zzzoom]

I'd give this a shot but someone familiar with python setuptools shenanigans should figure it out much faster than me.

[haampie]

Looks like a regression between 8.0.0 and 8.1.1.

Maybe easiest to find the issue with spack develop / git bisect

[zzzoom]

They switched from distutils to setuptools in wrappers/python/setup.py but I don't know how python setup.py install --prefix={self.prefix} results in an egg being created and then unpacked.

-from distutils.core import setup
+from setuptools import setup

[haampie]

Yeah, also bisected it to 76be194cbd09c7f595624984ff7f38c73451377a

[haampie]

openmm/openmm#4538