Tool `delocate-wheel` alters tag in wheel name to seemingly invalid tag/number #214

andreas-el · 2024-04-08T10:21:15Z

Describe the bug
Tool delocate-wheel alters tag in wheel name to seemingly invalid tag/number
We are using GitHub-runners to generate som large macOS wheels for repository Carolina

To Reproduce

The wheel before we use delocate:

  Created wheel for carolina: filename=carolina-1.0.16.dev3+g42cf9bc-cp38-cp38-macosx_11_0_x86_64.whl size=32980 sha256=f719c488eded81a29c5685e636b6b290f516d8fab9cbfde7b626041e49240904
...

Running delocate-wheel:

delocate-wheel -w /tmp/carolina_dist -v $unfixed_wheel_path
...
...
-rw-r--r--  1 runner  wheel    17M Apr  5 13:42 carolina-1.0.16.dev3+g42cf9bc-cp38-cp38-macosx_13_6_x86_64.whl

I see that there are dependencies that would indicate this to be set to 13_x, but as far as I can tell the wheel won't work when 13_6 is used. If I rename the file I can install it locally. The same is true for 11_0.

Compatible tags: 1835
  cp38-cp38-macosx_13_0_x86_64
  cp38-cp38-macosx_13_0_intel
  cp38-cp38-macosx_13_0_fat64
  cp38-cp38-macosx_13_0_fat32
  cp38-cp38-macosx_13_0_universal2
  cp38-cp38-macosx_13_0_universal
  ...

Wheels used/generated

# generated using version 0.10.7 -- works as intended
carolina-1.0.16.dev2+g9c153fd-cp311-cp311-macosx_11_0_x86_64.whl

# generated using version 0.11.0 -- works when renaming to `13_0` (or `11_0`)
carolina-1.0.16.dev3+g42cf9bc-cp311-cp311-macosx_13_6_x86_64.whl

carolina-1.0.16.dev3+g42cf9bc-cp311-cp311-macosx_13_6_x86_64.whl.tar.gz

carolina-1.0.16.dev2+g9c153fd-cp311-cp311-macosx_11_0_x86_64.whl.tar.gz

Platform (please complete the following information):

OS versions macOS 13 and macOS 14
Delocate version: 0.11.0

Additional context
Works with previous release, 0.10.7

The text was updated successfully, but these errors were encountered:

HexDecimal · 2024-04-08T18:39:58Z

Can you try setting the environment variable MACOSX_DEPLOYMENT_TARGET=11.0 (set it to whichever version you're trying to target) before running delocate-wheel?

HexDecimal · 2024-04-08T18:45:14Z

Which dependencies are forcing 13.6? If those are built locally then maybe set MACOSX_DEPLOYMENT_TARGET=11.0 for the entire workflow.

andreas-el · 2024-04-09T07:32:05Z

Can you try setting the environment variable MACOSX_DEPLOYMENT_TARGET=11.0 (set it to whichever version you're trying to target) before running delocate-wheel?

Thanks for the quick replies.

We have already set this variable, but in the case of macos-14-large this does not work when using delocate 0.11.0.

Fixing: /tmp/carolina_dist_unfixed/carolina-1.0.16.dev3+gfb7b791-cp312-cp312-macosx_11_0_x86_64.whl
  File "/Users/runner/hostedtoolcache/Python/3.12.2/x64/bin/delocate-wheel", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/delocate/cmd/delocate_wheel.py", line 110, in main
    copied = delocate_wheel(
             ^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/delocate/delocating.py", line 1004, in delocate_wheel
    out_wheel_fixed = _check_and_update_wheel_name(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/delocate/delocating.py", line 839, in _check_and_update_wheel_name
    raise DelocationError(
delocate.libsana.DelocationError: Library dependencies do not satisfy target MacOS version 11.0:
/private/var/folders/02/qm7jv9vx7_ld2jykcqjpgkgw0000gn/T/tmp_kontn8r/wheel/carolina.dylibs/libboost_serialization.dylib has a minimum target of 14.0
/private/var/folders/02/qm7jv9vx7_ld2jykcqjpgkgw0000gn/T/tmp_kontn8r/wheel/carolina.dylibs/libsoga.dylib has a minimum target of 14.0
/private/var/folders/02/qm7jv9vx7_ld2jykcqjpgkgw0000gn/T/tmp_kontn8r/wheel/carolina.dylibs/libdream.dylib has a minimum target of 14.0
/private/var/folders/02/qm7jv9vx7_ld2jykcqjpgkgw0000gn/T/tmp_kontn8r/wheel/carolina.dylibs/libjega.dylib has a minimum target of 14.0
/private/var/folders/02/qm7jv9vx7_ld2jykcqjpgkgw0000gn/T/tmp_kontn8r/wheel/carolina.dylibs/libteuchoscomm.13.0.dylib has a minimum target of 14.0
/private/var/folders/02/qm7jv9vx7_ld2jykcqjpgkgw0000gn/T/tmp_kontn8r/wheel/carolina.dylibs/libconmin.dylib has a minimum target of 14.0
...

https://github.com/equinor/Carolina/actions/runs/8611588703/job/23599070801?pr=105

This has me a bit stumped though (why does the error indicate it needs target 14.0 ?)

# When exporting MACOSX_DEPLOYMENT_TARGET=11.0 for macos-14-large (x86_64)
/private/var/folders/02/qm7jv9vx7_ld2jykcqjpgkgw0000gn/T/tmp_kontn8r/wheel/carolina.dylibs/libteuchoscomm.13.0.dylib has a minimum target of 14.0

This does not affect macOS-14 arm64-builds where we had to specify 13.0 as target.

# When exporting MACOSX_DEPLOYMENT_TARGET=13.0 for macos-14 (arm64)
INFO:delocate.delocating:Modifying install name in carolina.dylibs/libteuchoscomm.13.0.dylib from @rpath/libteuchosparameterlist.13.dylib to @loader_path/libteuchosparameterlist.13.0.dylib

Using the following setup works fine:

delocate 0.10.7
MACOSX_DEPLOYMENT_TARGET=11.0
macos-14-large

...
INFO:delocate.delocating:Modifying install name in carolina.dylibs/libteuchoscomm.13.0.dylib from @rpath/libteuchoscore.13.dylib to @loader_path/libteuchoscore.13.0.dylib
...
Output new carolina wheel to /tmp/carolina_dist
total 35704
-rw-r--r--  1 runner  wheel    17M Apr  9 07:20 carolina-1.0.16.dev4+g72314b7-cp312-cp312-macosx_11_0_x86_64.whl

Please note that I am using the same build-files when assembling this wheel.
Meaning that both boost and Dakota were pre-built in during another step in the workflow. I have yet to test exporting the MACOSX_DEPLOYMENT_TARGET variable when performing the actual build.

HexDecimal · 2024-04-09T08:44:17Z

I'll clarify that Delocate 0.11.0 has PR #198 applied which is the cause of the "error". For other devs, this was able to reveal compatibility issues unknown before the release of Delocate 0.11.0. Delocate 0.10.7 does not verify MacOS version compatibility, Delocate 0.11.0 does and will error on any detected compatibility issues.

It's possible that the wheels being fixed by Delocate 0.10.7 are not compatible with MacOS 11.5 and are mislabeled (because Delocate 0.10.7 does not verify MacOS versions). It's possible libraries on a macos-14-large runner are not compatible with earlier versions of MacOS.

A libraries minimum MacOS version is determined from that libraries LC_BUILD_VERSION.

I'm not experienced enough to explain why the libraries from macos-14-large have strange minimum versions for its libraries compared to macos-14.

andreas-el · 2024-04-11T13:07:03Z

I'll clarify that Delocate 0.11.0 has PR #198 applied which is the cause of the "error". For other devs, this was able to reveal compatibility issues unknown before the release of Delocate 0.11.0. Delocate 0.10.7 does not verify MacOS version compatibility, Delocate 0.11.0 does and will error on any detected compatibility issues.

It's possible that the wheels being fixed by Delocate 0.10.7 are not compatible with MacOS 11.5 and are mislabeled (because Delocate 0.10.7 does not verify MacOS versions). It's possible libraries on a macos-14-large runner are not compatible with earlier versions of MacOS.

A libraries minimum MacOS version is determined from that libraries LC_BUILD_VERSION.

I'm not experienced enough to explain why the libraries from macos-14-large have strange minimum versions for its libraries compared to macos-14.

I just wanted to stress that the list provided by the tool regarding compatible tags does not include the one it ends up inserting into the filename. I've assumed this is a list of compatible tags for the wheel in question.

Compatible tags: 1835
  cp38-cp38-macosx_13_0_x86_64
  cp38-cp38-macosx_13_0_intel
  cp38-cp38-macosx_13_0_fat64
  cp38-cp38-macosx_13_0_fat32
  cp38-cp38-macosx_13_0_universal2
  cp38-cp38-macosx_13_0_universal
...

HexDecimal · 2024-04-11T16:25:07Z

Which tool/command gives a list of compatible tags?

andreas-el · 2024-04-12T06:45:12Z

I realise i mistook the origin of that output, wrongfully thinking it came from delocate-listdeps, but it origin is actually python -m build --wheel . --outdir /tmp/carolina_dist_unfixed.

Sorry about that.

I still don't understand why the wheels actually work when I rename them.
Most tags I've seen used for wheels are on the format of 12_0, 13_0, 14_0 just indicating the major version.

HexDecimal · 2024-04-12T07:06:17Z

If the wheels fixed by Decloate in macos-14-large output with a later version of MacOS but still work on earlier versions then the libraries provided by that runner have an incorrect target version (LC_BUILD_VERSION is wrong and should be lower). That's my conclusion when I follow the logic at the moment.

The libraries in macos-14-large should have the same targets as macos-14 but don't. I think this because in the CI run you've posted the macos-14 builds manage to pass and this is my only explanation for why.

The delocate package used to construct the usd-core wheel on MacOS was recently updated to 0.11.0. This version introduced new behavior that modified the name of the generated wheel, which caused issues later in our process. This behavior has affected other projects; see: matthew-brett/delocate#214. In the meantime, we pin delocate to the prior 0.10.x version to get things working again. Support for Python 3.6 was dropped in 0.10.3, so we use a "compatible release" version specifier that gets us 0.10.2 for Python 3.6 and the latest 0.10.x version (currently 0.10.7) for Python 3.7+. (Internal change: 2324479)

jschueller · 2024-05-03T18:53:28Z

same issue here, my wheel gets renamed with an unsupported tag: macosx_13_6_x86_64

HexDecimal · 2024-05-03T18:55:46Z

same issue here, my wheel gets renamed with an unsupported tag: macosx_13_6_x86_64

What was the tag you expected? And did you set MACOSX_DEPLOYMENT_TARGET?

jschueller · 2024-05-03T20:12:27Z

I was expecting macosx_13_0_x86_64

Now if I set MACOSX_DEPLOYMENT_TARGET=13.0 I see the reason ; it flags all my python extensions modules as incompatible (but not my c++ library on which they depend):

delocate.libsana.DelocationError: Library dependencies do not satisfy target MacOS version 13.0:
memoryview.cpython-310-darwin.so has a minimum target of 13.6
...
_statistics.cpython-310-darwin.so has a minimum target of 13.6

note that it does not happen on arm64

HexDecimal · 2024-05-03T20:38:03Z

I was expecting macosx_13_0_x86_64

Now if I set MACOSX_DEPLOYMENT_TARGET=13.0 I see the reason ; it flags all my python extensions modules as incompatible (but not my c++ library on which they depend):

Was that Python installation built locally? Perhaps with MACOSX_DEPLOYMENT_TARGET=13.6 in the environment?

jschueller · 2024-05-04T06:32:56Z

no, I'm using Python from homebrew but its not even linked to Python explicitely:

otool -L openturns/memoryview.cpython-38-darwin.so
openturns/memoryview.cpython-38-darwin.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.0.0)

HexDecimal · 2024-05-04T06:54:13Z

How does one affect the MACOSX_DEPLOYMENT_TARGET or MacOS.version of a program built by brew? I'm not experienced with it and I don't have the platform to mess around with it.

jschueller · 2024-05-04T06:58:59Z

It turns out it was cmake that decided set the min osx version, I worked around it with:

-DCMAKE_OSX_DEPLOYMENT_TARGET=13.0

https://cmake.org/cmake/help/latest/variable/CMAKE_OSX_DEPLOYMENT_TARGET.html

andreas-el added the bug label Apr 8, 2024

andreas-el mentioned this issue Apr 8, 2024

Limit macOS wheels to 3.8, 3.11 & 3.12 and use macOS-14 equinor/Carolina#104

Merged

cqc-alec mentioned this issue Apr 15, 2024

MacOS arm64 wheels incompatible with earlier MacOS versions CQCL/tket#1341

Closed

agriyakhetarpal mentioned this issue May 15, 2024

Use wider-compatible LLVM-OpenMP, bundle libgfortran dylibs for macOS wheels pybamm-team/PyBaMM#4092

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tool `delocate-wheel` alters tag in wheel name to seemingly invalid tag/number #214

Tool `delocate-wheel` alters tag in wheel name to seemingly invalid tag/number #214

andreas-el commented Apr 8, 2024 •

edited

HexDecimal commented Apr 8, 2024

HexDecimal commented Apr 8, 2024

andreas-el commented Apr 9, 2024

HexDecimal commented Apr 9, 2024

andreas-el commented Apr 11, 2024

HexDecimal commented Apr 11, 2024

andreas-el commented Apr 12, 2024

HexDecimal commented Apr 12, 2024

jschueller commented May 3, 2024

HexDecimal commented May 3, 2024

jschueller commented May 3, 2024 •

edited

HexDecimal commented May 3, 2024

jschueller commented May 4, 2024 •

edited

HexDecimal commented May 4, 2024

jschueller commented May 4, 2024

Tool delocate-wheel alters tag in wheel name to seemingly invalid tag/number #214

Tool delocate-wheel alters tag in wheel name to seemingly invalid tag/number #214

Comments

andreas-el commented Apr 8, 2024 • edited

HexDecimal commented Apr 8, 2024

HexDecimal commented Apr 8, 2024

andreas-el commented Apr 9, 2024

HexDecimal commented Apr 9, 2024

andreas-el commented Apr 11, 2024

HexDecimal commented Apr 11, 2024

andreas-el commented Apr 12, 2024

HexDecimal commented Apr 12, 2024

jschueller commented May 3, 2024

HexDecimal commented May 3, 2024

jschueller commented May 3, 2024 • edited

HexDecimal commented May 3, 2024

jschueller commented May 4, 2024 • edited

HexDecimal commented May 4, 2024

jschueller commented May 4, 2024

Tool `delocate-wheel` alters tag in wheel name to seemingly invalid tag/number #214

Tool `delocate-wheel` alters tag in wheel name to seemingly invalid tag/number #214

andreas-el commented Apr 8, 2024 •

edited

jschueller commented May 3, 2024 •

edited

jschueller commented May 4, 2024 •

edited