Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Fix limited API under mingw #13171

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

amcn
Copy link
Contributor

@amcn amcn commented May 2, 2024

Link to the correct Python limited API DLL under mingw

This aims to fix #13167

@amcn amcn requested a review from jpakkane as a code owner May 2, 2024 12:08
@amcn
Copy link
Contributor Author

amcn commented May 2, 2024

Disappointingly the tests seem to have all passed which makes me think we are missing something which tests precisely whether the right library was linked or not by shelling out to ldd/dumpbin/otool.

I'm going to add that now into unittests/pythontests (where I should have added such a test in the initial limited api submission now that I think about it).

@lpsinger
Copy link

lpsinger commented May 3, 2024

I find that this PR fixes the issue: nasa-gcn/hpx#21

@amcn amcn force-pushed the mingw-python-limited-api branch from fd9c8b0 to 155a3e9 Compare May 8, 2024 17:08
@amcn
Copy link
Contributor Author

amcn commented May 8, 2024

I've reworked this PR to do two things:

  1. Tests have been updated: a new unittest has been added (for windows only), and the existing limited API test has been extended with an extra section to load the built module.
  2. The original fix has been split into two commits: one to move the code that has to be moved, and a second to implement the fix.

Hopefully this is easier to review now. Feedback very welcome.

@amcn amcn force-pushed the mingw-python-limited-api branch from 155a3e9 to 2a7aeb3 Compare May 8, 2024 20:18
Copy link
Member

@eli-schwartz eli-schwartz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous commit made it possible for a PythonPkgConfigDependency to be used in a context where previously only a PythonSystemDependency would be used.

It sounds like maybe commits 3 and 4 should be flipped around maybe?

args: [files('test_limited.py')],
env: { 'PYTHONPATH': meson.current_build_dir() },
workdir: meson.current_source_dir()
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File endings look weird. :P

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oof, I always forget about this. Fix incoming.

Comment on lines 94 to 99
testdir = os.path.join(
self.src_root,
'test cases',
'python',
'9 extmodule limited api'
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a good reason to split this across 6 different lines, we don't do so elsewhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix incoming.

Comment on lines +113 to +114
output = subprocess.check_output(['dumpbin', '/DEPENDENTS', limited_library_path],
stderr=subprocess.STDOUT)
self.assertIn(limited_dep_name, output.decode())
elif shutil.which('objdump'):
# mingw
output = subprocess.check_output(['objdump', '-p', limited_library_path],
stderr=subprocess.STDOUT)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want to look in stderr as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly out of paranoia. When I was implementing this it was handy to have both stderr and stdout so that when it didn't work correctly, the contents of both stdout and stderr ended up in the log. Should this be removed?

@amcn
Copy link
Contributor Author

amcn commented May 27, 2024

The previous commit made it possible for a PythonPkgConfigDependency to be used in a context where previously only a PythonSystemDependency would be used.

It sounds like maybe commits 3 and 4 should be flipped around maybe?

Yeah, flipping them around is clearer. I'll push a new series up now with them flipped (and other issues addressed).

@amcn amcn force-pushed the mingw-python-limited-api branch from 2a7aeb3 to 4ec0e34 Compare May 27, 2024 09:58
amcn added 5 commits May 27, 2024 13:33
Based on the example in GH issue mesonbuild#13167, the limited API test has been
extended with a test to load the compiled module to ensure it can be
loaded correctly.
This new unittest runs the existing Python limited API
test case, and checks that the resulting library was linked to
the correct library.
This is in preparation for a future commit which makes it
possible for a PythonPkgConfigDependency to be used in a
context where previously only a PythonSystemDependency would
be used.
The Python Limited API support that was added in 1.2 had
special handling of Windows, but the condition to check for
Windows was not correct: it checked for MSVC and not for
the target's OS. This causes mingw installations to not have
the special handling applied.

This commit fixes this to check explicitly for Windows.
This commit fixes GH issue mesonbuild#13167 by linking to the correct
library under MINGW when the 'limited_api' kwarg is specified.
@amcn amcn force-pushed the mingw-python-limited-api branch from 9677704 to 06c45b3 Compare May 27, 2024 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants