Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gh-116303: Skip test module dependent tests if test modules are unavailable #117341

Merged
merged 7 commits into from Apr 3, 2024
Merged

gh-116303: Skip test module dependent tests if test modules are unavailable #117341

merged 7 commits into from Apr 3, 2024

Conversation

erlend-aasland
Copy link
Contributor

@erlend-aasland erlend-aasland commented Mar 28, 2024 •
edited by bedevere-app bot

@bedevere-app bedevere-app bot mentioned this pull request Mar 28, 2024
Closed
@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Mar 28, 2024 •
edited

Some issues regarding C recursion depth still remain1: perhaps we should turn test.support.Py_C_RECURSION_LIMIT into a function2 that raises unittest.SkipTest if _testcapi is missing.

Footnotes

  1. test_ast.test_ast_recursion_limit and test_sys_settrace.test_trace_lots_of_globals ↩

  2. test.support.get_c_recursion_limit()? ↩

erlend-aasland
erlend-aasland commented Mar 28, 2024
View reviewed changes
Lib/test/support/__init__.py Outdated
@@ -1168,6 +1168,16 @@ def requires_limited_api(test):
return unittest.skip('needs _testcapi and _testlimitedcapi modules')(test)
return test


TEST_MODULES = sysconfig.get_config_var('TEST_MODULES')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps name this TEST_MODULES_ENABLED and make it a boolean.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a good idea.
Also, IMO if TEST_MODULES has an unknown value the tests should not be skipped -- they should fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, IMO if TEST_MODULES has an unknown value the tests should not be skipped -- they should fail.

Well, you might as well make the whole test suite fail in that case; if TEST_MODULES has an unknown value, something is more than a little bit off.

@erlend-aasland erlend-aasland marked this pull request as ready for review March 28, 2024 23:57
encukou
encukou reviewed Mar 29, 2024
View reviewed changes
Copy link
Member

@encukou encukou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for combing through these!

Lib/test/test_call.py Outdated
(1, 2), {}, (expected_self, (1, 2), NULL_OR_EMPTY)),
])
@classmethod
def setUpClass(cls):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either do these in the class body, or revert the additions in tearDownClass.
Otherwise, they accumulate in repeated runs (like -R:).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... or just require test modules for this test case; IMO we should not add too much complexity in order to make the test suite pass when test modules are unavailable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it's not too bad to do it in the class body. See 7efd0ce.

Lib/test/support/__init__.py Outdated
@@ -1168,6 +1168,16 @@ def requires_limited_api(test):
return unittest.skip('needs _testcapi and _testlimitedcapi modules')(test)
return test


TEST_MODULES = sysconfig.get_config_var('TEST_MODULES')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a good idea.
Also, IMO if TEST_MODULES has an unknown value the tests should not be skipped -- they should fail.

@erlend-aasland
Copy link
Contributor Author

[...] perhaps we should turn test.support.Py_C_RECURSION_LIMIT into a function2 that raises unittest.SkipTest if _testcapi is missing.

Draft PR:

serhiy-storchaka
serhiy-storchaka approved these changes Apr 3, 2024
View reviewed changes
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@erlend-aasland erlend-aasland added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 3, 2024
@bedevere-bot

This comment was marked as outdated.

Sign in to view
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 3, 2024
@erlend-aasland erlend-aasland merged commit ea94b3b into python:main Apr 3, 2024
113 checks passed
@erlend-aasland erlend-aasland deleted the disable-test-modules/adapt-test-suite branch April 3, 2024 13:11
@erlend-aasland
Copy link
Contributor Author

Thanks for the reviews, everyone.

diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@erlend-aasland @diegorusso
pythongh-116303: Skip test module dependent tests if test modules are…
8b7aa61 
… unavailable (python#117341)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes

@berkerpeksag berkerpeksag Awaiting requested review from berkerpeksag berkerpeksag is a code owner

@brettcannon brettcannon Awaiting requested review from brettcannon brettcannon is a code owner

@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

@ncoghlan ncoghlan Awaiting requested review from ncoghlan ncoghlan is a code owner

@warsaw warsaw Awaiting requested review from warsaw warsaw is a code owner

@iritkatriel iritkatriel Awaiting requested review from iritkatriel iritkatriel is a code owner

@encukou encukou Awaiting requested review from encukou encukou is a code owner

Assignees
No one assigned
Labels
skip news
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@erlend-aasland @bedevere-bot @encukou @serhiy-storchaka