Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: support Python 3.13.0b1 (PEP 667 fix) #5127

Merged
merged 5 commits into from
May 24, 2024

Conversation

henryiii
Copy link
Collaborator

@henryiii henryiii commented May 10, 2024

Description

Since beta 1 is out, time to start testing all the time on Python 3.13, not just PR’s with a dev label.

Suggested changelog entry:

* Run CI on Python 3.13

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
@henryiii
Copy link
Collaborator Author

henryiii commented May 23, 2024

Some destructors are not being called when they used to be. The following three tests break:

FAILED ../../tests/test_factory_constructors.py::test_init_factory_alias - assert [3, 3] == [0, 0]
FAILED ../../tests/test_factory_constructors.py::test_init_factory_dual - assert [9, 6] == [7, 4]
FAILED ../../tests/test_virtual_functions.py::test_override - assert 2 == 0

In these, .alive() is called, which runs the gc, then the number of times .destroyed() is called is less on 3.13.0b1 than 3.13.0a6.

Running git bisect between v3.13.0a6 and v3.13.0b1 with the follow script:

#/usr/bin/env bash
set -euo pipefail

./configure --prefix $PWD/../python
make -j8
make install -j8

cd ..
uv pip install --python=./Python/bin/python3 pytest
cmake -S. -Bbuild -DPython_ROOT_DIR=Python -DPYBIND11_TEST_OVERRIDE='test_factory_constructors.py;test_virtual_functions.py'
cmake --build build --target pytest

Showed this broke with python/cpython#115153, PEP 667.

These are now replaced with new functions:

$ git grep -E 'PyEval_GetLocals|PyEval_GetGlobals|PyEval_GetBuiltins|\.f_locals'
include/pybind11/detail/internals.h:446:    state_dict = reinterpret_borrow<object>(PyEval_GetBuiltins());
include/pybind11/pybind11.h:1347:    PyObject *p = PyEval_GetGlobals();
include/pybind11/pybind11.h:2773:            PyObject *locals = PyEval_GetLocals();
include/pybind11/pybind11.h:2820:                       "        self_caller = frame.f_locals[frame.f_code.co_varnames[0]]\n"
tests/constructor_stats.h:116:        PyObject *globals = PyEval_GetGlobals();

These are now no-ops:

$ git grep -E 'PyFrame_FastToLocalsWithError|PyFrame_FastToLocals|git grep PyFrame_LocalsToFast'
include/pybind11/pybind11.h:2797:        PyFrame_FastToLocals(frame);

If I update to use the new functions (only a couple of instances, the rest are for PyPy), then nothing changes (Edit: never mind, messed up the version number, forgot a 0).

@henryiii henryiii force-pushed the henryiii/ci/py313 branch 2 times, most recently from 0a457ea to b5a125a Compare May 23, 2024 21:59
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
@henryiii henryiii changed the title ci: add Python 3.13 fix: support Python 3.13.0b1 (PEP 667 fix) May 23, 2024
@henryiii
Copy link
Collaborator Author

FYI, four tests fail (related to refcounts) when using free-threaded 3.13 but holding the GIL.

@henryiii henryiii merged commit 4d0fced into pybind:master May 24, 2024
82 checks passed
@henryiii henryiii deleted the henryiii/ci/py313 branch May 24, 2024 15:00
@github-actions github-actions bot added the needs changelog Possibly needs a changelog entry label May 24, 2024
PyObject *locals = PyEval_GetLocals();
Py_INCREF(locals);
Copy link
Collaborator

@rwgk rwgk May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@henryiii This Py_INCREF(locals) appears before the locals != nullptr test, therefore I think it could segfault in object.h: op->ob_refcnt++;. Did you consider that already? Did you mean to use Py_XINCREF(locals) here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I did, thanks! I think it’s very unlikely that this will be null, but best to be correct!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs changelog Possibly needs a changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants