Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix refcounting of default arguments with CYTHON_AVOID_BORROWED_REFS #6130

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

da-woods
Copy link
Contributor

@da-woods da-woods commented Apr 6, 2024

This affects PyPy and Limited API I think.

References to the function default arguments were being incremented but never decremeneted in the case where the argument was actually provided. I suspect this mostly didn't matter, since arguments are usually just globals with an extended lifetime, but it's worth getting right.

For the more common case where we aren't avoiding borrowed refs, the ref counting becomes a no-op so it doesn't matter.

Needs backporting to 3.0.x (so I'd rather use % formatting than update to fstrings)

This affects PyPy and Limited API I think.

References to the function default arguments were being incremented
but never decremeneted in the case where the argument was actually
provided. I suspect this mostly didn't matter, since arguments
are usually just globals with an extended lifetime, but it's
worth getting right.
@scoder
Copy link
Contributor

scoder commented Apr 8, 2024

I think we originally used borrowed refs in values[] and did all increfs afterwards when assigning to the function argument variables. That invariant must have been lost when we added the vectorcall calling convention for 3.0.

So, the new invariant is that values[] is always reference counted? Is that required for default values? It's probably safer now that we store tuples, right? PyPy could otherwise be unhappy, as could any Python implementation that optimises tuples internally.

@da-woods
Copy link
Contributor Author

da-woods commented Apr 8, 2024

I think we originally used borrowed refs in values[] and did all increfs afterwards when assigning to the function argument variables. That invariant must have been lost when we added the vectorcall calling convention for 3.0.

This was actually added in #5550 to do with Limited API and CYTHON_AVOID_BORROWED_REFS

So, the new invariant is that values[] is always reference counted? Is that required for default values? It's probably safer now that we store tuples, right? PyPy could otherwise be unhappy, as could any Python implementation that optimises tuples internally.

The new invariant is: we always generate code to reference count values[]. For most builds that code is preprocessed to nothing. So for the CYTHON_COMPILING_IN_CPYTHON they remain borrowed. When CYTHON_AVOID_BORROWED_REFS is set then genuine reference counting happens. I think that probably is better for things like PyPy because the contents of values[] can come from items inside tuples and dicts with the lifetime of the references lasting the whole function call.

@da-woods
Copy link
Contributor Author

da-woods commented Apr 9, 2024

The other option is: we could undo this and just use borrowed references universally here (since it seemed to be working before).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants