Python 3.12: Add 3.12.2 #15955
Python 3.12: Add 3.12.2 #15955
Conversation
|
Just first suggestion: please switch to |
Bill-Sommerfeld
force-pushed
the
py312
branch
from
3bd7796
to
dedb6b3
Compare
|
now down to three main loose ends:
and then need to go over all the patches and clean up warts (particularly the 00*.patch hacks). |
Bill-Sommerfeld
force-pushed
the
py312
branch
from
e39900e
to
c41b120
Compare
Congrats! |
|
At this point, the one remaining loose end is Thoughts? |
|
FYI, there is |
|
Thank you for working on python 3.12. I do not have any comments for your recent question. Is there a reason that openssl1.1 needs to be used? Please consider changing "USE_OPENSSL11= yes" to 'OPENSSL_VERSION= 3.1" in the Makefile. |
|
I would do not bother with Thank you! |
Fixed as you suggested. I started with |
Done. |
The equivalent file in the solaris-userland patch has an odd copyright statement in it that grants rights only to the PSF so I'm not touching it. I took a closer look at the data structure differences between 3.9 and 3.12 and it didn't look that bad. I've managed to tweak our version into building; to run it I'll need to build a corresponding |
|
@Bill-Sommerfeld, please do NOT squash yet. The history contains valuable information that might be needed during review. Thank you. |
| CONFIGURE_ENV += ac_cv_func_epoll_ctl=no | ||
| CONFIGURE_ENV += ac_cv_func_epoll_create1=no | ||
|
|
||
| CONFIGURE_ENV += ac_cv_func_getentropy=no |
There was a problem hiding this comment.
This is similar to line 119 below, but it is commented there. The linked bug (see below) is closed as resolved (with fix merged) for many years now. Could you please make sure this one is still needed? If so, then please add some reasonable comment explaining the reason.
Thank you.
There was a problem hiding this comment.
As far as I can tell disabling getentropy is not needed and these lines will be gone in the next push. Truss on a short python script shows that os.urandom() generates a call into the getrandom() syscall passing a zero flag (which is the desirable fast-path, unlikely to block) rather than calling getentropy().
There was a problem hiding this comment.
As far as I can tell disabling getentropy is not needed and these lines will be gone in the next push. Truss on a short python script shows that os.urandom() generates a call into the getrandom() syscall passing a zero flag (which is the desirable fast-path, unlikely to block) rather than calling getentropy().
I assume you trussed with these lines removed... (just to be sure we are on the same page :-))
There was a problem hiding this comment.
The code in Python/bootstrap_hash.c is structured such that it will only build code to use getentropy if getrandom isn't present:
#if defined(HAVE_GETRANDOM) || defined(HAVE_GETRANDOM_SYSCALL)
#define PY_GETRANDOM 1
...
#elif defined(HAVE_GETENTROPY)
#define PY_GETENTROPY 1
...
#endif /* defined(HAVE_GETENTROPY) && !(defined(__sun) && defined(__SVR4)) */
Just to be sure I'll test this again with a build with the lines removed.
There was a problem hiding this comment.
Just to be sure I'll test this again with a build with the lines removed.
I've retested. As expected, removing the ...getentropy=no lines from the Makefile had no effect on the behavior - os.urandom() results in a getrandom() syscall with them either present or removed.;
|
@Bill-Sommerfeld, you probably already noticed, but if not, then: https://github.com/Bill-Sommerfeld/oi-userland/pulls |
Bill-Sommerfeld
force-pushed
the
py312
branch
from
fb73e63
to
42d94af
Compare
Bill-Sommerfeld
changed the title
| @@ -9,7 +9,7 @@ | |||
| "library/libffi", | |||
| "library/ncurses", | |||
| "library/readline", | |||
| "library/security/openssl-31", | |||
| "library/security/openssl-11", | |||
There was a problem hiding this comment.
Something is broken with openssl. :-(
There was a problem hiding this comment.
Ah, you switched back to 1.1. That explains this change.
There was a problem hiding this comment.
Yes. When I get a chance I'll dig into the test failures but until then, working 1.1 is better than broken 3.1...
|
|
||
| Without this patch, python detects and uses epoll which only exists in | ||
| OmniOS for lx zones and Linux compatibility. It is not quite the same as | ||
| the Linux implementation and can cause socket related failures in python. |
There was a problem hiding this comment.
This is something that needs some love:
- we are not OmniOS :-)
- we do not have lx zones
- it is strange that OmniOS claims (this patch was apparently copied from OmniOS) that since epoll is for Linux compatibility then it should not be used. And the reason is that our implementation is not quite the same as on Linux. The epoll support was added to be fully compatible with Linux and to make porting of software from Linux easier. If there is some notable difference, then it is a bug in illumos that should be (at least) reported, and then investigated and eventually fixed.
- it should be checked/tested if this patch is still needed; if it is still needed then it means that something in illumos epoll does not work as expected (see above), or there is some bug in Python itself. Either way we need to report this as a bug.
Thanks.
There was a problem hiding this comment.
The epoll support was not intended to be fully compatible as some behaviours which were considered as buggy in the Linux implementation were not replicated (there was a presentation by Cantrill along these lines), therefore illumos epoll can only be used reliably in a subset of the situations encountered. I cannot remember the details but there are situations where the Linux epoll "works" which should be considered as undefined behaviour, like referencing resources that are long gone. Mimicing Linux's behaviour would rely on border effects. In such cases disabling epoll was the only reasonable way to go. The good news is that usually epoll would just hang on illumos when such edge cases are met.
There was a problem hiding this comment.
For the same reason epoll is disabled in Xorg and other graphical toolkits.
There was a problem hiding this comment.
Okay, understood. But still we need better comment here in this patch. Reference to more info would be great. For exactly the reason you stated: "I cannot remember..."
Also, hang in epoll (is the hang in illumos or in the epoll consumer?) cannot be considered as good news. It sounds like a bug.
There was a problem hiding this comment.
The hang was in the consumer and we were told that the use case was of one of those not covered by the illumos implementation because the Linux implementation only "works" because of a border effect.
Sorry that I cannot remember better, we had a thread a few years ago but a lot has happened since then.
There was a problem hiding this comment.
It looks like there are many problems related to epoll in Python itself.
Examples:
- on Linux: Calling _run_once with an empty event loop hangs forever python/cpython#112741
on Windows (yes, they use epoll even on Windows!): asyncio gets stuck on event loop close python/cpython#111604(EDIT: event loop is implemented differently there)
But I found only one (long time ago solved) issue reported there related to illumos and epoll:
So we definitely need more bug reports :-).
There was a problem hiding this comment.
It looks like the epoll(7) man page contains all needed information, including caveats about Linux epoll implementation.
|
I think it would be better to mark this as draft so it is not merged accidentally :-). |
|
Once #16009 is merged please rebase on top of it and generate new sample-manifest. |
Bill-Sommerfeld
force-pushed
the
py312
branch
from
ab61b89
to
58341bc
Compare
Done. |
…nifests to match
…of ...-solaris2.11
Bill-Sommerfeld
force-pushed
the
py312
branch
from
24fc5bc
to
16a665b
Compare
|
Any news here? |
This is a very very very rough first draft; it makes it all the way throughgmake packageand passes most of the tests.patches-todo has patches I haven't dealt with yet.patches has a bunch of hacks that need to be cleaned up.