Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-111997: C-API for signalling monitoring events #116413

Merged
merged 102 commits into from May 4, 2024
Merged
Show file tree
Hide file tree
Changes from 91 commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
b2d19c3
gh-111997: C-API for signalling monitoring events
iritkatriel Mar 6, 2024
1b9487c
remove unnecessary events and add a basic test
iritkatriel Mar 6, 2024
150011a
enhance test
iritkatriel Mar 6, 2024
ebc36b5
enable disable tests. api test functions return the new 'active' value
iritkatriel Mar 7, 2024
1ad872c
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Mar 7, 2024
34179c1
update PCbuild
iritkatriel Mar 7, 2024
3c9f8bb
use int
iritkatriel Mar 7, 2024
344fd26
implement _PyMonitoringScopeBegin and add CodeLike type
iritkatriel Mar 11, 2024
ecaee5e
more tests
iritkatriel Mar 11, 2024
5c68095
make functions public
iritkatriel Mar 12, 2024
7661865
move tests from testinternalcapi to testcapi
iritkatriel Mar 13, 2024
580a9f3
add include
iritkatriel Mar 13, 2024
19a1f90
PyMonitoringScopeBegin --> PyMonitoring_BeginScope
iritkatriel Mar 13, 2024
745a2b3
IF_ACTIVE --> _PyMonitoring_IF_ACTIVE
iritkatriel Mar 13, 2024
ba5e7e0
upate tests
iritkatriel Mar 13, 2024
aa19675
offset is int, event_types is const, remove "opaque"
iritkatriel Mar 13, 2024
68aa352
PyMonitoring_FireLineEvent takes lineno as int
iritkatriel Mar 13, 2024
db4e3b8
add news
iritkatriel Mar 13, 2024
99722f8
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Mar 13, 2024
445b19f
remove debug stuff
iritkatriel Mar 13, 2024
33ce7aa
not static
iritkatriel Mar 13, 2024
cf6e20d
Revert "not static"
iritkatriel Mar 13, 2024
89b05f1
ignore PyCodeLike_Type
iritkatriel Mar 14, 2024
5f72725
typo
iritkatriel Mar 14, 2024
6368b42
space to tab
iritkatriel Mar 14, 2024
e6d7c6a
skip test if no _testcapi
iritkatriel Mar 18, 2024
58f2bcf
Code review comments
iritkatriel Mar 19, 2024
db5a286
remove unused
iritkatriel Mar 19, 2024
0f329ae
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Mar 19, 2024
38346e8
add test with two events
iritkatriel Mar 19, 2024
738921d
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Mar 19, 2024
f5c40fc
add _PyMonitoring_EndScope
iritkatriel Mar 19, 2024
95c5275
Begin/End --> Enter/Exit
iritkatriel Mar 19, 2024
427e116
error checking
iritkatriel Mar 19, 2024
3bc47df
review comments
iritkatriel Mar 19, 2024
5f11cda
uint32_t --> int
iritkatriel Mar 19, 2024
e719e38
put ifdef back
iritkatriel Mar 19, 2024
be236b1
add opaque
iritkatriel Mar 20, 2024
8c1eacf
always define the private versions. PyUnstable_ for the inlined
iritkatriel Mar 20, 2024
23f579b
add tests for the Unstable version of the API
iritkatriel Mar 21, 2024
eee637e
tidy up declarations
iritkatriel Mar 21, 2024
24cd7c4
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Mar 21, 2024
29c01e3
private versions are not in limited API
iritkatriel Mar 21, 2024
0df7473
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Mar 25, 2024
ef9a4a1
rename macro
iritkatriel Mar 25, 2024
dea138e
Merge branch 'main' into monitoring-capi
iritkatriel Mar 26, 2024
c0bcc22
update limited API
iritkatriel Mar 26, 2024
48176fc
no need for two versions of EnterScope/ExitScope
iritkatriel Mar 27, 2024
c52dddc
Revert "update limited API"
iritkatriel Mar 28, 2024
69cfe88
remove stable version. Rename Unstable functions
iritkatriel Mar 28, 2024
20ce59f
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Apr 1, 2024
811402a
Delete Modules/_sqlite/connection-dd92fcfb.o.tmp
iritkatriel Apr 1, 2024
86d2f67
Delete Modules/_testcapi/monitoring.c
iritkatriel Apr 1, 2024
e60f83f
Revert "Delete Modules/_testcapi/monitoring.c"
iritkatriel Apr 1, 2024
8c052d3
Delete Modules/_sqlite/cursor-8e377591.o.tmp
iritkatriel Apr 1, 2024
e2f9f2f
Delete Python/pylifecycle-c98da669.o.tmp
iritkatriel Apr 1, 2024
7b02a82
Delete Python/pystate-0cb7e25d.o.tmp
iritkatriel Apr 1, 2024
c605827
Merge branch 'main' into monitoring-capi
iritkatriel Apr 1, 2024
a6d45c4
Merge branch 'main' into monitoring-capi
iritkatriel Apr 2, 2024
e98827c
Merge branch 'main' into monitoring-capi
iritkatriel Apr 2, 2024
5fdce63
remove a few bits
iritkatriel Apr 4, 2024
21537da
Move non-limited API to Include/cpython/ (#56)
encukou Apr 4, 2024
8eac91d
Merge branch 'main' into monitoring-capi
iritkatriel Apr 4, 2024
1956867
Merge branch 'main' into monitoring-capi
iritkatriel Apr 5, 2024
882a205
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Apr 9, 2024
aaea28e
Petr's comments
iritkatriel Apr 10, 2024
2f0a4d1
int -> int32_t for offset
iritkatriel Apr 10, 2024
88e770f
give EnterScope/ExitScope a return value
iritkatriel Apr 10, 2024
2bed2e3
add doc
iritkatriel Apr 10, 2024
573fbf8
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Apr 10, 2024
85e9033
fix typo
iritkatriel Apr 11, 2024
5162a03
update doc conf
iritkatriel Apr 11, 2024
0438389
document PyMonitoringState
iritkatriel Apr 11, 2024
f53ceed
Merge branch 'main' into monitoring-capi
iritkatriel Apr 11, 2024
75e5103
Update Doc/c-api/monitoring.rst
iritkatriel Apr 11, 2024
c7bd7f4
Petr's comments
iritkatriel Apr 11, 2024
2c09788
update docs
iritkatriel Apr 11, 2024
5936200
fix doc
iritkatriel Apr 11, 2024
3f924e4
whitespace
iritkatriel Apr 11, 2024
d42b7c8
fix doc ref
iritkatriel Apr 11, 2024
d3a5dd5
fix ref
iritkatriel Apr 11, 2024
0c4e65d
add title
iritkatriel Apr 11, 2024
a59568b
doc update
iritkatriel Apr 12, 2024
71e6bd3
Merge branch 'main' into monitoring-capi
iritkatriel Apr 12, 2024
1ed891d
Merge remote-tracking branch 'upstream/main' into monitoring-capi
iritkatriel Apr 18, 2024
262977f
update doc per Petr's comments
iritkatriel Apr 18, 2024
7d530b1
reentered
iritkatriel Apr 19, 2024
45a7895
Apply suggestions from code review
iritkatriel Apr 19, 2024
cf9dfd0
whitespace
iritkatriel Apr 19, 2024
03382bc
Merge branch 'main' into monitoring-capi
iritkatriel Apr 22, 2024
3a6e583
add exceptions and disabling to doc
iritkatriel Apr 22, 2024
a704c4a
VM takes care of exception state for exception-related events
iritkatriel Apr 26, 2024
c7de609
bool -> int
iritkatriel Apr 26, 2024
e673b5a
exception events use 'current exception'
iritkatriel Apr 28, 2024
5f00efb
update doc
iritkatriel Apr 28, 2024
69e900e
raise on missing exception
iritkatriel Apr 29, 2024
841eb6c
typo in doc
iritkatriel Apr 29, 2024
2840f12
Petr's comments
iritkatriel Apr 29, 2024
c8499e5
Merge branch 'main' into monitoring-capi
iritkatriel May 1, 2024
aa6aa8a
Merge branch 'main' into monitoring-capi
iritkatriel May 3, 2024
6c65373
Merge branch 'main' into monitoring-capi
iritkatriel May 3, 2024
bf84396
Merge branch 'main' into monitoring-capi
iritkatriel May 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions Doc/c-api/index.rst
Expand Up @@ -25,3 +25,4 @@ document the API functions in detail.
memory.rst
objimpl.rst
apiabiversion.rst
monitoring.rst
160 changes: 160 additions & 0 deletions Doc/c-api/monitoring.rst
@@ -0,0 +1,160 @@
.. highlight:: c

.. _monitoring:

Monitorong C API
================

Added in version 3.13.

An extension may need to interact with the event monitoring system. Subscribing
to events and registering callbacks can be done via the Python API exposed in
:mod:`sys.monitoring`.

Generating Execution Events
===========================

The functions below make it possible for an extension to fire monitoring
events as it emulates the execution of Python code. Each of these functions
accepts a ``PyMonitoringState`` struct which contains concise information
about the activation state of events, as well as the event arguments, which
include a ``PyObject*`` representing the code object, the instruction offset
and sometimes additional, event-specific arguments (see :mod:`sys.monitoring`
for details about the signatures of the different event callbacks).
The ``codelike`` argument should be an instance of :class:`types.CodeType`
or of a type that emulates it.

The VM disables tracing when firing an event, so there is no need for user
code to do that.

It is expected that monitoring functions are not called with an exception set,
except for those which are firing exception-related events.
iritkatriel marked this conversation as resolved.
Show resolved Hide resolved

.. c:type:: PyMonitoringState

Representation of the state of an event type. It is allocated by the user
while its contents are maintained by the monitoring API functions described below.
iritkatriel marked this conversation as resolved.
Show resolved Hide resolved


All of the functions below return 0 on success and -1 (with an exception set) on error.
iritkatriel marked this conversation as resolved.
Show resolved Hide resolved

They should not be called with an exception set; you might want to wrap the
call in :c:func:`PyErr_GetRaisedException` and :c:func:`PyErr_SetRaisedException`.

See :mod:`sys.monitoring` for descriptions of the events.

.. c:function:: int PyMonitoring_FirePyStartEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset)

Fire a ``PY_START`` event.


.. c:function:: int PyMonitoring_FirePyResumeEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset)

Fire a ``PY_RESUME`` event.


.. c:function:: int PyMonitoring_FirePyReturnEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject* retval)

Fire a ``PY_RETURN`` event.


.. c:function:: int PyMonitoring_FirePyYieldEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject* retval)

Fire a ``PY_YIELD`` event.


.. c:function:: int PyMonitoring_FireCallEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject* callable, PyObject *arg0)

Fire a ``CALL`` event.


.. c:function:: int PyMonitoring_FireLineEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, int lineno)

Fire a ``LINE`` event.


.. c:function:: int PyMonitoring_FireJumpEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *target_offset)

Fire a ``JUMP`` event.


.. c:function:: int PyMonitoring_FireBranchEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *target_offset)

Fire a ``BRANCH`` event.


.. c:function:: int PyMonitoring_FireCReturnEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *retval)

Fire a ``C_RETURN`` event.


.. c:function:: int PyMonitoring_FirePyThrowEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *exception)

Fire a ``PY_THROW`` event.


.. c:function:: int PyMonitoring_FireRaiseEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *exception)

Fire a ``RAISE`` event.
iritkatriel marked this conversation as resolved.
Show resolved Hide resolved


.. c:function:: int PyMonitoring_FireCRaiseEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *exception)

Fire a ``C_RAISE`` event.


.. c:function:: int PyMonitoring_FireReraiseEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *exception)

Fire a ``RERAISE`` event.


.. c:function:: int PyMonitoring_FireExceptionHandledEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *exception)

Fire an ``EXCEPTION_HANDLED`` event.


.. c:function:: int PyMonitoring_FirePyUnwindEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *exception)

Fire a ``PY_UNWIND`` event.


.. c:function:: int PyMonitoring_FireStopIterationEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *exception)

Fire a ``STOP_ITERATION`` event.


Managing the Monitoring State
-----------------------------

Monitoring states can be managed with the help of monitoring scopes. A scope
would typically correspond to a python function.

.. :c:function:: int PyMonitoring_EnterScope(PyMonitoringState *state_array, uint64_t *version, const uint8_t *event_types, Py_ssize_t length)

Enter a monitored scope. ``event_types`` is an array of the event IDs for
events that may be fired from the scope. For example, the ID of a ``PY_START``
event is the value ``PY_MONITORING_EVENT_PY_START``, which is numerically equal
to the base-2 logarithm of ``sys.monitoring.events.PY_START``.
``state_array`` is an array with a monitoring state entry for each event in
``event_types``, it is allocated by the user but populated by
``PyMonitoring_EnterScope`` with information about the activation state of
the event. The size of ``event_types`` (and hence also of ``state_array``)
is given in ``length``.

The ``version`` argument is a pointer to a value which should be allocated
by the user together with ``state_array`` and initialized to 0,
and then set only by ``PyMonitoring_EnterScope`` itelf. It allows this
function to determine whether event states have changed since the previous call,
and to return quickly if they have not.
encukou marked this conversation as resolved.
Show resolved Hide resolved
iritkatriel marked this conversation as resolved.
Show resolved Hide resolved

The scopes referred to here are lexical scopes: a function, class or method.
``PyMonitoring_EnterScope`` should be called whenever the lexical scope is
entered. Scopes can be reentered, reusing the same *state_array* and *version*,
in situations like when emulating a recursive Python function. When a code-like's
execution is paused, such as when emulating a generator, the scope needs to
be exited and re-entered.


.. :c:function:: int PyMonitoring_ExitScope(void)

Exit the last scope that was entered with ``PyMonitoring_EnterScope``.
1 change: 1 addition & 0 deletions Doc/conf.py
Expand Up @@ -131,6 +131,7 @@
('c:func', 'vsnprintf'),
# Standard C types
('c:type', 'FILE'),
('c:type', 'int32_t'),
('c:type', 'int64_t'),
('c:type', 'intmax_t'),
('c:type', 'off_t'),
Expand Down
5 changes: 4 additions & 1 deletion Doc/library/sys.monitoring.rst
Expand Up @@ -255,7 +255,10 @@ No events are active by default.
Per code object events
''''''''''''''''''''''

Events can also be controlled on a per code object basis.
Events can also be controlled on a per code object basis. The functions
defined below which accept a :class:`types.CodeType` should be prepared
to accept a look-alike object from functions which are not defined
in Python (see :ref:`monitoring`).

.. function:: get_local_events(tool_id: int, code: CodeType, /) -> int

Expand Down
1 change: 1 addition & 0 deletions Include/Python.h
Expand Up @@ -84,6 +84,7 @@
#include "setobject.h"
#include "methodobject.h"
#include "moduleobject.h"
#include "monitoring.h"
#include "cpython/funcobject.h"
#include "cpython/classobject.h"
#include "fileobject.h"
Expand Down