You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running a kopf operator with namespace selector AND deleting this namespace that contains a resource with a finalizer: kopf may not call the delete handler method.
Deleting the namespace stops the watch-stream for this namespace before the resources contained in that namespace get the chance have its delete method to be called.
Therefore, the finalizer is never removed and the namespace deletion is stuck waiting all resources to be deleted (but they won't since the watch-stream is stopped).
We manage to find a workaround using the clusterwide setting.
Earlier, we say "may not call"; in some cases the delete handler is called but it is quickly canceled by a CancelledError coming from kopf. We suppose it may be relatee to the event queue size. In the simple example provided, we never manage to get the delete handler to be called.
Adding the finalizer, thus preventing the actual deletion.
Patching with: {'metadata': {'finalizers': ['kopf.zalando.org/KopfFinalizerMarker']}}
Creation is in progress: {'apiVersion': 'kopf.dev/v1', 'kind': 'KopfExample', 'metadata': {'annotations': {'kubectl.kubernetes.io/last-applied-configuration': '{"apiVersion":"kopf.dev/v1","kind":"KopfExample","metadata":{"annotations":{"someannotation":"somevalue"},"labels":{"somelabel":"somevalue"},"name":"kopf-example-1","namespace":"example"},"spec":{"duration":"1m","field":"value","items":["item1","item2"]}}\n', 'someannotation': 'somevalue'}, 'creationTimestamp': '2024-01-10T15:07:37Z', 'finalizers': ['kopf.zalando.org/KopfFinalizerMarker'], 'generation': 1, 'labels': {'somelabel': 'somevalue'}, 'managedFields': [{'apiVersion': 'kopf.dev/v1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:finalizers': {'.': {}, 'v:"kopf.zalando.org/KopfFinalizerMarker"': {}}}}, 'manager': 'kopf', 'operation': 'Update', 'time': '2024-01-10T15:07:37Z'}, {'apiVersion': 'kopf.dev/v1', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:metadata': {'f:annotations': {'.': {}, 'f:kubectl.kubernetes.io/last-applied-configuration': {}, 'f:someannotation': {}}, 'f:labels': {'.': {}, 'f:somelabel': {}}}, 'f:spec': {'.': {}, 'f:duration': {}, 'f:field': {}, 'f:items': {}}}, 'manager': 'kubectl-client-side-apply', 'operation': 'Update', 'time': '2024-01-10T15:07:37Z'}], 'name': 'kopf-example-1', 'namespace': 'example', 'resourceVersion': '334932', 'uid': '829ca9ed-c7a5-4426-baab-7b3762f9b7fb'}, 'spec': {'duration': '1m', 'field': 'value', 'items': ['item1', 'item2']}}
Handler 'create_fn' is invoked.
Handler 'create_fn' succeeded.
Creation is processed: 1 succeeded; 0 failed.
Patching with: {'metadata': {'annotations': {'kopf.zalando.org/last-handled-configuration': '{"spec":{"duration":"1m","field":"value","items":["item1","item2"]},"metadata":{"labels":{"somelabel":"somevalue"},"annotations":{"someannotation":"somevalue"}}}\n'}}}
CREATED!!!
Something has changed, but we are not interested (the essence is the same).
Handling cycle is finished, waiting for new changes.
Stopping the watch-stream for kopfexamples.v1.kopf.dev in 'example'.
Additional information
After some code digging trying to understand what's going on, we think the problem is here and the namespace is removed from the watching list without any check:
Long story short
When running a kopf operator with namespace selector AND deleting this namespace that contains a resource with a finalizer: kopf may not call the delete handler method.
Deleting the namespace stops the watch-stream for this namespace before the resources contained in that namespace get the chance have its delete method to be called.
Therefore, the finalizer is never removed and the namespace deletion is stuck waiting all resources to be deleted (but they won't since the watch-stream is stopped).
We manage to find a workaround using the clusterwide setting.
Earlier, we say "may not call"; in some cases the delete handler is called but it is quickly canceled by a CancelledError coming from kopf. We suppose it may be relatee to the event queue size. In the simple example provided, we never manage to get the delete handler to be called.
Kopf version
1.36.2
Kubernetes version
1.27.4
Python version
3.11.1
Code
Logs
Additional information
After some code digging trying to understand what's going on, we think the problem is here and the namespace is removed from the watching list without any check:
kopf/kopf/_core/reactor/observation.py
Line 202 in 91e8fd6
The text was updated successfully, but these errors were encountered: