Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing event which comes during processing of the previous event and 2 update handlers #1072

Open
crypo opened this issue Oct 27, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@crypo
Copy link

crypo commented Oct 27, 2023

Long story short

An event gets missed in the following scenario:

There are two fields in CRD: fieldA and fieldB
There are two on-update handlers (handler1 and handler2): the first one has no filter, the second one has a filter by fieldB
The first handler takes 1 minute to execute
Create resource.
Submit update for fieldB , then in a few seconds submit a separate update for fieldA

Expected behavior:
t+0: handler1 has started (for fieldB)
t+60: handler1 has finished (for fieldB)
t+61: handler2 has started (for fieldB)
t+62: handler2 has finished (for fieldB)
t+63: handler1 has started (for fieldA)
t+64: handler1 has finished (for fieldA)

Actual behavior:
t+0: handler1 has started (for fieldB)
t+60: handler1 has finished (for fieldB)
t+61: handler2 has started (for fieldB and fieldA <- this may be ok)
t+62: handler2 has finished (for fieldB and fieldA <- this may be ok)
handler1 for fieldA is not starting (unexpected)

Kopf version

1.36.2

Kubernetes version

1.27

Python version

3.11

Code

@kopf.on.update()
async def on_a_b_update()
    await asyncio.sleep(60)

@kopf.on.field(field="spec.fieldB")
async def on_b_update()
    logger.info("ok")

Logs

No response

Additional information

What is interesting, if we submit update for fieldA first and then for fieldB - then everything is working.
The order of the handlers in the code and the order of events submitted matters for reproducing.
The test case is reproducible in 100% of the time.

Potential Root Cause

The first event (for fieldB) should be consumed by 2 handlers. By the time the second handler fires (which has a filter for fieldB), the event for fieldA has arrived - now they are both consumed by second handler (which may be ok) but this is causing the first handler not to fire for fieldA (maybe because the first handler has the order before the second one)

@crypo crypo added the bug Something isn't working label Oct 27, 2023
@crypo crypo changed the title Missing event which comes during processing of the previous event and two event handlers Missing event which comes during processing of the previous event and 2 boolean fields Oct 27, 2023
@crypo crypo changed the title Missing event which comes during processing of the previous event and 2 boolean fields Missing event which comes during processing of the previous event and 2 update handlers Oct 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant