Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No running event loop error observed when doing heavy update #7642

Closed
berland opened this issue Apr 11, 2024 · 3 comments
Closed

No running event loop error observed when doing heavy update #7642

berland opened this issue Apr 11, 2024 · 3 comments
Assignees
@JHolba @xjules
Labels
bug

Comments

@berland
Copy link
Contributor

berland commented Apr 11, 2024

Observed while running the poly-case on version 9.0.4 on 1000 realizations, during the update step:

(2024.04.rc0-py38) [havb@be-linrgsn001:~/projects/ert/test-data/poly_example] main$ ert gui --enable-scheduler poly.ert
Exception ignored in: <coroutine object WebSocketCommonProtocol.close_connection at 0x7f2c8c56c340>
Traceback (most recent call last):
  File "/prog/res/komodo/2024.04.rc0-py38-rhel7/root/lib64/python3.8/site-packages/websockets/legacy/protocol.py", line 1337, in close_connection
    await self.close_transport()
  File "/prog/res/komodo/2024.04.rc0-py38-rhel7/root/lib64/python3.8/site-packages/websockets/legacy/protocol.py", line 1355, in close_transport
    if await self.wait_for_connection_lost():
  File "/prog/res/komodo/2024.04.rc0-py38-rhel7/root/lib64/python3.8/site-packages/websockets/legacy/protocol.py", line 1379, in wait_for_connection_lost
    async with asyncio_timeout(self.close_timeout):
  File "/prog/res/komodo/2024.04.rc0-py38-rhel7/root/lib64/python3.8/site-packages/websockets/legacy/async_timeout.py", line 74, in timeout
    loop = asyncio.get_running_loop()
RuntimeError: no running event loop

This is probably the Ensemble Evaluator that is too busy doing the maths for the update from an iteration to the next, and not setting aside time to maintain the websocket connection, and then some bad handling of this problem.

The poly case progresses fine so this is not an Error, just a Warning. But this should not be shown to users.
@xjules xjules added the bug label Apr 15, 2024
@berland
Copy link
Contributor Author

berland commented Apr 15, 2024

Related to #7275

@jonathan-eq jonathan-eq self-assigned this Apr 18, 2024
@jonathan-eq jonathan-eq mentioned this issue Apr 18, 2024
Closed
9 tasks
@berland berland mentioned this issue Apr 18, 2024
Open
@jonathan-eq jonathan-eq removed their assignment Apr 19, 2024
@JHolba JHolba self-assigned this May 14, 2024
@xjules xjules mentioned this issue May 14, 2024
Closed
9 tasks
@xjules
Copy link
Contributor

xjules commented May 29, 2024

Now I start to understand why this happens. Not sure if this is an inherit flakiness of checking self._connection.openwhich should be the proper way of testing it before calling connection.close() in monitor. However, when having this extra check then all hell got loose: https://github.com/equinor/ert/actions/runs/9284739204/job/25547764539#step:7:541

@xjules xjules self-assigned this May 29, 2024
@xjules
Copy link
Contributor

xjules commented May 31, 2024

Closing this issue as this has been not observed in last two weeks.

@xjules xjules closed this as completed May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JHolba JHolba

@xjules xjules

Labels
bug
Projects
SCOUT
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make sure not to close the evaluator _ws_thread when done is not set yet xjules/ert
4 participants
@berland @JHolba @xjules @jonathan-eq