-
Notifications
You must be signed in to change notification settings - Fork 299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Events Streams Blocking Console When Start Events Not Sent #2950
Comments
On Firefox console is loading forever if there are 6 open tabs - if e.g. one of the working tabs is refreshed - one of the loading tabs unblocks, but that seems to fail sometimes. In any case, I tried sending the start/end messages, but that did not change anything - it still hangs after there are 6 tabs. Note: all tabs are at data view of a gateway (2 different ones) |
@kschiffer any ideas? |
That is expected and a limitation unsing event streams while not using HTTP/2:
I have trouble reproducing the original issue since I only have access to two connected gateways on the staging environment. |
I can confirm this. The requests will stall if they don't get a response header. After six such connections being open, all subsequent XHRs will stall as well. Before #2565 this was not a problem since the event stream would always sent a "start stream message" immediately, which resulted in the headers being sent. As such, I don’t see it as a frontend issue. The only thing that would make sense to change on the frontend is making sure that stalled stream connections are killed after a certain time. |
In my experience sending "initial message" does not help with this issue. Please try yourself with the following diff:
Here's a video: |
I do see a difference. On
On
|
We have to differentiate this issue:
To mitigate 2, the response header needs to be sent immediately. One thing to note here as far as the console goes, we should look into also cancelling pending requests. Currently the requests can only be cancelled once the stream connection has been established. |
@kschiffer can you give steps to reproduce the issue in 2. ? |
The easiest way is to create a completely fresh and unconnected gateway since it will not send any messages. Then navigating to its overview page and back to the gateway list six times. This is not a problem specific to gateway events but all event streams by the way. |
Unassigning me since this is not an issue originating from the Console. |
Closed by #2989 |
A temporary solution was introduced in #2989 - a more robust solution is yet to be found. |
So this is either something in our dependencies (grpc-gateway not properly flushing headers) or in our shared code (maybe some grpc or http middleware not flushing the headers). Removing |
Please open a new issue if there's still something that needs to be addressed. |
Summary
Since #2565 the events streams in the console block after clicking around in the console.
Steps to Reproduce
What do you see now?
At some point the console keeps loading. This seems to be caused by "too many open connections" because
/api/v3/events
requests aren't closed.What do you want to see instead?
The console should stay responsive.
Environment
Build based on the current
master
branch 3df51cd (don't pay attention to the 3.8.5 that is shown by the console)How do you propose to implement this?
I suspect this is caused by the fact that headers are only written right before the first message on the stream. If there's no message, there's no header and the browser keeps waiting.
So to investigate, I suggest to:
How do you propose to test this?
Try the reproduction steps
The text was updated successfully, but these errors were encountered: