RosbridgeProtocol instance clean-up hangs when client disconnects under specific conditions.

[ramlab-jose]

Description
Under specific conditions that I have not yet been able to pin down a client disconnection will not be gracefully handled, leading to the server attempting to forward messages to that (closed) client's websocket and thus spamming errors. This also leads to a leakage in resources and eventual lock up of the rosbridge process.

The problem seems to come from this last part of the Protocol.incoming function. After adding a bunch of logging it seems that this blocks the IncomingQueue.run loop and thus the protocol.finish of a given client is never triggered.

As mentioned I have yet to find a minimal way to reproduce the problem, but we have frequently encountered this when there are rapid connections/disconnections happening and the rosbridge instance is under load.

I was able to "fix" the problem by improving the behavior regarding the remaining message that is kept in self.buffer here but would like some input on why this is here and how we can fix it properly.

Thanks in advance, and I believe this could explain some of the other issues that went stale in the past.

• Library Version: latest ros2 branch
• ROS Version: Humble
• Platform / OS: Ubuntu 22.04 (docker)

Steps To Reproduce
I have yet to find a reliable way of reproducing the problem, but from my experience the following conditions seem to trigger the problem:

• 10 clients that rapidly connect, subscribe to a few topics and call a "long" running service, followed by a disconnection. From the debugging I have done it doesn't seem that the service call should be needed but perhaps it helps use-up some resources that help trigger the buffering of the websocket.

Expected Behavior
A client disconnecting should always result in the respective RosbridgeProtocol instance (and respective Capabilities) being cleaned up.

Actual Behavior
Sometimes the clean up (.finish) seems to hang and resources remain being used against a closed websocket.