WebSocketClosedError from Streaming Indicators app

[MarcSkovMadsen]

When running https://awesome-panel.org/streaming_indicators on panel 0.12.6 and panel 0.13.0rc5 I sometimes get an error like below when I refresh the page.

2022-04-07 20:23:23,880 Task exception was never retrieved
future: <Task finished name='Task-9129' coro=<WebSocketProtocol13.write_message.<\.venv\lib\site-packages\tornado\websocket.py:1100> exception=WebSocketClosedErro
Traceback (most recent call last):
  File "c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
2022-04-07 20:23:55,807 Task exception was never retrieved
future: <Task finished name='Task-51737' coro=<WebSocketProtocol13.write_message.l\.venv\lib\site-packages\tornado\websocket.py:1100> exception=WebSocketClosedErr
Traceback (most recent call last):
  File "c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.
    await fut
retrieved                                        sed
retrieved
future: <Task finished name='Task-72631' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.py:1100> exception=WebSocketClosedError()>
Traceback (most recent call last):
  File "c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.py", line 1102, in wrapper
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.py", line 1104, in wrapper
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
2022-04-07 20:25:58,076 Task exception was never retrieved
future: <Task finished name='Task-142937' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.py:1100> exception=WebSocketClosedError()>
Traceback (most recent call last):
  File "c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.py", line 1102, in wrapper
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\repos\private\awesome-panel\.venv\lib\site-packages\tornado\websocket.py", line 1104, in wrapper
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError

[MarcSkovMadsen]

Try running this example and refresh it multiple times and maybe also open multiple sessions. In general the server seems to have issues running this example. Sometimes the page does not refresh. Sometimes you get the above or other error messages in the console.

Code

"""This example shows a streaming dashboard with Panel.

Panel runs on top of the Tornado server. Tornado is a fast, asynchronous web server built to
support streaming use cases.

In panel it's very easy to support periodic updates. Here it's done via
`pn.state.add_periodic_callback(_create_callback(indicator), period=1000, count=200)`

This Dashboard is work in progress. I would like to add some different types of stats cards
including some with splines/ plots. I would also like to add some icons to make it look nice.
"""
from typing import List, Tuple

import numpy as np
import panel as pn

STYLE = """
.pn-stats-card div {
  line-height: 1em;
}
"""

ACCENT = "blue"
OK_COLOR = "green"
ERROR_COLOR = "red"

if not STYLE in pn.config.raw_css:
    pn.config.raw_css.append(STYLE)

pn.extension(sizing_mode="stretch_width")

def _increment(value):
    draw = np.random.normal(1, 0.1, 1)[0]
    value *= draw
    value = max(0, value)
    value = min(100, value)
    return int(value)


def _create_callback(card):
    def update_card():
        card.value = _increment(card.value)

    return update_card



"""Returns an app"""
template = pn.template.FastGridTemplate(
    title="Streaming Indicators",
    row_height=140,
    accent_base_color=ACCENT,
    header_background=ACCENT,
    prevent_collision=True,
    save_layout=True,
)
template.main[0:3, :] = "Normally an intro section"

for row in range(0, 3):
    for col in range(0, 6):
        colors: List[Tuple[float, str]] = [(66, OK_COLOR), (100, ERROR_COLOR)]
        title = "Sensor " + str(row * 6 + col + 1)
        indicator = pn.indicators.Number(
            name=title,
            value=65,
            format="{value}%",
            colors=colors,
            css_classes=["pn-stats-card"],
        )
        template.main[row + 3, 2 * col : 2 * col + 2] = indicator

        pn.state.add_periodic_callback(_create_callback(indicator), period=1000, count=200)

for row in range(3, 5):
    for col in range(0, 3):
        title = "Sensor " + str(3 * row + col + 10)
        colors = [(0.7, OK_COLOR), (1, ERROR_COLOR)]
        indicator = pn.indicators.Gauge(
            name=title, value=65, bounds=(0, 100), colors=colors, align="center"
        )
        template.main[2 * row : 2 * row + 2, 4 * col : 4 * col + 4] = pn.Row(
            pn.layout.HSpacer(),
            indicator,
            pn.layout.HSpacer(),
        )

        pn.state.add_periodic_callback(_create_callback(indicator), period=1000, count=200)


if __name__.startswith("bokeh"):
    template.servable()

[hoxbro]

I have seen the same error with this:

import panel as pn
import time
import param


class Test(param.Parameterized):
    button = param.Event()
    text = param.String()

    @param.depends("button", watch=True)
    def countdown(self):
        x = 0
        while True:
            x += 1
            print(x)
            self.text = str(x)
            time.sleep(1)


pn.panel(Test()).servable()

crash.mp4

(Crash happens at 25)

---

I also get an error if I make countdown an async function, though with a slightly different error message:

2022-04-08 14:09:39,340 Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOMainLoop object at 0x7f9f73afcca0>>, <Task finished name='Task-68' coro=<async_execute.<locals>.wrapper() done, defined at /home/shh/Development/holoviz/panel/panel/io/server.py:111> exception=NameError("name 'asyncio' is not defined")>)
Traceback (most recent call last):
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/shh/Development/holoviz/panel/panel/io/server.py", line 114, in wrapper
    return await func(*args, **kw)
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 665, in caller
    yield from function()
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 396, in _depends
    yield from func(*args, **kw)
  File "/home/shh/Development/holoviz/multi2.py", line 17, in countdown
    await asyncio.sleep(1)
NameError: name 'asyncio' is not defined

[philippjfr]

Isn't that just a missing asyncio import ?

>  File "/home/shh/Development/holoviz/multi2.py", line 17, in countdown
    await asyncio.sleep(1)
NameError: name 'asyncio' is not defined

[philippjfr]

And for your example @MarcSkovMadsen I do get a bunch of "Failed sending message as connection was closed" warnings and sometimes StreamClosed errors but the app consistently loads fine and renders fine.

[hoxbro]

Isn't that just a missing asyncio import ?

Nope. It works fine until 20ish seconds after I close the browser.

import panel as pn
import asyncio
import param


class Test(param.Parameterized):
    button = param.Event()
    text = param.String()

    @param.depends("button", watch=True)
    async def countdown(self):
        x = 0
        while True:
            x += 1
            print(x)
            self.text = str(x)
            await asyncio.sleep(1)


pn.panel(Test()).servable()

2022-04-08 14:16:59,193 Starting Bokeh server version 2.4.2 (running on Tornado 6.1)
2022-04-08 14:16:59,193 User authentication hooks NOT provided (default user enabled)
2022-04-08 14:16:59,195 Bokeh app running at: http://localhost:5006/multi2
2022-04-08 14:16:59,195 Starting Bokeh server with process id: 43062
2022-04-08 14:17:06,714 WebSocket connection opened
2022-04-08 14:17:06,715 ServerConnection created
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
2022-04-08 14:17:28,275 WebSocket connection closed: code=1001, reason=None
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
2022-04-08 14:17:50,944 Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOMainLoop object at 0x7fa2aea9f760>>, <Task finished name='Task-73' coro=<async_execute.<locals>.wrapper() done, defined at /home/shh/Development/holoviz/panel/panel/io/server.py:111> exception=NameError("name 'asyncio' is not defined")>)
Traceback (most recent call last):
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/shh/miniconda3/envs/holoviz/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/shh/Development/holoviz/panel/panel/io/server.py", line 114, in wrapper
    return await func(*args, **kw)
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 665, in caller
    yield from function()
  File "/home/shh/Development/holoviz/param/param/parameterized.py", line 396, in _depends
    yield from func(*args, **kw)
  File "/home/shh/Development/holoviz/multi2.py", line 17, in countdown
    await asyncio.sleep(1)
NameError: name 'asyncio' is not defined

[philippjfr]

Oh right, yes, that's when the bokeh clean job will destroy the module. I'm not quite sure how we can also have it cancel any tasks running on the asyncio loop.

[MarcSkovMadsen]

As a user getting a lot of warnings and exceptions you don't know any thing about or how to handle is makes you wonder if the server is robust. The world is much simpler without these if they are not needed.

[Prinzhorn]

I'm just passing by from a global GitHub search for tornado.websocket.WebSocketClosedError and didn't read everything in detail. This might be caused by a race condition in tornado, which would cause the cleanup code in on_close to run before the open handler even creates self.connection

tornadoweb/tornado#2958

panel/panel/io/jupyter_server_extension.py

Lines 175 to 177 in 168d9af

	def on_close(self):
	if self.connection is not None:
	self.connection.detach_session()