Notes on using Lonboard in a remote (esp JupyterHub) environment

[kylebarron]

I was pairing with @wildintellect on debugging why Lonboard wasn't working in a JupyterHub environment and we came up with a couple notes.

In particular, it's very easy to get lost in environments. Generally a JupyterHub server has at least two Python environments. There's a server environment in which JupyterHub itself is installed and from which it's launched, plus one or more user client environments. As far as I can tell, a Jupyter widget needs to be installed in both environments, and presumably those versions must match.

The widget installed on the server environment is necessary to provide the JS bundle as part of the webpage the user loads, while the widget installed on the user environment is necessary so the Python model works.

• Are there ways to provide helpful errors when no JS code is found? Only a JS console error is produced. Is there some way to catch that and provide a more helpful error to the user?
• Is there a good way to validate that the frontend and backend versions of lonboard match?

Aside from lonboard-specific tasks, I wonder if there are tasks that JupyterHub could do that would simplify the loading of widgets so that they don't need to be separately pre-installed on the server. Additionally, ensuring versions match between the user's environment and the global server environment sounds messy.

I've noticed that Colab handles this problem quite well (I think it's with their googlecolab/colab-cdn-widget-manager project). In particular, the first time that a widget is attempted to be rendered, the JS code for that widget is dynamically added to the page. This means that

!pip install lonboard
from lonboard import viz
viz([])

will just work, even without restarting the kernel or reloading the browser tab, because the JS side doesn't need to exist already when the Python side is created.

cc @batpad

[kylebarron]

Unsurprisingly, there's been discussion on this before. See especially manzt/anywidget#385 (comment).

Note that the architecture of anywidget is that anywidget serves as a bridge between "fully bundled" widgets that need to be installed in full on the server and "dynamically imported" widgets. Anywidget itself loads the JS from the anywidget-based widget. Therefore only a compatible version of anywidget needs to be installed on the server. In our case, lonboard does not need to be installed on the server, only anywidget.

There are further questions about ensuring compatible versions of anywidget between frontend and server, but that's a more general question outside of lonboard's scope.

Solara also does some nice checks around this: manzt/anywidget#385 (comment)

[paddymul]

FWIW In my own IPYwidget library buckaroo, I addressed some of this.
Users frequently pip installed the library without restarting the server. This causes a JS error with no python error. I added a check on the MTime of the python library, and the start time of the server.

https://github.com/paddymul/buckaroo/blob/main/buckaroo/widget_utils.py#L37-L55

I'm interested to hear about other approaches around this problem. It's a really poor experience for an initial user install. Maybe you could source the JS from unpackage when the user would encounter the problem.