Add a Python WebSocket class to the pyscript namespace

[ntoll]

Related to #1987 - we should add a consistent and well supported Pythonic means of using web sockets to the pyscript namespace. As with our approach to fetch, we should try to come up with an unopinionated "Pythonic" version of the underlying JS WebSocket API built into the browser.

This discussion is so we can arrive at a consensus for a solution before any implementation takes place.

I'd like at least a thumbs up from @WebReflection and others related to PyScript before we implement anything (so we avoid the refactor we did with fetch). Prodding @JeffersGlass @fpliger @FabioRosado @madhur-tandon @mchilvers @antocuni for more 👀 on this. 🙏

To kick things off, here's a straw man example 😱 that should work in exactly the same way with both Pyodide and MicroPython:

from pyscript import WebSocket, window


# Initialisation should handle the same args (including optional protocols) as per the 
# underlying JS class. The Python WebSocket class should also reveal the underlying 
# JS WebSocket's properties once the JS object is created. See:
# https://developer.mozilla.org/en-US/docs/Web/API/WebSocket#instance_properties
ws = WebSocket(url="wss://example.com/endpoint")


# Create and attach the event handlers, all of which receive a single event proxy.

def my_on_error(event):
  """
  Handle an error.
  """
  window.console.error(event)

def my_on_open(event):
  """
  Called when the connection is successfully open.
  """
  ws.send("Hello for the first time!")

def my_on_message(event):
  """
  Handle incoming data from the WS.

  The event data type should by a Pythonic object of the appropriate type, 
  related to the binaryType value. See: 
  https://developer.mozilla.org/en-US/docs/Web/API/WebSocket/message_event
  """
  window.console.log(event.data)

ws.onclose = my_on_close
ws.onopen = my_on_open
ws.onmessage = my_on_message
# Also ws.onclose is possible.


# Send a message.

# Check readyState. See:
# https://developer.mozilla.org/en-US/docs/Web/API/WebSocket/readyState
if ws.readyState == WebSocket.OPEN:
  # We need to decide what Python types the send method will accept (and 
  # how they map to the underlying JS types). See:
  # https://developer.mozilla.org/en-US/docs/Web/API/WebSocket/send
  ws.send("Hello again!")


# Manually close the connection.
ws.close()

# readyState is as expected.
assert ws.readyState == WebSocket.CLOSED

THIS IS A FIRST DRAFT and likely missing things, wrong or clunky. I've tried to add comments to the appropriate MDN docs for the protocol, and highlight where we need to make design decisions - mostly relating to how JS / Python types line up for the sending and receiving of data. My gut feeling is we should do the same Python<->JS translations as per the existing fetch API (as discussed from here).

Please add feedback, ideas and constructive critique.

Thank you! 👍

[AndrewD]

This sounds really useful. I'm thinking about streaming from a mqtt server, but haven't thought about the details. Or maybe wrapping a serial terminal session?

[ntoll]

Exactly, wherever you have a "flow" of data to/from the browser over a network, you have web sockets. The MQTT example is one I've already been thinking about, and wrapping a serial connection is also a good use case... the web socket is just the bridge to/from the wider world via the network.

[WebReflection]

beside the entry point where we should normalize options such as url (is that the only one? WebSockets doesn't have other options AFAIK so this should be enough WebSocket("wss://example.com/endpoint") with optional protocols we can ignore for now) I don't see the same value we provided by fetch which does much more by normalizing options across interpreters, provide bytearray() methods as well as json() that reflect Python dictionaries, in WebSocket everything is handled by the events and the data these contain (usually a string but I am OK normalizing non strings as Python buffers, if that's worth it).

All I am saying is that WebSockets are already possible via js.WebSocket.new(url) and I'd love to hear pain points from whoever used that API to date before wrapping our head for corners we've never hit so far ... is anyone in the list of cc asking for this and/or why? 🤔

To be clear, I am not opposed, I just struggle to see ourselves re-purposing potentially the whole js namespace for little gain ... in the case of fetch the gain was actually huge (imho), here I am not sure, I'd like to know or understand more, thanks!

[WebReflection]

P.S. that draft example will always fail because WebSocket are async in connecting so that close() will happen before the readyState gets to be OPEN, IIRC.

[ntoll]

I know that... 😉

But it's for illustrative purposes.

[ntoll]

@WebReflection Many Python folk don't know web APIs and having a (thin) wrapper around the built in API would, I believe, be very helpful. The value comes in several ways:

• There's always one "blessed" way to do it, rather than everyone re-doing a shim around the JS API.
• We recommend this way, because it works in all interpreters and we ensure this is the case.
• We do some subtle but very helpful convenience things for Pythonistas (such as binding the handlers, and Pythonising the data related objects).
• Most of the web docs will also make sense for our Python version.

I appreciate you pushing back @WebReflection - you make an important point. But the TL;DR is simply that a whole host of Pythonistas have a blessed Pythonic way to get stuff done. 😃

[WebReflection]

We do some subtle but very helpful convenience things for Pythonistas (such as binding the handlers, and Pythonising the data related objects).

your example doesn't bind the handlers, it attaches these directly and that means no create_proxy or whatsoever is ever needed because that's how current interpreters bindings work already.

If we want to create a thing layer that adds only those accessors it's another story, but then I can already foresee people asking why they can't add multiple listeners to the same socket ... do we want that?

If it's about simplifying the create_proxy dance for ws.addEventListener then fine ... if it's about providing ws.add_event_listener instead, it's fine too, but ws.onmessage = lambda x: x works already out of the box with everything ... so where are we making things better in this regard?

About the data pythonic reference there's only the Typed Array case to consider as strings are already strings in Python and Blobs ... we don't get to introspect these even in JS so are cumbersome regardless. Should we convert typed arrays into buffers out of the box? Yes is a valid answer as we do that already with fetch and bytearray() but I wonder if this is a goal of your proposal.

[WebReflection]

@ntoll I guess all I am saying is that when I've seen your example all its missing to work already is the second line of this counter-example:

from pyscript import window
from js import WebSocket

ws = WebSocket.new("wss://example.com/endpoint")

def my_on_error(event):
  window.console.error(event)

def my_on_open(event):
  ws.send("Hello for the first time!")

def my_on_message(event):
  window.console.log(event.data)

ws.onclose = my_on_close
ws.onopen = my_on_open
ws.onmessage = my_on_message

if ws.readyState == WebSocket.OPEN:
  ws.send("Hello again!")

ws.close()

assert ws.readyState == WebSocket.CLOSED

what are we then providing if not more confusion around the url named property to instantiate a WebSocket when the native API is sealed and documented already to accept just that url as only argument? (the protocols is a different story, the only one we should eventually tackle if we ever want to)

[WebReflection]

OK, we clarified this in the latest Discrod call and I am OK with implementing this, as result.

[FabioRosado]

Imo that would be helpful for folks, but I think perhaps we could go a level up and offer a websocket manager to manage connections and handle things for users, but perhaps that could be a bit too opinionated? 🤔

[Neon22]

IMHO a recipe may be as good as wrapping 'everything'.
If a python user is unfamiliar with JS web socket implementations (more than likely) then a reference, even one using complex code, is worth its weight in gold because I don't have to do tons of research to discover what to do and where all the gotchas are.

This is the case even for a pythonic wrapper. IMHO the same work, in terms of examples, needs to be done.

Having said that - wrapping aspects like json, bytearray is well worth the effort for ease of use and minimizing implementation errors.

As an example I refer to Jeff's recipe for uploading and downloading a file. Could have made special wrapper to do this but actually better to spell out the step by step approach. Everyone learns.

• https://pyscript.recipes/2024.1.1/basic/file-upload/

In fact this is a super useful reference which has been referred to several times to help people with questions about this action.

[fpliger]

Just adding this for posterity since we all agreed during the community call today: I'm definitely +1 on the proposal, keeping the same attention to "levels of abstraction" that considered around fetch (full featured and simplified)

[WebReflection]

this MR should address and somehow comply with the proposed solution: #2042