Add a fetch function to the pyscript namespace in the "standard library"

[ntoll]

A common task in the browser is to fetch resources from the web. JavaScript has the fetch function to do this. We should provide a wrapper for this capability that works with both MicroPython and Pyodide.

We should probably also do the same for WebSocket objects too.

I imagine code that looks something like (rough draft, subject to refinement):

import json
from pyscript import fetch

response = await fetch(
    "https://example.com", 
    method="GET", 
    headers={
        "Content-Type": "application/json", 
        "Authorization": "Bearer some_token_123",
    },
    body=json.dumps({
        "foo": "bar",
    })
)

response.status  # 200
response.data  # The raw data
response.json  # A Python data structure generated by json.loads on the data.
response.headers  # A dict of the response headers
... etc...

[JeffersGlass]

I like this idea - it may be worth looking at Pyodide's pyfetch function in their FFI for inspiration - or at least, maybe we put in the docs that this is different from pyfetch to minimize confusion.

[WebReflection]

@hoodmane already mentioned this indeed but this API requires its own documentation as few things are easier/different but of course we should mention other approaches too, including pyfetch and underlying at the end all we are doing is just to provide a more Python friendly fetch.

The MR now showcases that and I think I really like it: feel free to review it 🙏 #1994

[WebReflection]

To be honest we already have that:

import json
from pyscript import window

response = await window.fetch(
    "https://example.com", 
    method="GET", 
    headers={
        "Content-Type": "application/json", 
        "Authorization": "Bearer some_token_123",
    },
    body=json.dumps({
        "foo": "bar",
    })
)

From a worker this would/should work out of the box, from the main thread this would require awkward dictionary to object literal conversion ... I think it is indeed probably better to actually expose a fetch that automatically does such conversion without needing to pass through the window or the js import but I don't think we should go too fancy about the API as it's already very well documented all over the Web: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API

[ntoll]

Bravo. Perfect. 🚀

[WebReflection]

OK I have all things working in both Pyodide and MicroPython except the blob() which is problematic as it's not clear even in JS what kind of data it represents and, if opaque is used it's completely useless in all worlds ... are we sure blob() case is really important? The way blob is used on the JS side is really JS dependent and it has to deal with JS APIs only that accepts blobs ... I don't think in this case we really need a conversion to Python because the meaning of that primitive, for Python code, has no usage.

The bytearray seems to be better to me, as long as it's not converted back to JS as that would make the thing slower for no gain, but the blob in particular, I feel like it should be left as it is returned and not changed or we just skip that one too ...

[WebReflection]

last, but not least, Pyodide converts array buffer via EMScripten equivalent https://github.com/pyodide/pyodide/blob/ece8e95916ad6965f96114eb49a5f0b04f45ce09/src/js/types.ts#L72-L76

I am not sure that's a bytearray but it feels like it's safe to assume that (it's not, keep reading) ... I wonder if we should try though to use the FFI to convert these although that might lead to further issues in case these FFIs don't agree on the final datatype.

For instance, Pyodide is currently transforming an .arrayBuffer().to_py() as <memory at 0xf10938> which is still a buffer and it has no view attached to it, and as the name suggests maybe returning a bytearray there would be a stretch over the API?

My gut tells me that .arrayBuffer() should return just a buffer as meant by either Pyodide or MicroPython FFI (whenever the latter one is available) while .bytearray() should return a bytearray() in both cases. This will make things more aligned and easier to explain in the bytearray() case and less surprising in the arrayBuffer() as meant by the original fetch API intent and as represented as real buffer in Python too. What do you think?

[ntoll]

Hey hey.

OK... let's drop blob as you suggest. I like your explanation of .arrayBuffer() and .bytearray():

My gut tells me that .arrayBuffer() should return just a buffer as meant by either Pyodide or MicroPython FFI (whenever the latter one is available) while .bytearray() should return a bytearray() in both cases. This will make things more aligned and easier to explain in the bytearray() case and less surprising in the arrayBuffer() as meant by the original fetch API intent and as represented as real buffer in Python too.

LGTM.

[WebReflection]

it's green #1994 ... and I've left the .blob() as it is as it might be handy and it requires no maintenance for the time being ... happy to remove it but I feel like it can stay until we have reasons to drop or change it

[WebReflection]

this is in with our best API yet