Allowing for an async function event to take place within webr::eval_js()

[coatless]

I'm trying to write an async function to potentially be included in the {webr} support package. The function seeks to check if data associated with a URL can be retrieved and used in the webR session. The determination is built on the criteria of the URL:

1. using the https protocol, and
2. a request can be made for its contents as it is CORS compatible.

The problem arises when I'm seeking to run the underlying JS function that uses await fetch() to retrieve the HEAD of the response (not body contents to avoid duplicate downloading). This portion is given by:

async function checkCORSTestURL(url) {

    const response = await fetch(url, { method: 'HEAD' })
    .then(
        async (response) => {
            return response.ok === true ? true : -5;
        }
    )
    .catch(
        (error) => {
            return -1;
        }
    )

    return response;
}

// Test URLs
const goodURL = 'https://raw.githubusercontent.com/coatless/raw-data/main/common.csv';
const badURL = 'https://a-random-url-that-goes-nowhere-anytime-soon.com'

// Call the async function with good URL
const webrCORSTestURLGood = await checkCORSTestURL(goodURL);
console.log('Is CORS supported?', webrCORSTestURLGood);

// Call the async function with bad URL
const webrCORSTestURLBad = await checkCORSTestURL(badURL);
console.log('Is CORS supported?', webrCORSTestURLBad);

For all intents and purposes, this works nicely in web dev tools:

When I move the function over to use webr::eval_js(), I end up receiving:

Error in webr::eval_js("...") : 
  An error occurred during JavaScript evaluation:
  await is only valid in async functions and the top level bodies of modules

If I remove the async and await portions of the above code, then I end up getting a promise being sent back that will be undefined and, thus, return a clean status code of 0.

Is there a good way forward to having promises get resolved? Or should I host a web form that says "check your data URL here"?

[georgestagg]

Currently, there is no way to use async/await or to wait for promises directly within a webr::eval_js() evaluation. This is a limitation in the nature of the default webR communication channel, where R blocks inside the JavaScript worker thread. The blocking means that the thread never yields to the JavaScript event loop, and so asynchronous events (such as fetch() requests) never fire [1].

There are a few things that might work (in order of ease of use), though none are perfect:

1. If possible, use a synchronous XHR rather than the fetch() API. This is allowed by the browser since R is running in a worker thread.

2. Try to use R's later and promises packages to implement the asynchronous request. This might or might not work, depending on if and when the fetch() actually fires when running with the default communication channel. I would have to experiment to say for sure if this can work.

3. Send a system message to the main thread, asking the main thread to make the fetch() request, then send the result back from the main thread, probably by invoking a function pointer through the internal invokeWasmFunction() function. This works because the main thread is not blocked in the same way. See here for current examples. Be aware this approach will be very tricky to make work well, and I wouldn't want to add further types of system messages lightly. Still, it is a technically valid approach.

---

Now, after saying all that, I think this should be solved in a more general way. Eventually, I'd like a family of eval_js() functions that can take R object arguments, converting them into JS objects for evaluation. Crucially, I'd similarly like these functions to be able to return other types of JS objects, not just numbers, converting them into R objects on the way back.

WebR already has code in place to convert different types of JS objects into R objects, and we can take advantage of that system when returning objects with future versions of eval_js().

In the long term, I'd also like that system to be able to understand JS Promise objects and turn them into promises in R[2]. With that, any async JS code could be turned into a promise accessible from R, capturing the above use case. Probably the implementation will either involve some form of the 3) method above, or directly make use of Wasm promises once browser support improves.

---

[1] Compiling with Emscripten's Asyncify support would allow us to yield, but it does not work well for us due to the way the R interpreter works and the large asyncify overhead induced.

[2] I'm as yet unsure how best to implement this. I will need to investigate internal promises, {promise}, {future}, {mirai} etc.