Rust: ops/extensions within a Worker?

[alshdavid]

I am embedding Deno within a Rust application to use as a JavaScript plugin engine and want to be able to send work to multiple Deno contexts (seperate threads) in a "round robin" fashion.

They don't need to be workers, they can be multiple independent instances of Deno.

To embed Deno I have vendored the cli folder into my project because the available Deno crates on crates.io are not sufficient to run a complete Deno instance with support for the Node.js standard library + Deno extras.

I don't need a lot of the functionality in the deno_cli crate (like the lsp, test runner, benchmark runner, etc) but it's very challenging to lean out the crate due to the many interdependencies + making changes to the vendored deno_cli package will make syncing with deno updates quite difficult.

Due to conflicting dependencies (swc, amongst others) - I have had to silo Deno into a seperate dylib crate and consume it from my main application via Rust's FFI (Rust's FFI is unstable but in my case the Deno lib is compiled specifically for the consumer using the same settings).

I have rewritten the entrypoint of Deno's CLI so that it exports the CLI commands as library entry points

deno run

becomes

mod deno_cli;

fn main() {
  std::thread::spawn(move || {
    deno_cli::deno_current_thread(async {
    
      let cli_main_worker = deno_cli::run(Flags{
        subcommand: DenoSubcommand::Run(RunFlags {
          script: "/file/path/main.js",
          watch: None,
        }),
        allow_all: true,
        custom_extensions: vec![] // <-- I added this to allow injecting my own custom extensions
        ..Default::default(),
      })

      cli_main_worker .run().await.unwrap();
      
    });
  }).join().unwrap();
}

reference

Initially, I simply tried to spawn multiple instances of Deno, each with its own channel which I use to send work to it.

fn main() {
  let mut senders = vec![];

  std::thread::spawn(move || {
     let (tx, rx) = channel::<DenoWorkerRequest>(); 
     deno_cli::deno_current_thread(async { /* as above */ });
     senders.push(tx);
  })
  
  std::thread::spawn(move || {
     let (tx, rx) = channel::<DenoWorkerRequest>(); 
     deno_cli::deno_current_thread(async { /* as above */ });
     senders.push(tx);
  })

  let deno_worker_farm = DenoWorkerFarm::new(senders);

  deno_worker_farm.send(DenoWorkerRequste::HelloWorld);
  deno_worker_farm.send(DenoWorkerRequste::HelloWorld);
}

Running multiple instances of Deno like this resulted in a segfault when more than one instance is launched (I assume this is because Deno CLI is using static globals for some internal state).

So I figured, if multiple instances of Deno can't coexist, then I should try to spawn multiple worker threads and talk to them from the host process.

I am using my custom ops to communicate to/from the JavaScript context so I figured I could just access them from within a worker however the create_web_worker_callback() function in Deno CLI does not accept custom extensions.

I tried to add custom extensions and run custom_extensions.clone() to pass them into the web worker, however this didn't work because Extension cannot be cloned.

So I tried to create a Vec<Vec<Extension>> where I would create multiple copies of my extensions (equivalent to the number of threads I will spawn) and each worker will grab their extensions with custom_extensions.pop(). This didn't work because something doesn't implement Send, so I wrapped the extensions in an Arc<Mutex<Vec<Vec<Extension>>>> but this also didn't work.

In the end, I modified the Deno CLI crate to hard code my extensions into the worker init options

This compiled and the ops were available in the workers - but obviously any changes to the vendored deno_cli package makes it harder to sync with Deno when there are updates.

However - this didn't work at runtime. Each worker calls my "hello world" extension which prints "hello world" - but it only prints once despite running on all workers.

My next experiment is to spawn multiple instances of the deno_embed.so dynamic library via libloading multiple times in the hope that I can create multiple independent Deno instances that don't share state.

Any advice would be welcome

[bartlomieju]

I am using my custom ops to communicate to/from the JavaScript context so I figured I could just access them from within a worker however the create_web_worker_callback() function in Deno CLI does not accept custom extensions.

We can accept a PR that adds ability to pass additional extensions to web workers.

However - this didn't work at runtime. Each worker calls my "hello world" extension which prints "hello world" - but it only prints once despite running on all workers.

That's most likely caused by the fact that you only initialize once and then clone. Should be solved if you can pass additional extensions to web worker and initialize them when passing.

[alshdavid]

Ok, cool.

So I can't pass Vec<Extension> to the create_web_worker_callback function because Extension cannot be cloned. It looks like it's not easy to modify to make it cloneable (because of op_state_fn and middleware_fn).

But to look past this and modify the WorkerOptions in create_web_worker_callback to add my ops directly, it does work.

The problem is my ops supply values into state, and I can't see a way to access the JsRuntime of a WebWorker to retrieve the values from the state.

Neither CliMainWorker or MainWorker appear to have accessors for workers

I could perhaps store the state in a static global, I will have trouble with storing types that require a reference to the JsRuntime (like callback functions)

[bartlomieju]

I'm not sure I understand this bit - could you maybe link to some code that you expect to work so I can get a better idea here?

[alshdavid]

Haha sorry for spamming your repo with questions, I'm really trying my best to get Deno integrated into my project. Here's the same question with some examples.

For the first part:

So I can't pass Vec<Extension> to the create_web_worker_callback function because Extension cannot be cloned. It looks like it's not easy to modify to make it cloneable (because of op_state_fn and middleware_fn).

I have an example with one hello_world() extension that just prints "Hello world" from Rust.

To make the op available to my workers I have to hardcode it into the WorkerOptions in:

• worker.js#592 (create_custom_worker)
• worker.js#783 (create_web_worker_callback)

Initially I tried to simply pass custom_extensions down into create_web_worker_callback however the fails to compile because Extension cannot be cloned. worker.js#531

Hardcoding the ops into deno_cli works but adds an extra step in syncing with upstream which is not ideal. Making Extension clone-able would allow me to reuse the custom_extensions argument in create_custom_worker().

The second problem:

The problem is my ops supply values into state, and I can't see a way to access the JsRuntime of a WebWorker to retrieve the values from the state.

I am passing a callback to my ops from JavaScript and will call that function from Rust at a later point in time.

Extension.register_callback(() => {
  console.log('called from Rust')
})

Normally, from your op, you'd put the callback into the op state

pub type CallbackState = Option<v8::Global<v8::Function>>

#[op2]
pub fn op_register_callback(
  #[state]: state: &mut CallbackState,
  #[global] callback: v8::Global<v8::Function>,
) {
  state.replace(callback);
}

deno_core::extension!(
  custom_register_callback,
  ops = [op_register_callback],
  esm_entry_point = "ext:custom_register_callback/src/custom_extensions/register_callback/register_callback.js",
  esm = ["src/custom_extensions/register_callback/register_callback.js"],
  state = |state| {
    state.put(CallbackState::None);
  },
);

Then later call the Js function by pulling it out of the op state within JsRuntime

let function: CallbackState = js_runtime.op_state().borrow_mut().take();

let request = {
    let mut scope = &mut js_runtime.handle_scope();
    let request = serde_v8::to_v8(scope, &[]).unwrap();
    Global::new(&mut scope, request)
  };

  let call = js_runtime.call_with_args(&function.unwrap(), &[request]);

  js_runtime
    .with_event_loop_promise(call, PollEventLoopOptions::default())
    .await.unwrap();

However when I have multiple workers storing their callbacks into their respective op states - the abstractions within deno_cli (CliMainWorker MainWorker) don't offer any obvious means to access the JsRuntime instances for the workers.

This means I can't access the JsFunctions from workers to later call from Rust.

So my workaround has been to have the JS callback remain in the op closure but block the completion of the function with a loop over a channel.

I then send the channel sender to the main thread to handle orchestrating messages.

• main.rs#L14
• main.rs#29
• worker_farm.rs#L31

This works but is a bit weird. I generally like to avoid static globals where possible.

Another question - Do workers run on their own threads? I don't see where another thread is spawned for the worker context. Is that handled internally within v8?