[feat] worker thread pool and other performance optimizations

[naz]

Was looking into how resource intensive worker creation is through bree. I did not come across concrete numbers of the overhead associated with worker_thread instantiation. Based on the reference below it should not be considered as a "cheap" operation to run.

From Node.js worker threads documentation:

use a pool of Workers instead for these kinds of tasks. Otherwise, the overhead of creating Workers would likely exceed their benefit.
When implementing a worker pool, use the AsyncResource API to inform diagnostic tools (e.g. in order to provide asynchronous stack traces) about the correlation between tasks and their outcomes. See "Using AsyncResource for a Worker thread pool" in the async_hooks documentation for an example implementation.

(linked Worker thread pool implementation)

Was wondering if there are concrete performance metrics available illustrating an overhead associated with worker thread creation? Is it planned to utilize worker thread pool pattern (there's a Pool implementation available in bthreads)

Another idea I was contemplating was around optimizing non-CPU intensive jobs. Would it make sense to bypass worker creation and run such jobs in the main thread without breaking out of event loop? What's the threshold which would determine if it's more resource efficient to run the task on the main thread or offload it to a worker thread?

[niftylettuce]

This would be absolutely sick to have. I know that definitely some processes run WAY faster when not in worker threads - but the whole idea is the job isolation in their own threads (which is the "better" approach). I guess just get a faster CPU 😄