Loading delayed when PTHREAD_POOL_SIZE is set

[aharondavid]

I have a delay in loading when linking with PTHREAD_POOL_SIZE=60.
as debugging in Chrome DevTools, I see that for every thread, two files are loaded:

• myProject.Emscripten.worker.js
• myProject.Emscripten.js (The main JS, and it is big)

I would glad to know why the main JS should be loaded for every worker.js.
is there any optimization that I can do ?

[sbc100]

Yes, each worker/thread does need to load all of the JS code and all of the wasm code.

Basically each thread needs to be complete copy of the program, much like it would be when you create a thread on a native platform.

[sbc100]

Do you really need to set -sPTHREAD_POOL_SIZE=60, i.e. does your main thread need to be able to create and use all those threads right away without returning the event loop?

(As long as you return to event loop in your program before communicating with new threads you shouldn't need to use a thread pool at all).

[aharondavid]

No it doesn't need to use all of them right away.
The problem is that the Thread Pool is contained all the threads, including those that I not need to use them right away.
I think, it would be great if we have the control on which created thread will use the Pool Threads.
In that case, we can significantly reduce the size of PTHREAD_POOL_SIZE, because, we will create a thread by the Pool Threads only if we know that this thread is going to be used right away after it been created. otherwise, the thread will be created independently (not insert to the Pool Threads)

[sbc100]

Note the might in the warning message. For many types of application growing the pthread pool at runtime is just fine.

[aharondavid]

After that warning I get a stuck - probably a deadlock.
so, if the Thread Pool grow on demand, is it mean that I can use in a thread immediately after it created ?
(I am using in emcc 3.1,9)

[sbc100]

The main thread need to return to even loop before new thread is actually created, usable.

One solution is use PROXY_TO_PTHREAD or to otherwise move the code that wants to start threads, to a background thread.

[aharondavid]

I thought that setting the PTHREAD_POOL_SIZE is a solution for using a created thread immediately without return to event loop (as it mention here: https://emscripten.org/docs/porting/pthreads.html#special-considerations)
may be I wrong, but, actually, I feal that it is works (using the created thread without going to event loop) when I set PTHREAD_POOL_SIZE

[sbc100]

Yes, PTHREAD_POOL_SIZE does allow this. But one downside of requiring a large pool is that your application will take a long time to start.

Some applications only require immediate access to some amount of initial threads so that have a small initial pool, and then growing that pool for later threads can work. Other applications use PROXY_TO_PTHEAD, or spawn there threads from a thread other than the main thread (to allow the main thread to run the event loop). There are many workarounds, but specifying an enormous PTHREAD_POOL_SIZE is probably not very practical.

[aharondavid]

about 'growing pool...', I did a test that do a creating a lot of pthread.
I expected to see the list of worker are growing, but it didn't.

[sbc100]

You can't great a pthread without creating a worker, so if you create N pthreads you will always see at least N workers.

If you have create N pthreads the pool will grow to at least N. All new workers go into the pool.

[aharondavid]

Actually, with PTHREAD_POOL_SIZE=60 the loading time is very fast for a real domain Server.
it is slow (about 9 seconds) when the URL is localhost, with or without cache.
not clear to me why. (tested in Chrome)