Ensure that a user-defined thread count takes precedence over getNumberOfLogicalThreads() #460
Conversation
…erOfLogicalThreads() This commit ensures that a user-defined thread count consistently takes precedence over the detected number of logical threads. Previously, Embree would fail if the user-provided thread count was larger than the internally determined one.
|
I would appreciate feedback on this and would also be happy to adjust the PR as needed if there are still some issues with it. Thanks! |
|
Hi! Thanks for the PR! This was merged into our internal devel branch a while ago and is already released: fff2e0f. It would be interesting to know why use use the Embree's internal tasking system. What's the reason why you don't use TBB? I only ask because we discuss from time to time whether we want/can remove the internal tasking system to reduce maintenance overhead. But if this is something that is really valuable to you we'll of course keep it alive. Cheers, |
|
Ah great, I didn't realize it got merged! We encountered this issue when using parallel scene commit using Mitsuba (https://github.com/mitsuba-renderer/mitsuba3). Mitsuba doesn't use TBB, it uses it's own threading system. As far as I can tell, the only interaction with Embrees internal tasks is during scene construction. I am not sure what else the embree task system supports, but for us the main use is to run |
|
To add further historical context: Mitsuba/Dr.Jit implement a JIT-compiler that automatically parallelizes user-provided code. The multi-threading aspect of this originally relied on TBB via the In an application that already maintains an internal thread pool, it seems wasteful to spin up another one to build a BVH. So the fact that Embree has a "bring your own threads" interface via |
Hi,
This pull request ensures that a user-defined thread count (i.e., in the device's config argument) consistently takes precedence over the detected number of logical threads. Previously, Embree would fail if the user-provided thread count was larger than the internally determined one. It internally allocated threading related buffers based on the return value of
getNumberOfLogicalThreads(), which would then cause a segfault when invokingrtcJoinCommitSceneusing a thread pool that has a larger number of threads. Concretely, I encountered this when running Mitsuba 3 on a cluster setup. Currently, Embree effectively requires the user thread pool to be smaller or equal togetNumberOfLogicalThreads(), but this is not always straightforward to guarantee. Moreover, I would argue that this is neither documented nor immediately obvious for users.This fix seems to work for me, but I am open to suggestions if it is not general enough to be merged in this state.
Best,
Delio