-
Notifications
You must be signed in to change notification settings - Fork 978
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
callback on idle threads #992
Comments
Hi @fwyzard, could you please describe more your use case? Since the still loop is a part of task stealing mechanism it is might be considered as a working state. In balanced scenarios this might introduce some overheads because we will insert the V-call on the hot path. |
Hi @pavelkumbrasev, of course. CMSSW is the software used by the CMS experiment at CERN for the simulation, physics reconstruction and analysis of the experimental data, and its framework relies heavily on TBB to implement task-based multithreading. An optimised application may run order of 10k TBB tasks per second per CPU thread. However we also have cases where the threads are idle, mostly because of two reasons:
We implemented a "service" inside the application that tracks how much (cpu and real) time is spent inside each "module" of the application (roughly, a module maps to a TBB task). However we don't have a way to measure how much time the threads spend idling, for example because of one of the two reasons above. Hence my attempt to extend TBB to let it notify us about idling tasks :-) So far, I thought to generate a pair of calls ( Let me know what you think, and if I can provide more information ! |
Seems your right and |
Yes, my goal is to monitor the active vs idle time, not to affect the properties of the threads. I know that the fraction of time spent idle in scenario 1. can be significant. I do not know yet what the fraction of time ending up in scenario 2. could be - first I would need to find a way to monitor when it happens :-) |
I can review your PR once you submit it and after it we could discuss applicability of this API in general case. |
Sure, thanks. Though, I have one question first. Thank you, |
TBH that pretty complicated logic so on first glance |
@pavelkumbrasev, thanks for the suggestions. Please find at #995 my first attempt to implement the idle thread notifications. |
It's split into two commits:
|
From a first round of checks in our application, it seems to behave as intended. Next I'll try to implement some proper monitoring on top of it, and then measure the impact on the application performance. |
Hi @fwyzard any updates regarding your research? |
Hi,
I am looking for a way to keep track of when and how long any TBB thread stays idle within (the framework used by) my application.
After reading a bit of the oneTBB code, my understanding is that I can consider a thread to be idle when it is in the stealing loop at https://github.com/oneapi-src/oneTBB/blob/v2021.8.0-rc1/src/tbb/task_dispatcher.h#L193-L232:
oneTBB/src/tbb/task_dispatcher.h
Lines 193 to 232 in bd619c5
Is this reasonable? That is,
As for a way to notify my application when a thread is idle, I have been thinking to extend the
task_scheduler_observer
adding two methodson_thread_idle()
andon_thread_active()
.Does this seem like a good approach ?
Thanks for any comments and suggestions - next I'll see if I can implement something along these lines.
.Andrea
The text was updated successfully, but these errors were encountered: