You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Beaker experiments have significant overhead. When we're running many small Tango steps, we can save some time by running multiple steps at once.
An issue is that we don't know ahead of time how long a step will take, so we don't know which steps make sense to group together. When running like this, it would make sense for Beaker experiments to poll the workspace for new work. This would be a drastic change in how we execute things, and it would overlap with Beaker functionality in uncomfortable ways. This feature requires some careful thought.
Alternatives
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
The caveats in the description of this issue stand, but we can probably find sub-graphs of the whole execution graph that always make sense to group together into one Beaker experiment. A run of steps where each one depends on the previous step, and no other dependencies are unfulfilled, those could always be grouped. It would cut the overhead of my current experiment by 66%.
A run of steps where each one depends on the previous step, and no other dependencies are unfulfilled, those could always be grouped. It would cut the overhead of my current experiment by 66%.
That was my thought as well. These steps would also have to have matching step resource requirements.
馃殌 The feature, motivation and pitch
Beaker experiments have significant overhead. When we're running many small Tango steps, we can save some time by running multiple steps at once.
An issue is that we don't know ahead of time how long a step will take, so we don't know which steps make sense to group together. When running like this, it would make sense for Beaker experiments to poll the workspace for new work. This would be a drastic change in how we execute things, and it would overlap with Beaker functionality in uncomfortable ways. This feature requires some careful thought.
Alternatives
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: