Home

Welcome to the Maestro wiki!

Roadmap, path to 2.0+

Feature ideas

• looping on steps
• looping on subgraphs (multiple steps)
• looping on specs
• chaining specs
• dependency types/tokens for managing iterations/prev study workspace accessibility (i.e. $(step.workspace) constructs)
• new dependency types for steps meant to be run at the same time
  • start steps after an initial step is running instead of completed, and close that initial step upon completion of the others
• enhanced step controls
  • user controlled monitoring (periodic or simultaneous paired steps) to use real time step output monitoring to trigger stoping the step and mark failed/success/etc
    • e.g. useful for simulations where you don't know run times up front, or to catch a timestep crash early that needs stopping and tweaking to fix
• unified resource spec to enable true system portability
  • big question here is what to do about new style launcher syntax with respect to system specific resources like gpus since not all tasks can utilize those
• composable specs (reuse batch blocks/env bits/etc): new maestro compose [specs] commmand
• orchestration (facilities for bootstrapping allocations to submit work to
• sub conductors: better coordinate execution in multipel allocations/across machines
• non-scalar variable types
  • need formatting options for ensuring token replacement generates strings that are useful on cli's
• dynamic graphs
  • ability to add parameter combos to in progress workflows
  • ability to add new steps to existing workflows? (mindful of the provenance implications)
  • executed graph not prescribed up front -> conditional steps/loops/branches changing the path insitu