Launching a tool process within a flux job

[RichardBuntLinaro]

Hi All,

Does Flux provide support for launching tool processes on compute nodes within a job? I looked around the Flux source code an it indicates support for MPIR Process Acquisition. However, I do not see implementations for both MPIR_executable_path and MPIR_server_arguments that can be used to provide tool launch.

Many thanks,

Rich.

[grondo]

Currently MPIR_executable_path and MPIR_server_arguments are not supported in Flux. We can open an issue to support this style of tool launch though, thanks.

The current best practice for launching a tool alongside a job is to use the alloc-bypass jobtap plugin to launch a job using the same R (resource set) as the job in question. This requires loading the plugin in the instance, flux jobtap load alloc-bypass.so, fetching the R for the job in question, and then submit a job with the equivalent of flux run --setattr=alloc-bypass.R="$R" tool args.... E.g.

$ flux jobtap load alloc-bypass.so
$ flux submit -N4 sleep inf
ƒD4tZxAj
$ flux run --setattr=alloc-bypass.R="$(flux job info ƒD4tZxAj R)" -N4 -n4  hostname
pi1
pi3
pi2
pi4

Note: loading the alloc-bypass plugin requires instance owner privileges, which means this scheme only works in a Flux subinstance, e.g. a job created by flux batch or flux alloc.

We are working on a method to accomplish co-located tool launch via the job-shell which would drop this limitation, but no eta on that support yet.

[grondo]

See #5727

[grondo]

MPIR_executable_path and MPIR_server_arguments are now supported in flux job attach as of version 0.60.0. Warning: this has had no real world testing yet because I was unable to convince known parallel debuggers that support the MPIR tool launch extensions to recognize flux job attach as an MPI launcher process.