SCF Workchain: New AiiDA feature - Finish convergence via monitors

[janssenhenning]

AiiDA v2.2 was released yesterday and while looking over the changelog I noticed a feature that might be interesting especially for the SCF workchain, which is live calculation job monitoring

This allows to attach functions which can read the files on the remote computer and decide to take actions, while the job is still running

This can be useful for two scenarios:

• Cancelling calculations that don't converge (either if it exceeds a certain distance or if the slope of distances is too shallow)
• Converging with respect to quantities that the fleur code does not consider as convergence criteria, especially total energy or torques (for the FleurRelaxTorqueWorkChain). Atm this is handled after each fleur job is finished which might be too many iterations, wasting resources

There are interesting open questions with the second point

• How would a monitor cancel a job and AIiDA still pick it up as a successful finish.
  • One idea could be for the monitor to create a flag file like JUDFT_NO_MORE_ITERATIONS, which would tell fleur to gracefully exit after finishing the next iteration. This requires fleur code changes (Added for the next fleur release)
  • Maybe the information which monitor cancelled the job could be passed through to take it into account