Bug: Easystack dependency resolution does not deduplicate submitted jobs correctly when using the --job argument.

[jkwmoore]

Bug description:

If you submit an EasyStack for installations of multiple software using job submission (I have tested only with Slurm), EasyBuild does not appear to correctly deduplicate dependencies before job submission.

i.e. if I install multiple bits of software (with --robot) that have a dependency on software "A", each of them will fire off job request to install "A".

Duplicated / extra requests for the same "A" dependency will fail (due to the open lock file) and this then breaks the build chain for the waiting upstream software installation jobs.

How to reproduce:

For a MVP reproduction, submit the following EasyStack file while using the --job argument (with other sensible choices in argument) and observe the duplicate jobs fail for the M4-1.4.19 install.

binutils.yaml

easyconfigs:
  - binutils-2.37.eb
  - binutils-2.38.eb
  - binutils-2.39.eb
  - binutils-2.40.eb

Possible fix:

Highly speculative as I have not read through the code, could deduplicate by building a list of jobs to be submitted, deduplicating these by easyconfig file and changing dependency references for the upstream jobs, then sending to Slurm or the chosen job engine with the updated (corrected) job dependencies.