Add debugging output to spack and ramble installations

[cdunbar13]

When running the spack and ramble installations if something fails it's difficult to know which node to check for failure because it depends on which got the lock first. This is the first step to make debugging easier.

This update prints the hostname of the node that has the lock to the lock directory and if it fails it will print out the contents of the lock directory, which will then have the hostname of the node that failed.

After this PR is approved, the next step should be to get the stderr of any command that fails, write it to a file in the lock directory, then print the contents in the rescue block of the ansible playbook.

This was tested by deploying a blueprint that uses spack and ramble, waiting until a node had gotten the lock, then suspending that node, forcing the other nodes to timeout and print the new debug messages.

Failed output looks like:

May 10 13:57:17 ramble-test-0 google_metadata_script_runner: startup-script: TASK [Wait for lock] ***********************************************************
May 10 14:01:01 ramble-test-0 systemd: Started Session 4 of user root.
May 10 14:02:43 ramble-test-0 systemd: Starting GCE Workload Certificate refresh...
May 10 14:02:43 ramble-test-0 gce_workload_cert_refresh: 2024/05/10 14:02:43: Done
May 10 14:02:43 ramble-test-0 systemd: Started GCE Workload Certificate refresh.
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script: fatal: [localhost]: FAILED! => {
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script:     "changed": false,
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script:     "elapsed": 600
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script: }
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script: MSG:
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script: Timeout when waiting for file /shared/.install_spack_lock/done
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:17 ramble-test-0 google_metadata_script_runner: startup-script: TASK [Timed out on lock, get install directory contents] ***********************
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: changed: [localhost]
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: TASK [Print install directory contents with host that failed to install spack] ***
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: ok: [localhost] => {}
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: MSG:
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: total 8
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: -rw-r--r-- 1 root root    0 May 10 13:57 ramble-test-4
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: drwxr-xr-x 2 root root 4096 May 10 13:57 .
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: drwxr-xr-x 4 root root 4096 May 10 13:57 ..
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: TASK [Failed to get lock] ******************************************************
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: fatal: [localhost]: FAILED! => {
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script:     "changed": false
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: }
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: MSG:
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script:
May 10 14:07:18 ramble-test-0 google_metadata_script_runner: startup-script: Failed to get lock, exiting