tf apply should use the corresponding plan file from tf plan

[tammersaleh]

This came out of the cg-provision meeting on May 21st

Our cg-provision pipeline (also aws-broker and maybe others) is configured to run terraform plan, and in a later job run terraform apply. However, that apply doesn't make use of a saved plan file. This means infrastructure drift during plan review and before the apply job is triggered can result in unexpected and possibly destructive changes.

Instead, we should save the plan output from the terraform plan job and make use of it in the apply job.

options:

• Switch to using a different tf resource that saves the plan?
• Modify our tf task to upload plan to S3 and pass it to the apply stage?