Upload pre-processed template to S3 before sending to CloudFormation for execution

[dannosaur]

So I'm writing a behemoth of a stack, that can't really be nested and split into separate templates. Now I note that iidy can reference templates on S3, presumably for pre-processing and uploading to CloudFormation via the request body for execution, but I'm not seeing anything that suggests that iidy can upload the pre-processed template to S3 for execution via an S3 path (thus allowing for a larger template to be used).

Basically, I need to solve this problem...

error Your cloudformation template is larger than the max allowed size. You need to upload it to S3 and reference it from there.

A process that involves iidy render, a manual S3 upload (doesn't matter how), then somehow getting iisy to execute the remote S3 path, seems a bit..clunky, and it certainly makes sense that iidy should be able to do this.

Any pointers/ideas?

[tavisrudd]

You could probably do this via some shell commands triggered by the CommandsBefore: option in stack-args.yaml at the moment. I've encountered that issue previously and considered adding more transparent support for it. The template approval command implementation has some of the s3 upload plumbing required to implement it.

@jpb thoughts?

[jpb]

CommandsBefore would certainly work as a workaround for now, something like:

Template: remote-template.yaml
CommandsBefore:
  - aws s3 cp local-template.yaml s3://bucket-name/remote-template.yaml

I agree that handling this transparently with something similar to ApprovedTemplateLocation would be a great improvement.

[dannosaur]

@jpb yea, that's exactly what I ended up doing.

Template: https://s3-eu-west-2.amazonaws.com/bucket/remote.template.yml
CommandsBefore:
  - iidy render stack-args.yml | aws s3 cp - s3://bucket/remote.template.yml

Works remarkably well, though I did spot another bug which I'll raise as a separate issue.