optimizing to reduce AWS costs

[urbien]

Problem

We need to further reduce the costs of running a stack in AWS. As we do that we need to learn about similar services in Azure and GCP.
In fact a good approach would be to create equivalent Terraform (or CDK) resources so that we build an equivalent to Cloudformation for Azure and GCP as we review and improve our AWS setup.

Importance

When a bank is running MyCloud the costs are already negligible but for their SME client this cost could present a barrier to entry.

Details

1. The bulk of current costs is EC2 which we use for running 3rd party code in Docker containers managed by ECS. We need to continue our migration to Docker in Lambda, at least for the facematching and liveness detection services.
2. 15% Cloudwatch - if I recall correctly we have one hour time to live (TTL) for log events in Cloudwatch before we move the data to S3. Perhaps this period can be 10 min, but we need to account for any possible outage in S3 so that we do not loose any logs.
3. 10% S3 - each cloud still generates a massive amount of logs, one for each service each minute. Do we have a TTL for the logs in S3? If not, this could be added. Also, we can migrate logs in s3 to a cheaper storage tier - AWS allows that automatically.
4. 5% Lambda - the bulk of our costs here are for warming up Lambdas, see @spwilko's comment on issue stack launch in AWS sub-account fails #26. As an alternative, we need to review the costs of reserved capacity for Lambdas that AWS offers now. We also need to review why we send warm up requests so often that we reach over 50 concurrent requests. Perhaps we can reduce the rate there and, also, perhaps not all Lambdas need the same rate of warm up.

@martinheidegger, please work with @spwilko to analyze the costs and prioritize based on cost reduction.