No reference of how to specify cloudwatch metrics for autoscaling

[mamccorm]

Found a bug? Maybe our Slack Community can help.

Describe the Bug

Hi there, I have a query regarding this terraform module, for reference:
The following terraform module:

• https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-autoscaling

The description is as follows:

Terraform module to autoscale ECS Service based on CloudWatch metrics

The module itself seems to define the criteria for the auto-scaling, such as minimum and maximum amount of tasks, however I don't see any info or arguments to specify the Cloudwatch metric(s) to associate with. i.e how do we use this module to setup autoScaling with a Cloudwatch metric, such as CPU or memory utilisation, whenever there are no arguments to support this.

As an example, if I were to do this utilising the built-in terraform modules, i'd first look at leveraging the following to create a target:

• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_target

Then i'd attach a policy to it, with the targeted CloudWatch metric utilising:

• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy

Expected Behavior

Given the description of this module states that it is for creating ECS auto-scaling based on Cloudwatch metrics, id expect that it would allow me to specify the Cloudwatch metric to associate with it.

Steps to Reproduce

N/A - but details above

Screenshots

N/A - detail above.

Environment (please complete the following information):

N/A - details above

[nitrocode]

@mamccormick isn't that what the following does ?

terraform-aws-ecs-cloudwatch-autoscaling/main.tf

Line 15 in 7a9c37e

resource "aws_appautoscaling_target" "default" {

terraform-aws-ecs-cloudwatch-autoscaling/main.tf

Lines 24 to 29 in 7a9c37e

	resource "aws_appautoscaling_policy" "up" {
	count = module.this.enabled ? 1 : 0
	name = module.scale_up_label.id
	service_namespace = "ecs"
	resource_id = "service/${var.cluster_name}/${var.service_name}"
	scalable_dimension = "ecs:service:DesiredCount"

terraform-aws-ecs-cloudwatch-autoscaling/main.tf

Lines 43 to 48 in 7a9c37e

	resource "aws_appautoscaling_policy" "down" {
	count = module.this.enabled ? 1 : 0
	name = module.scale_down_label.id
	service_namespace = "ecs"
	resource_id = "service/${var.cluster_name}/${var.service_name}"
	scalable_dimension = "ecs:service:DesiredCount"

[gausnes]

If this module is consumed without using the outputs incomplete monitors will be created:

If you click through the AWS console wizard it instructs you to create or target a CloudWatch alarm. The Terraform within this module doesn't provide any context on what to scale on, just how to scale. Looks like the outputs should be wired into alerts to create functional monitors.

[nitrocode]

Ah then you probably need to use this module with the ecs sns module

https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-sns-alarms

[daltonsena]

As @gausnes already mentioned. This module won't create the autoscaling action by itself. What we need to do is combine the outputs (up and down) with a CloudWatch alarm.

One example that works for me:

module "cs_autoscaling_label" {
	source     = "../modules/null-label"
	name       = "cs-autoscaling"

	context = module.this.context
}

module "ecs_cloudwatch_autoscaling" {
	source                = "cloudposse/ecs-cloudwatch-autoscaling/aws"
	context               = module.cs_autoscaling_label.context
	cluster_name          = aws_ecs_cluster.ecs_cluster.name
	service_name          = module.content_service_fargate.service_name
	min_capacity          = var.min_capacity
	max_capacity          = var.max_capacity
	scale_up_adjustment   = var.scale_up_adjustment
	scale_up_cooldown     = var.scale_up_cooldown
	scale_down_adjustment = var.scale_down_adjustment
	scale_down_cooldown   = var.scale_down_cooldown
}

module "alarms_label" {
	source  = "../modules/null-label"
	name = "alarms"
	context = var.context
}

resource "aws_sns_topic" "sns_topic" {
	name              = module.alarms_label.id
	display_name      = "Foo bar"
	tags              = module.alarms_label.tags
}

module "ecs_cloudwatch_sns_alarms" {
	source  = "cloudposse/ecs-cloudwatch-sns-alarms/aws"
	.....
	cpu_utilization_high_alarm_actions = [
	                aws_sns_topic.cs_sns_topic.arn, 
	                module.ecs_cloudwatch_autoscaling.scale_up_policy_arn
	]
	.....
	cpu_utilization_low_alarm_actions = [
	        aws_sns_topic.cs_sns_topic.arn, 
	        module.ecs_cloudwatch_autoscaling.scale_down_policy_arn
	]
	.......
	memory_utilization_high_alarm_actions = [
	            aws_sns_topic.cs_sns_topic.arn, 
	            module.ecs_cloudwatch_autoscaling.scale_up_policy_arn
	]
	........
	memory_utilization_low_alarm_actions = [
	              aws_sns_topic.cs_sns_topic.arn, 
	              module.ecs_cloudwatch_autoscaling.scale_down_policy_arn
        ]
	....
	context = module.this.context
}

The differences in this above sample and the default example are the cpu_utilization_high_alarm_actions, cpu_utilization_low_alarm_actions, memory_utilization_high_alarm_actions, memory_utilization_low_alarm_actions we need to pass the outputs from autoscaling module (scale_up_policy_arn and scale_down_policy_arn) to the alarm actions we want. This configuration will produce the expected behaviour, something like this:

[nitrocode]

Awesome. Thanks for figuring it out @daltonsena !

Perhaps one of you could put in a PR to update our README.yaml file to include the cloudposse/ecs-cloudwatch-sns-alarms/aws module reference in the usage? Then others will not hit the same issue.

[razorsedge]

When I do this, the ECS UI shows that I only get scaling on CPU. Shouldn't there be a module.ecs_cloudwatch_autoscaling-mem to be the target of memory_utilization_*_alarm_actions?

[MrZablah]

AWS will not allow you to use 2 alerts/metrics to control 1 aws_appautoscaling_policy that means we need 1 aws_appautoscaling_policy for each CPU and Memory alert/metric.

That means that we required 4 aws_appautoscaling_policy to have CPU Up, CPU Down, MEM Up, MEM Down.

This Module however only gives us 2 and since it try to create the aws_appautoscaling_target each time we call the module we can't use this module to create the 4 aws_appautoscaling_policy.

Maybe we should update the module and make the creation of aws_appautoscaling_target optional, athough this module is outdated in the metrics as well so maybe requieres additional changes like the metric_aggregation_type.