[Blueprints, 02-at-scale] Set Topology for 1 AZ in Storage Class

[carlosrodlop]

There are two option to prevent from posible node affinity conflict during controllers restarts when using EBS volumens: make topology aware volume to the same AZs, or designing Autoscaling Groups following what is explained in the AWS article Creating Kubernetes Auto Scaling Groups for Multiple Availability Zones (one ASG per AZ for EBS volume and one single ASG per Multiple AZ for EFS volumes). At the moment of publishing this blueprints, terraform-aws-modules/eks/aws does not support availability_zones atribute for the embedded aws_autoscaling_group resource, then the first option is applied in g3 Storage Class.

Only with setting g3 topology Storage Class is not enough. It is required to assigned Pod to 1 AZ only

[carlosrodlop]

Possible modification in the main.tf

locals {
  az_a = ["${local.region}a"]
}

resource "kubernetes_storage_class_v1" "gp3" {
  metadata {
    name = "gp3"

    annotations = {
      "storageclass.kubernetes.io/is-default-class" = "true"
    }
  }

  storage_provisioner    = "ebs.csi.aws.com"
  allow_volume_expansion = true
  reclaim_policy         = "Delete"
  volume_binding_mode    = "WaitForFirstConsumer"

  parameters = {
    encrypted = "true"
    fsType    = "ext4"
    type      = "gp3"
  }

  allowed_topologies {
      match_label_expressions {
         key    = "topology.ebs.csi.aws.com/zone"
         values = local.az_a
     }
 }
}