v0.6.7

IMPORTANT:

!!Do not use this release!! There is a critical bug in this release that causes cluster deletion failures, which is solved in v0.6.8. This bug does not impact any other release.

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

• Action required
  Controllers policy updated with missing KMS permissions required to use EKS encryption, if you are planning to use EKS encryption then you will need to update your controllers policy by running clusterawsadm bootstrap iam create-cloudformation-stack again. And then when you create a cluster with encryption enabled you will need to use a KMS key that has an alias name starting with cluster-api-provider-aws-. For further information see docs. (#2448, @richardcase)

Changes by Kind

Feature

• Taints supported on EKS node groups created via AWSManagedMachinePool (#2450, @richardcase)

Bug or Regression

• AWSMachinePool controller removes one old LaunchTemplate version before creating a new version, preventing the number of versions from growing without bound, and reaching the maximum limit. (#2531, @dlipovetsky)
• Fix for ELB deletion when there are more than 20 ELBs. (#2512, @faiq)
• Fix for only deleting Security Groups managed by CAPA controllers, not the overridden ones. (#2560, @sedefsavas)
• Patch VPC ID immediately after VPC creation, to deal with edge case where multiple VPCs may get created with the same tags. (#2587, @sedefsavas)
• Fix for reconciling LaunchTemplates. (#2410, @dkoshkin)
• Fix typo in AWSFargateProfile validation webhook which cause the webhook not called. (#2446, @jzhoucliqr)

Other (Cleanup or Flake)

• Updated dependencies (#2486, @randomvariable)

See CHANGELOG.md for dependency updates.

v0.6.6

Release notes for Cluster API Provider AWS (CAPA) v0.6.6

Documentation

Changelog since v0.6.5

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

!Important Release Note !(This node is added after the release)
A new Launch Template Version gets created almost at each reconcile and since there is a 10,000 versions per launch template limitation, AWSMachinePools stop working after 2 months and needs to be recreated (#2368). This will be fixed in the next release.

!Action required!
Controllers policy updated with missing key pairs permission, if you are using or plan to use AWSManagedMachinePool with an SSH key then you will need to update your controllers policy by running clusterawsadm bootstrap iam create-cloudformation-stack again. (#2408, @richardcase)

Changes by Kind

Bug or Regression

• Update EKS Nodepool min/max to match the AWSManagedMachinePool spec, overriding changes to min/max made via the AWS Console, CLI, or SDK (#2407, @richardcase)
• Fix for reconciling LaunchTemplates after "clusterctl move"(#2394, @sedefsavas) (#2410, @dkoshkin)
• Fix specifying no SSH key for machine pool launch templates (#2362, @jimmidyson)
• Update LaunchTemplate with a valid bootstrap token after ASG scale (#2401, @dlipovetsky)
• Add identity ref support for fargate controller (#2406, @jzhoucliqr)

The images for this release are:
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.6
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.6
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.6

Thanks to all our contributors.

v0.6.5

Release notes for Cluster API Provider AWS (CAPA) v0.6.5

Documentation

Changelog since v0.6.4

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

• The behaviour when selecting which subnets to use with machine pools (AWSMachinePool & AWSManagedMachinePool) when subnet ids haven't been included has changed. There is now a defined order of precedence that will determine which subnets to use:
  1. Subnets defined explicitly in the spec of AWSMachinePool/AWSManagedMachinePool
  2. If AvailabilityZones is specified on AWSMachinePool/AWSManagedMachinePool then the subnets associated with those AZs will be used
  3. If failureDomains are specified in the MachinePool then subnets that are in those failureDomains (a.k.a. Availability Zones) will be used
  4. All the private subnets from the control plane are used (#2302, @richardcase)
• EKS: New AWSManagedMachinePool resources with non-empty remoteAccess now require remoteAccess.public: true in order to allow public access to SSH on port 22 (#2243, @michaelbeaumont)

Changes by Kind

API Change

• Add the following cluster-scoped resources for multi-tenancy support:
  • AWSClusterStaticIdentity - Static credentials using a Access Key ID and Secret Key
  • AWSClusterControllerIdentity - A singleton resource that states a cluster can use inherited credentials
  • AWSClusterRoleIdentity - An IAM role definition (#2253)
• Add ability to toggle the new AWS Capacity Rebalance feature by setting a new .spec.capacityRebalance field in AWSMachinePool objects. (#2288, @trutx)

Feature

• Add Multi-tenancy support (multi-tenancy proposal) (#2253)
• For migration of current cluster resources to the new multi-tenancy model, there is a new experimental controller
  AutoControllerIdentityCreator that will create and apply AWSClusterControllerIdentity to all existing resources. This will be enabled by default until v1alpha4.
  No additional steps are needed to migrate existing clusters, multi-tenancy model is fully backward-compatible when this controller is kept enabled (#2253)
• Add clusterawsadm ami list command to list AMIs that can be filtered by region, OS, and Kubernetes version. (#2304, @sedefsavas)
• clusterawsadm ami commands now support --source-region to copy AMIs across regions (#2345)
• clusterawsadm ami commands now output versioned AWSAMIList and AWSAMI resources to stdout (#2345, @randomvariable)
• Add the ability to enable the AWS SDK debug logging (#2229, @shuheiktgw)
• Adopt the release-notes tool from kubernetes/release to generate the changelog for a release (#2247, @richardcase)
• PRs now require release-notes code block, which is used in the release notes generation (#2232, @richardcase)
• EKS: Ability to declaratively remove the Amazon VPC CNI when using an alternate CNI (#2292, @richardcase)
• EKS: Add the AWSFargateProfile resource for managing EKS Fargate profiles (#2265, @michaelbeaumont)
• EKS: Add/update conditions for the AWSManagedControlPlane to detect when EKS control plane is being created or updated. (#2246, @michaelbeaumont)
• EKS: Add new cluster template for a GPU-accelerated EKS cluster (#2278, @richardcase)

Documentation

• Add documentation for IAM permissions and clusterawsadm, dynamically generated via clusterawsadm itself (#2342, @randomvariable)
• Add Published AMIs page that gets updated by a lambda function every hour (#2345, @randomvariable)
• Add auto-generated CRD reference documentation for core APIs, EKS controlplane and experimental features (#2347) (#2352, @randomvariable)
• Add multitenancy documentation with examples (#2319, @sedefsavas)
• EKS: Add a guide on how to develop EKS Control Plane locally using Tilt (#2234, @kenichi-shibata)
• EKS: New ADR to document the decision of how Fargate Profiles will be represented. (#2250, @michaelbeaumont)

Failing Test

• Enable EventBridgeInstanceState feature in e2e tests (#2293, @sedefsavas)

Bug or Regression

• AWSMachine: Add filters support for additional security groups (#2241, @alexander-demichev)
• Fix AWSCluster & AWSMachine validation webhooks to accept an empty string for the SSHKeyName field (#2308, @dlipovetsky)
• Fix bug where custom bootstrap user name was not accepted (#2341, @randomvariable)
• Restore GovCloud and other AWS partition support for CloudFormation generation. If using EKS, you must provide the relevant value for partition in your clusterawsadm configuration file. (#2289, @randomvariable)

Other (Cleanup or Flake)

• Add validation for loadbalancer scheme to allow only Internet-facing and internal values (#2290, @sedefsavas)
• Add test coverage to test grid (#2350, @sedefsavas)
• Add upgrade to Kubernetes main test (#2313, @sedefsavas)
• EKS: Removal of AWSManagedCluster from templates/docs to help with the future deprecation in v1alpha4 (#2264, @richardcase)

Support

• @sedefsavas joined to the maintainers of Cluster API Provider AWS (#2279, @richardcase)

The images for this release is:
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.5
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.5
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.5

Thanks to all our contributors!

Special thanks to @detiber for his contributions to CAPA from its inception who moves to emeritus status now.

v0.6.4

Changes since v0.6.3

✨ New Features

• Add fine-grained service rate limiters (#2201)
• Add list of instances to AWSMachinePool status (#2197)
• Add support for AWS_CONTROLLER_IAM_ROLE environment variable using Kiam or IRSA (#2100)
• clusterawsadm: Allow to take a configuration file to print IAM policy documents (#2147)
• clusterawsadm: Add ability to copy AMIs to a target account with encryption (#2112, #2203)
• EKS: addon support (#2202)
• EKS: Support custom AMI lookup (#2057)
• EKS: Secondary cidr support for workload clusters (#2086)
• EKS: Add provisioning of IRSA to workload clusters (#2070)

🐛 Bug Fixes

• Fix OpenAPI defaulting for secrets backend and add OpenAPI testing (#2135)
• Updated service account for leader election (#2183)
• AWSCluster: Error if a loadbalancer exist with the same name in the same region with a different scheme (#2154)
• AWSCluster: Fix NPE when comparing load balancers (#2163)
• AWSMachinePool: Trigger rolling replacement upon launch template change (#2193)
• AWSMachinePool: Remove unused ID field on launch template spec (#2184)
• EKS: Set Subnets as an optional property (#2140)
• clusterawsadm: Add UpdateAutoScalingGroup to controller IAM (#2194)

📖 Documentation

• EKS console documentation (#2187)
• Fix broken links and emojis (#2159, #2150)
• Add config example to docs specifying IAM role (#2151)
• Updated wording on prerequisites doc (#2149)
• Update consuming-existing-aws-infrastructure.md to provide context to where networkSpec belongs (#2103)
• ADR: EKS packaging (#2126)
• ADR: e2e test structure (#2127)
• AMI Updates (#2164, #2190, #2206)

🌱 Others

• EKS: e2e tests added (#2168, #2220, #2199, #2188, #2211, #2214)
• EKS: deletion checks (#2175)
• AWSCluster: CAPI E2E tests (#2138)
• Consolidating boskos scripts for CAPA (#2101)
• Upgrade AWS SDK version to 1.36.26 (#2204)
• Add Interruptible field to AWSMachine status (#2120)
• Upgrade CAPI version to v0.3.12 (#2129, #2133, #2198)
• Refactor image build and release process (#2213, #2215, #2216, #2217)
• Enable use of shared configuration file in clusterawsadm (#2077)
• Update CAPA maintainer and reviewers (#2031, #2161)
• Refactor release process for Github and Staging (#2096)

The images for this release is:
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.4
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.4
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.4

Thanks to all our contributors!

Special thanks to @ncdc for his contributions to Cluster API from its inception who moves to emeritus status.

Shoutout to @richardcase for joining the maintainers of Cluster API Provider AWS, as well as @michaelbeaumont, @dthorsen, and @sedefsavas joining the project reviewers.

😊

v0.6.3

Changes since v0.6.2

🐛 Bug Fixes

• awsmachinetemplates: Allow cloudInit.secureSecretsBackend to be defaulted (#2111)
• fix ASG event message (#2108)

🌱 Others

• Upgrade e2e Kubernetes version to 1.19.4 (#2114)
• Remove old terraform scripts (#2113)
• Build new AMIs (#2109)
• refactor: e2e test reorganisation (#2102)

The images for this release are:
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.3
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.3
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.3

Thanks to all our contributors! 😊

v0.6.2

Changes since v0.6.1

🐛 Bug Fixes

• Validate that additional security groups can not have filters (#2072)
• AWSManagedMachinePool: Fix cloud provider key usage and nodegroup IAM role name (#2085)
• Fallback to cluster subnets on update ASG (#2095)
• Fix ownerref group on kube secret and configmap (#2092)
• Add CNI defaulting to the controller (#2090)
• Update in-controller AWSMachine CloudInit defaulting logic (#2082)
• Add fallback if subnets not provided on AWSMachinePool (#2051)
• Make the regex less restrictive to allow underscore etc. for ssh key file name (#2071)
• Add required iam permission for managed node groups (#2043)
• Ensure env var enables AWSMachinePool webhooks (#2046)
• Add more conditions for resource status filtering (#2049)
• Validate subnet's AZ with failure domain when subnet id is passed (#2011)

📖 Documentation

• Update docs to use capa-system namespace (#2094)
• Add ADR template and first ADR (#2080)
• Updated flavours and docs for machine pools (#2030, #2044)
• Fix typos in EKS commands (#2038)

🌱 Others

• Remove unused allow additional roles feature flag related code (#2035)
• Updated Bastion node's AMIs. Changed the OS version to Ubuntu 20.04 from Ubuntu 16.04 (#2068)
• Makefile: Ensure manifests compile during verify (#2098)
• Fix roundtrip conversions between v1alpha2 and v1alpha3 (#2074)

The images for this release are:

• us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.2
• us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.2
• us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.2

Thanks to all our contributors! 😊

v0.6.1

Changes since v0.6.0

⚠️ Breaking Changes

• EKS Control Plane Provider (aws-eks) has been introduced (#1949, #1997, #1939, #1973, #1966, #1960, #1943)
  As part of this work the infrastructure manager has been refactored to remove the EKS control plane functionality. This means the new provider along with the existing EKS bootstrap provider (which has been renamed aws-eks) will need to be used if you want to create EKS based clusters with Cluster API Provider AWS. For example:

export EXP_EKS=true
clusterctl --infrastructure=aws --control-plane=aws-eks --bootstrap=aws-eks

✨ New Features

• Allow AWS Systems Manager Parameter Store to be used as a secrets backend for userdata for regions where AWS Secrets Manager is not available (#1924)
• Add a tag to instances during creation that matches the Cluster API machine name (#2015)
• Additional user & role mappings for aws-iam-authenticator are now configurable (#1995, #2002, #1938)
• AWSMachinePools support for EC2 autoscaling groups (#1860, #2010, #2014, #2007, #1863, #2021, #2006, #2000)
• AWSManagedMachinePool (EKS managed nodegroup) support (#1916, #2024, #2013)
• Allow specifying subnet IDs to be used for control plane load balancers (#1931)
• Add ability to specify instance tenancy, i.e. dedicated EC2 instances (#1926)
• Add conditions for the reconcile delete workflow (#1905)
• Cluster API Provider AWS can be configured to use custom endpoints for connecting to AWS services (#1858)
• Improved validation around fields that are passed in as AWS API parameters (#1978)

🐛 Bug Fixes

• Persist subnet changes into the AWSCluster spec early when using default subnets (#1915)
• Allow the usage of unmanaged VPCs without public subnets (#1884)
• Fix NPE when deleting security groups on cluster deletion (#1996)
• Add externalManagedControlPlane Status to allow node drains under EKS (#1992, #1994)
• Fix for allowing nodes to join the EKS cluster (#1962)
• Fix bastion reconcilation and connection error when using eks flavor (#1957)
• Retry with listing all ELBs when listing by tag fails, fixing an issue in environments where the ResourceTagging API is not available (#1952)
• AWSMachine ssh key should defer to that configured on the AWSCluster resource when nil (#1932)
• Add capi exp schema and fix manager args and rbac (#1936)
• Fix volume description to remove 'root' since it's also used for non-root volume (#2005)
• Fix incorrect capitalization for eks field (#1998)

📖 Documentation

• Cluster API Provider AWS has a new website: cluster-api-aws.sigs.k8s.io (#1947, #2017, #2028, #1981)
• Add docs on updating AWS credentials used by Cluster API Provider AWS (#1948)

🌱 Others

• Remove unused integration test on pull requests (#2012)
• e2e: Add test for spot instances (#1963)
• e2e: Verify code compiles on pull requests (#1953)
• e2e: Install CNI using ClusterResourceSet (#1816)
• unit tests: instances - Sort tag keys so unit tests can succeed (#1937)
• Show more helpful error message when duplicate clusters are created across namespaces, resulting in duplicate VPCs (#1880)
• Golang version updated to 1.13.15 (#1944)
• Controller runtime updated to 0.5.11 (#1950)

The images for this release is:

Core AWS Controller: us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.1
EKS Bootstrap Controller: us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.1
EKS ControlPlane Controller: us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.1

Thanks to all our contributors! 😊

v0.6.0

Changes since v0.5.5

✨ New Features

• Support for using custom AMIs for the bastion instance (#1864)

• Support for using io2 volumes (#1889)

• Add support for multiple block device mappings (#1878)

• Support for using spot instances (#1868, #1873)

• Initial support for Amazon EKS (#1815, #1724, #1879, #1856, #1913, #1910, #1909, #1848, #1914, #1853)
  This version adds the first implementation of Cluster API Provider AWS for use with Amazon EKS.
  To be able to install the required bootstrap provider with the clusterctl release from v0.3.9, add (or create) the following to ~/.cluster-api/clusterctl.yaml:

  providers:
    - name: "eks"
      url: "https://github.com/kubernetes-sigs/cluster-api-provider-aws/releases/latest/eks-bootstrap-components.yaml"
      type: "BootstrapProvider"

  Further instructions on using Cluster API Provider AWS with Amazon EKS can be found in https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/v0.6.0/docs/eks.md

🐛 Bug Fixes

• Fix clusterawsadm policy mix up (#1922)
• Fix bug with orphaned route tables by tagging on creation (#1920)
• Fix for Bastion struct field comment (#1901)
• Cleanup obsolete v1alpha2 annotation on Machine (#1881)

📖 Documentation

• Update prerequisites doc regarding bootstrap user (#1917)
• Update AMIs for latest Kubernetes patch releases (#1903)

Testing/Other

• Update e2e tests to Kubernetes v1.19 (#1904)
• Only use clusterctl-based e2e and conformance tests (#1890)
• Add metadata.yaml for defining contracts for clusterctl (#1883)
• Remove currently unsupported local zones from consideration as availability zones (#1865)

The image(s) for this release are:

• us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.0
• us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.0

v0.5.5

Changes since v0.5.4

✨ New Features

• Bastion instance type is selectable, and now defaults to t3.micro except us-east-1 (t2.micro) (#1831)
• Allow for the use filters to select subnet for machines (#1833)
• Capture metrics for every AWS API request, available on /metrics (#1807)
• Support for conditions on resources managed by Cluster API Provider AWS (#1777, #1740, #1834)
• Allow user defined ingress rules for CNI (#1747)
• Allow configuration of Bastion host's ingress rules with cidrBlocks field (#1761)
• Changes to the default subnetting logic, now supporting multi-AZ control planes (#1721)
• Parallel NAT Gateway creation, speeding up cluster creation (#1805)
• Allows configuring crossZoneLoadBalancing on controlPlaneLoadBalancer (#1801)
• Create a bastion host if it's enabled in the spec, regardless of managed or unmanaged VPC (#1759)
• clusterawsadm adds programmatic and YAML based API configuration, with new command set and help (#1734, #1820)

Please note that the new clusterawsadm bootstrap iam create-cloudformation-stack command does not create bootstrapper.cluster-api-provider-aws.sigs.k8s.io IAM user by default. If you rely on this functionality, use the old clusterawsadm alpha bootstrap create-stack command or specify bootstrapUser.Enable = true in the configuration file, e.g.

apiVersion: bootstrap.aws.infrastructure.cluster.x-k8s.io/v1alpha1
kind: AWSIAMConfiguration
spec:
  bootstrapUser:
    enable: true

🐛 Bug Fixes

• Workaround for CloudInit bug https://bugs.launchpad.net/cloud-init/+bug/1888822 (#1859)
• Fixes cluster creation with names starting with 'sg-' (#1802)
• Fix issue with NAT gateway failing due to EIP already been assigned (#1803)
• clusterawsadm: Ensure additional role policies work (#1763)
• Delete bastion regardless of VPC type (#1774)
• Preserve all VPCSpec fields between reconciliations (#1767)
• Ignore "not found" errors when deregistering ELB instances (#1758, #1737, #1788)

🌱 Others

• Refactoring for future EKS support (#1810)
• Migrate rest of e2e tests to use clusterctl framework (#1798, #1760, #1797, #1789, #1845, #1835, #1823, #1749)
• Makefile: Build clean binaries on release (#1790)
• Update to cluster-api v0.3.7 (#1825)
• Linter improvements (#1836, #1854, #1819, #1809, #1794)
• Update to AWS SDK for Go 1.33.3 (#1783)
• Add Seth Pellegrino to emeritus reviewers (#1781, #1775)
• Development documentation updates (#1766, #1793)

Thanks to all our contributors! 😊

Special note of thanks to Seth Pellegrino for his many contributions, who moves to emeritus
status (#1781)

v0.5.5-rc.0

Changes since v0.5.5-alpha.0

✨ New Features

• Update to cluster-api v0.3.7 (#1825)
• refactor: cluster scope and service refactor for future EKS support (#1810)
• controllers: Capture metrics for every AWS API request (#1807)
• allow configuration of Bastion host's ingress rules with cidrBlocks field (#1761)
• Creates NAT GWs in parallel (#1805)
• Allows configuring crossZoneLoadBalancing on controlPlaneLoadBalancer (#1801)
• Update to AWS SDK for Go 1.33.3 (#1783)

🐛 Bug Fixes

• Fixes cluster creation with names starting with 'sg-' (#1802)
• Fix issue with NAT gateway failing due to EIP already been assigned (#1803)
• Restricts AZ limit to 1 in e2e tests (#1789)
• clusterawsadm: Add bootstrap user back to deprecated alpha command (#1820)

The image for this release is: us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.5.5-rc.0.

Thanks to all our contributors! 😊