IAM Role for ServiceAccounts

[davisford]

Hi, I'm migrating one of my cluster deployments from CloudFormation to eksctl. One of the things I have setup in CloudFormation is IAM Roles for Service Accounts

In my cluster, however, we deploy a separate namespace for each team member and they have a full-stack for development. Thus, it might look like:

➜ kc get pod -A
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
bob           api-pod-7fd55b8c9f-d66qc                  1/1     Running   0          4h10m
sue           api-pod-84fdd79d5b-62tn6                  1/1     Running   0          4h10m

Let's say the api-pod needs ses or s3 permissions, so I did that in CloudFormation like:

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  Roles / Policies necessary for setting up Pod security 
Parameters:
  OIDCEndpointID:
    Type: String
    Description: Should be the string of numbers at the end of the output of the command `aws eks describe-cluster --name <cluster-name> --query "cluster.identity.oidc.issuer" --output text`
    Default: REDACTED
Resources:
  PosPodRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: !Sub "${AWS::Region}-PosPodRole"
      AssumeRolePolicyDocument:
                            Fn::Sub: 
                              - |
                                {
                                  "Version": "2012-10-17",
                                  "Statement": [
                                    {
                                      "Effect": "Allow",
                                      "Principal": {
                                        "Federated": "${IamOidcProviderArn}"
                                      },
                                      "Action": "sts:AssumeRoleWithWebIdentity",
                                      "Condition": {
                                        "ForAnyValue:StringEquals": {
                                          "${OidcProviderEndpoint}": [
                                            "system:serviceaccount:bob:api-pod-role",
                                            "system:serviceaccount:sue:api-pod-role"
                                          ]
                                        }
                                      }
                                    }
                                  ]
                                }
                              - {
                                  "IamOidcProviderArn": !Sub "arn:aws:iam::${AWS::AccountId}:oidc-provider/oidc.eks.${AWS::Region}.amazonaws.com/id/${OIDCEndpointID}",
                                  "OidcProviderEndpoint": !Sub "oidc.eks.${AWS::Region}.amazonaws.com/id/${OIDCEndpointID}:sub"
                                }
      Path: /
  EmailSender:
    Type: AWS::IAM::Policy
    Properties: 
      PolicyName: EmailSender
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: "ses:SendEmail"
            Resource: "*"
            Condition:
              StringLike:
                "ses:FromAddress": "*@mydomain.com"
          - Effect: Allow
            Action:
              - "s3:GetObject"
              - "s3:ListBucket"
            Resource:
              - "arn:aws:s3:::my-special-bucket"
              - "arn:aws:s3:::my-special-bucket/*"
      Roles:
        - !Ref ApiPodRole
Outputs:
  RoleARN:
    Value: !Ref ApiPodRole

However, looking at the eksctl example for IAM service roles you can either attach a policy or define one inline for a particular namespace.

I don't see a proper way to do this other than to pre-create the policy with another tool (e.g. aws cli, terraform, etc.) and then create several entries in the eksctl yaml like this:

  - metadata:
      name: bob-api
      namespace: bob
    attachRoleARN: arn:aws:iam::123:role/already-created-role-for-app
  - metadata:
      name: sue-api
      namespace: sue
    attachRoleARN: arn:aws:iam::123:role/already-created-role-for-app

What would be ideal is if I could create the standalone IAM role in the eksctl YAML, and then reference it per each namespace like I can do in CloudFormation. I know eksctl generates CloudFormation templates.

Is there a better way to accomplish what I am trying to do?

The other thing I'm trying to understand is how to update it if a new team member joins and I want to dynamically add their namespace. In CloudFormation this was an easy update operatorion. It's my understanding that you cannot modify a node group after it has been created with eksctl. You must create a new one and drain the old one.

Thus, if I gain a new team member and want to update the service account for them, it isn't easily done....?

[davisford]

I was able to resolve this. I did not realize you could create additional iam service accounts on the CLI, and also sync to the eksctl yaml file. The only thing I had to adjust was not creating the same service account name myself in the flux resources since it gets auto-generated.

I also found out if it gets deleted in the cluster, you have to delete it with eksctl (which deletes the cloud formation template), then you can re-create it.