IRSA configuration for providers installed via OCI image

[vilkovtato]

What problem are you facing?

I have created Crossplane OCI image and pushed it in the repo. Besides composition/xrd, this is the configuration file:

apiVersion: meta.pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: myconfig
spec:
  crossplane:
    version: ">=v1.13.1"
  dependsOn:
  - provider: xpkg.upbound.io/upbound/provider-aws-s3
    version: ">=v1.3.1"
  - provider: xpkg.upbound.io/upbound/provider-aws-iam
    version: ">=v1.3.1"

This package must be installed into the k8s cluster. In the cluster there are already aws providers which were installed manually (not with OCI image). These providers have specified existing ControllerConfig like this:

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  annotations:
    meta.helm.sh/release-name: aws-provider-f844c283
    meta.helm.sh/release-namespace: crossplane-system
spec:
  controllerConfigRef:
    name: aws-provider-f844c283-provider-family-aws
  package: xpkg.upbound.io/upbound/provider-aws-ec2:v0.42.0

This ControllerConfig looks like this:

apiVersion: pkg.crossplane.io/v1alpha1
kind: ControllerConfig
metadata:
  name: aws-provider-f844c283-provider-family-aws
  annotations:
    eks.amazonaws.com/role-arn: >-
      arn:aws:iam::xxxxxxx:role/k8s-crossplane-provider-aws-role-xxxxx

So the purpose is for providers to get (via their service accounts) annotation with the IAM role. So the providers are able to create AWS resources.

The problem is this - if the providers which are configured in the OCI image configuration dont exists in the k8s cluster yet, they will be automatically installed, but they (their service accounts) will not get the annotation with IAM role. I havent found a way how I can configure controllerConfigRef: in the configuration file (first yaml file).

How could Crossplane help solve your problem?

To summarise:

• we are able to specify ControllerConfig when manually installing Providers (second yaml)
• we are not able to specify ControllerConfig when installing Providers via OCI image (first yaml)

Or is there any other way?

Thank you.

[jbw976]

@vilkovtato is this the same scenario being discussed in #5587 by any chance? a quick glance made it seem they were similar, but I wasn't sure if there were a clear distinction between the two for you 🤔

[vilkovtato]

@jbw976 - yes, I created that question, it is the same scenario. unfortunately that link which was provided did not answer my quesiton. The problem is to automatically configure new Providers installed via package with picking up IRSA configuration...

[jbw976]

This old issue looks to have been asking for something similar with ControllerConfigs, dropping a breadcrumb here: #2367

[jbw976]

@bobh66 relevant comment: #5587 (reply in thread)

[vilkovtato]

This old issue looks to have been asking for something similar with ControllerConfigs, dropping a breadcrumb here: #2367

@jbw976 - thank you for your help. If I understand the above old issue, There is no way, how to configurate newly created Providers in the package configuration. The only option for now is, that after the Provider installation in the cluster we have to manually re-configure the Provider in the cluster. For example:

• there will be already DeploymentRuntimeConfig in the k8s cluster, which is configured with the IRSA

apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: irsa-runtimeconfig
spec:
  serviceAccountTemplate:
    metadata:
      annotations:
        eks.amazonaws.com/role-arn: arn:aws:iam::622346257358:role/my-custom-role

so the newly created Provider(s) must be re-configured via another template applied to the cluster, like this:

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-s3
spec:
  package: xpkg.upbound.io/upbound/provider-aws-s3:v0.37.0
  runtimeConfigRef:
    name: irsa-runtimeconfig

Is that correct?
Thank you

[bobh66]

@vilkovtato yes, that is the current method. The idea is to keep the Configurations generic so that they will work in any cluster, and provision the Providers with the specific needs for each cluster deployment. That being said, I think there is a need to allow a newly installed "family" provider to "find" a DeploymentRuntimeConfig that is probably already being used by most/all of the other Providers in that family. We can't eliminate the need for a DeploymentRuntimeConfig that has the IRSA details in it, but we might be able to allow for a "family default" that can be used if a specific instance is not configured. This is still in the discussion phase.