GCP Workload Identity Federation Webhook

This webhook is for mutating pods that will require GCP Workload Identity Federation access from Kubernetes Cluster.

Note: GKE or Anthos natively support injecting workload identity for pods. This webhook is useful mainly for Kubernetes clusters running in other cloud providers or on-premise.

Prerequisites

Kubernetes cluster v1.21 or later.
Configure kube-apiserver with --service-account-issuer and --service-account-jwks-uri properly.
Expose Your Kubernetes Cluster's ServiceAccount issuer discovery endpoint (OIDC discovery endpoint) to the public so that it can reach by the url set in --service-account-issuer

Hint: you can use S3/GCS to expose the endpoint to the public.
WARNING: If your public JWKS(JSON Web Key Set) endpoint are compromised by the malicious attacker, the attacker can hijack your JWKS endpoint (i.e., issue OIDC ID Tokens) that can impersonate any roles for GCP service accounts that are configured to trust to the issuer. Thus, JWKS endpoint must be secured all the time.

Walk Through

Create an external identity pool and provider with your OIDC issuer and allowed audience (default: sts.googleapis.com) in IAM for your project. It will need to configure attribute mappings and conditions from OIDC ID Tokens to identities in the pool.
Granting external identities permission to impersonate a service account so that a Kubernetes ServiceAccount can work as a federated workload identity (i.e., it can impersonate a GCP service account).

Annotate a Kubernetes ServiceAccount with the identity provider and GCP service account that it will impersonate.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-x
  namespace: service-a
  annotations:
    # assume
    #   you grant k8s service account "service-a/app-x" to impersonate "app-x" GCP service account
    #   this k8s cluster's service account belongs the annotated workload identity provider here
    cloud.google.com/workload-identity-provider: "projects/12345/locations/global/workloadIdentityPools/on-prem-kubernetes/providers/this-cluster"
    cloud.google.com/service-account-email: "app-x@project.iam.googleapis.com"

    # optional: Defaults to "sts.googleapis.com" if not set
    #   this value must be allowed in the annotated workload identity provider above
    cloud.google.com/audience: "sts.googleapis.com"

    # optional: Defaults to 86400 for expirationSeconds if not set
    #   Note: This value can be overwritten if specified in the pod
    #         annotation as shown in the next step.
    cloud.google.com/token-expiration: "86400"

    # optional: This value defines the container security context with runAsUser
    #           with the defined user. This could avoid problems related with root requirement from gcloud image
    cloud.google.com/gcloud-run-as-user: "1000"

    # optional: gcloud external configuration injection mode.
    #           The value must be one of 'gcloud'(default) or 'direct'.
    #           Refer to the next section for 'direct' injection mode
    cloud.google.com/injection-mode: "gcloud"

All new pods launched using the Kubernetes ServiceAccount will be mutated so that they can impersonate the GCP service account. Below is an example pod spec with the environment variables and volume fields mutated by the webhook.

apiVersion: v1
kind: Pod
metadata:
  name: app-x-pod
  namespace: service-a
annotations:
  # optional: A comma-separated list of initContainers and container names
  #   to skip adding volumeMounts and environment variables
  cloud.google.com/skip-containers: "init-first,sidecar"
  # optional: Defaults to 86400, or value specified in ServiceAccount
  #   annotation as shown in previous step, for expirationSeconds if not set
  cloud.google.com/token-expiration: "86400"
spec:
  serviceAccountName: app-x
  initContainers:
    ### gcloud-setup init container is injected by the webhook ###
  - name: gcloud-setup
    image: google/cloud-sdk:slim
    command:
    - sh
    - -c
    - |
      gcloud iam workload-identity-pools create-cred-config \
          $(GCP_WORKLOAD_IDENTITY_PROVIDER) \
          --service-account=$(GCP_SERVICE_ACCOUNT) \
          --output-file=$(CLOUDSDK_CONFIG)/federation.json \
          --credential-source-file=/var/run/secrets/sts.googleapis.com/serviceaccount/token
          gcloud auth login --cred-file=$(CLOUDSDK_CONFIG)/federation.json
    env:
    - name: GCP_WORKLOAD_IDENTITY_PROVIDER
      value: "projects/12345/locations/global/workloadIdentityPools/on-prem-kubernetes/providers/this-cluster"
    - name: GCP_SERVICE_ACCOUNT
      value: app-x@project.iam.gserviceaccount.com
    - name: CLOUDSDK_CONFIG
      value: /var/run/secrets/gcloud/config
    volumeMounts:
    - name: gcp-iam-token
      readOnly: true
      mountPath: /var/run/secrets/sts.googleapis.com/serviceaccount
    - name: gcloud-config
      mountPath: /var/run/secrets/gcloud/config
  - name: init-first
    image: container-image:version
  containers:
  - name: sidecar
    image: container-image:version
  - name: container-name
    image: container-image:version
    ### Everything below is added by the webhook ###
    env:
    - name: GOOGLE_APPLICATION_CREDENTIALS
      value: /var/run/secrets/gcloud/config/federation.json
    - name: CLOUDSDK_CONFIG
      value: /var/run/secrets/gcloud/config
    - name: CLOUDSDK_COMPUTE_REGION
      value: asia-northeast1
    volumeMounts:
    - name: gcp-iam-token
      readOnly: true
      mountPath: /var/run/secrets/sts.googleapis.com/serviceaccount
    - name: gcloud-config
      mountPath: /var/run/secrets/gcloud/config
  volumes:
  - name: gcp-iam-token
    projected:
      sources:
      - serviceAccountToken:
          audience: sts.googleapis.com
          expirationSeconds: 86400
          path: token
  - name: gcloud-config
    emptyDir: {}

Usage with non-root container user

When running a container with a non-root user, you need to give user id for GCloud SDK container using the annotation cloud.google.com/gcloud-run-as-user in the service account.

Experimental Direct Credential Injection Mode

In this mode, the Workload Identity Federation Webhook controller directly generates the Gcloud external credentials configuration and injects into the pod. This means the gcloud-setup init container is not required which can speed up pod start time.

To use direct injection mode:

Annotate a Kubernetes ServiceAccount with the injection-mode value of direct.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-x
  namespace: service-a
  annotations:

    # Set the injection mode to 'direct', instead of 'gcloud'.
    cloud.google.com/injection-mode: "direct"

Below is an example pod spec with the environment variables and volume fields mutated by the webhook. Notice there is no gcloud-setup init container or Volumes, instead there is an extra annotation and external-credential-config volume and volumeMount.

apiVersion: v1
kind: Pod
metadata:
  name: app-x-pod
  namespace: service-a
annotations:
  # optional: A comma-separated list of initContainers and container names
  #   to skip adding volumeMounts and environment variables
  cloud.google.com/skip-containers: "init-first,sidecar"
  #
  # The Generated External Credentials Json is added as an annotation, and mounted into the container filesystem via the DownwardAPI Volume
  #
  cloud.google.com/external-credentials-json: |-
    {
      "type": "external_account",
      "audience": "//iam.googleapis.com/projects/123456789/locations/global/workloadIdentityPools/on-prem-kubernetes/providers/this-cluster",
      "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
      "token_url": "https://sts.googleapis.com/v1/token",
      "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/app-x@project.iam.gserviceaccount.com:generateAccessToken",
      "credential_source": {
        "file": "/var/run/secrets/sts.googleapis.com/serviceaccount/token",
        "format": {
          "type": "text"
        }
      }
    }
spec:
  serviceAccountName: app-x
  initContainers:
  - name: init-first
    image: container-image:version
  containers:
  - name: sidecar
    image: container-image:version
  - name: container-name
    image: container-image:version
    ### Everything below is added by the webhook ###
    env:
    - name: GOOGLE_APPLICATION_CREDENTIALS
      value: /var/run/secrets/gcloud/config/federation.json
    - name: CLOUDSDK_COMPUTE_REGION
      value: asia-northeast1
    volumeMounts:
    - name: gcp-iam-token
      readOnly: true
      mountPath: /var/run/secrets/sts.googleapis.com/serviceaccount
    - mountPath: /var/run/secrets/gcloud/config
      name: external-credential-config
      readOnly: true
  volumes:
  - name: gcp-iam-token
    projected:
      sources:
      - serviceAccountToken:
          audience: sts.googleapis.com
          expirationSeconds: 86400
          path: token
  - downwardAPI:
      defaultMode: 288
      items:
      - fieldRef:
          apiVersion: v1
          fieldPath: metadata.annotations['cloud.google.com/external-credentials-json']
        path: federation.json
    name: external-credential-config

Usage

Usage of /gcp-workload-identity-federation-webhook:
  -annotation-prefix string
        The Service Account annotation to look for (default "cloud.google.com")
  -gcloud-image string
        Container image for the init container setting up GCloud SDK (default "google/cloud-sdk:slim")
  -gcp-default-region string
        If set, CLOUDSDK_COMPUTE_REGION will be set to this value in mutated containers
  -health-probe-bind-address string
        The address the probe endpoint binds to. (default ":8081")
  -kubeconfig string
        Paths to a kubeconfig. Only required if out-of-cluster.
  -metrics-bind-address string
        The address the metric endpoint binds to. (default ":8080")
  -setup-container-resources string
        Resource spec in json for the init container setting up GCloud SDK, e.g. '{"requests":{"cpu":"100m"}}'
  -token-audience string
        The default audience for tokens. Can be overridden by annotation (default "sts.googleapis.com")
  -token-expiration duration
        The token expiration (default 24h0m0s)
  -token-default-mode int
        DefaultMode for the token volume (default 0440)
  -zap-devel
        Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error) (default true)
  -zap-encoder value
        Zap log encoding (one of 'json' or 'console')
  -zap-log-level value
        Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
  -zap-stacktrace-level value
        Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').
  -zap-time-encoding value
        Zap time encoding (one of 'epoch', 'millis', 'nano', 'iso8601', 'rfc3339' or 'rfc3339nano'). Defaults to 'epoch'.

Installation

Pre-requisites

cert-manager: See cert-manager installation
(optional) prometheus-operator: See https://github.com/prometheus-operator/prometheus-operator

Deploy

Helm chart

$ helm repo add gcp-workload-identity-federation-webhook https://pfnet-research.github.io/gcp-workload-identity-federation-webhook
$ helm repo update
$ helm install gcp-wif-webhook gcp-workload-identity-federation-webhook/gcp-workload-identity-federation-webhook \
    --namespace gcp-wif-webhook-system --create-namespace

Kustomize

make deploy

Or, please inspect config/default directory.

Release

The release process is automated by tagpr. To release, just merge the latest release PR.

License

Acknowledgment

This project is greatly inspired by aws/amazon-eks-pod-identity-webhook.

Name		Name	Last commit message	Last commit date
Latest commit History 37 Commits
.github		.github
charts/gcp-workload-identity-federation-webhook		charts/gcp-workload-identity-federation-webhook
config		config
hack		hack
webhooks		webhooks
.dockerignore		.dockerignore
.gitignore		.gitignore
.tagpr		.tagpr
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
PROJECT		PROJECT
README.md		README.md
ct.yaml		ct.yaml
go.mod		go.mod
go.sum		go.sum
main.go		main.go

License

pfnet-research/gcp-workload-identity-federation-webhook

Folders and files

Latest commit

History

Repository files navigation

GCP Workload Identity Federation Webhook

Prerequisites

Walk Through

Usage with non-root container user

Experimental Direct Credential Injection Mode

Usage

Installation

Pre-requisites

Deploy

Helm chart

Kustomize

Release

License

Acknowledgment

About

Topics

Resources

License

Stars

Watchers

Forks

Languages