Reference secret values from Kubernetes secrets

[dominic-p]

Clear and concise description of the problem

It would be nice to be able to leverage Kubernetes native Secrets inside of Woodpecker pipelines.

Suggested solution

I'm imagining a workflow definition that looks like this:

steps:
  - name: docker
    image: docker
    commands:
      - echo $docker_username
      - echo $DOCKER_PASSWORD
    secrets:
    - docker_username
    - name: DOCKER_PASSWORD
      kubernetesRef:
        name: my-docker-secret
        key: password

So, the docker_username secret would be referenced as it normally is today. The value for the DOCKER_PASSWORD secret would come from a Kubernetes Secret called my-docker-secret that is defined in the same namespace that the pipeline runs in. That secret is expected to have a key named password that contains the value that we want to use for the secret.

This is loosely based on the way Drone does it and the way that Kubernetes allows you to reference secrets for env variables.

The exact syntax is, of course, flexible, but the idea is to allow for other external secret providers as well (see #929).

Alternative

Right now, it seems like the only way to define secrets is via the UI or the CLI. This works, but it isn't ideal if you already have the secrets defined in your cluster.

Another alternative would be to store the information in global env variables, but that doesn't seem ideal from a security standpoint.

Another option would be to expand the Kubernetes backend volume mounting logic to allow mounting Secrets in addition to PVs.

Additional context

This is part of my ongoing effort to get a container build system (specifically buildah) to run on Woodpecker on a CRI-O based Kubernetes cluster. I currently have it up and working, but I need a way to pass registry credentials to it. Otherwise, I can build images, but I can't push them to my repo when I'm done.

There may be a simpler way to do this that I'm just missing at the moment.

Validations

• Checked that the feature isn't part of the next version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[everflux]

An alternative would be to have all implementation specific parts in 'backend_options' - 'kubernetes'

[dominic-p]

Yes, that makes sense as well. We could have an implementation like this:

steps:
  - name: test
    image: alpine
    commands:
      - echo $my_password
    backend_options:
      kubernetes:
        secrets:
          - name: my_password
            secretRef: my-top-secret-password
            key: password

[zc-devs]

As I understand FR:
As Admin I want to let users to use (refer to) predefined (existing) Kubernetes secrets in their pipelines.
Am I right?

There might be security concerns... However, if we run workload in separate namespace, then probably OK. Feature can be gated by Agent's flag.

---

store the information in global env variables, but that doesn't seem ideal from a security standpoint

There are global and org secrets.

I can't push them to my repo when I'm done.

You can use WP secrets :)

allow for other external secret providers as well (see #929)

This is another thing. At least from implementation point of view.

[dominic-p]

Yes, that's the general idea, and yes, security is definitely something to think through here. I see a couple of options off the top of my head:

1. We could require that Secrets have a special annotation to allows them to be accessed by WP pipelines.
2. We could whitelist secret names in the WP server or agent config.

I like the annotation option because is provides for some flexibility. Depending on how the annotation is structured, it could even limit the scope of access to the Secret to a specific org or even repo.

---

You can use WP secrets :) There are global and org secrets.

Yes, I'm aware of WP secrets at the global and org level. But, since they have to be defined manually, keeping them in-sync with the Kubernetes Secrets becomes a problem.

This is another thing. At least from implementation point of view.

I'm not sure why integrating with other external secret providers has be different from an implementation point of view. But, I'm not very familiar with the WP codebase.

[zc-devs]

I'm not sure why integrating with other external secret providers has be different from an implementation point of view. But, I'm not very familiar with the WP codebase.

Because it is server-side. Server parses pipeline, goes to external provider, loads secret, then set env vars for Agent/backend.

And your example is purely Agent/backend side. I can just translate it into secretKeyRef Pod's definition.

1. Like repo name?
   Where it should be processed?
   If in Server, then server should be able to communicate with Kubernetes. More dependencies and configs (which already in Agent).
   If in Agent, when logic goes to... Agent :) Agent loads secret and checks if there is annotation with current repo name. Seems, not so difficult, but not just translation either.

2. This is awkward to me. Like hardcoded privileged plugins. Besides, you should store it somewhere (CRUD). Perhaps in DB => new Server GUI, etc. Too difficult, I think.

or agent config

It's just dirty hack :)

[dominic-p]

Thanks for the explanation. That makes a lot of sense. Keeping it with the other Kubernetes backend options seems like it would definitely be the path of least resistance.

Yes, we could support annotations like this maybe:

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: my-pipeline-secret
  annotations:
    woodpecker-ci.org/org-access: "my-org,another-org"
    woodpecker-ci.org/repo-access: "my-repo,another-one"
data:
  password: d29vZHBlY2tlciByb2NrcyE=

[zc-devs]

To be done:

• Map an entire secret to environment (Configure all key-value pairs in a Secret as container environment variables, K8s secrets reference from step #3655)
• Allow to select keys and map them to alternate names (Define container environment variables using Secret data)
• Mount .dockerconfigjson secrets to a file (Create a Pod that has access to the secret data through a Volume, K8s secrets reference from step #3655 (comment))
• Implement access controls (Reference secret values from Kubernetes secrets #3582 (comment))
• Implement secret masking (K8s secrets reference from step #3655 (comment))

[zc-devs]

Having last updates, syntax below came up in my mind.

apiVersion: v1
kind: Secret
metadata:
  name: aws
  namespace: woodpecker
data:
  key-id: N0lrZHNzb0xleGFtcGxlNkpJVnZDRXE=
  access-key: TXk0WE5BYXNleGFtcGxldXRKRHNUWHc=
type: Opaque

1. Map an entire secret to the environment:

backend_options:
  kubernetes:
    secrets:
      - name: aws

2. Select a key:

backend_options:
  kubernetes:
    secrets:
      - name: aws
        key: access-key

3. Map to an alternate name:

backend_options:
  kubernetes:
    secrets:
      - name: aws
        key: key-id
        target:
          env: AWS_ACCESS_KEY_ID
      - name: aws
        key: access-key
        target:
          env: AWS_SECRET_ACCESS_KEY

4. Mount as a file:

apiVersion: v1
type: kubernetes.io/dockerconfigjson
kind: Secret
metadata:
  name: reg-cred
  namespace: woodpcker
data:
  .dockerconfigjson: <base64 encoded JSON auth file here>

backend_options:
  kubernetes:
    secrets:
      - name: reg-cred
        key: ".dockerconfigjson"
        target:
          file: "~/.docker/config.json"

What do you think?

[dominic-p]

I love it. This syntax seems perfect to me.

[meln5674]

Being able to do the same things with ConfigMap's would be super swell for things like certificate bundles and site-specific config files that don't belong in git, and the logic should be nearly copy-paste.