Add support for non-swarm container secrets

[xeptore]

Closes #40046.

• What I did

  • Add support for non-swarm container secrets using APIs
  • Add support for non-swarm container secrets using command line

• How I did it

  • Added Secrets field to the github.com/docker/docker/api/types/container.Config type
  • Include secret file creation on host machine, and secrets mounting procedures to container's start time (similar to how Swarm secrets are created and mounted to the containers)

• How to verify it

  • Direct Daemon API Interaction
    • Call ContainerCreate daemon API with the request body containing the newly added Config.Secrets field
    • Check Config.Secrets section of the container's config.v2.json file to see secrets associated to the container
    • Starting the container will
      • Create secret files in /mounts/secrets/<secret-id> of the container root directory, similar to Swarm secrets
      • Mount secret files to defined paths in the container
  • Using Docker-Compose
    • Download and build forked docker-compose project from here
    • Use the sample compose configuration and example password.txt secret file below, you can verify that the secrets defined for the service, are being mounted to the container with proper permission and mode:
      • compose.yml:

         version: "3.9"
         services:
           web:
             image: nginx:latest
             secrets:
               - source: password
                 target: /var/run/my_secret
                 mode: 0400
                 uid: "95"
                 gid: "85"
               - password
         secrets:
           password:
             file: ./password.txt
             name: password

      • password.txt

         p4sSw0rD

• Description for the changelog

  • The proposed solution is similar to how secrets are being treated in Swarm mode.
  • I decided to store secret raw file content in the container's config.v2.json file, instead of storing them in an in-memory or file-based database, without any encryption/decryption mechanism, as it's a local container, and file permissions seems enough for me. Also, it's simpler!
  • Current implementation should be backward-compatible as it only adds a new field to the container config field, which if not used, does not change anything for the container, even contents of config.v2.json file.
  • Current implementation does not affect Swarm mode secrets, as the fields that hold information about secrets are different.
  • I tried to reduce code duplication between Swarm mode and non-Swarm mode secret management, but if there's anything that can be improved, I'm willing to refactor the changes.

• Considerations

  • Other than the security vulnerabilities that storing container secrets in its config.v2.json file might introduce, I cannot see any other major issues

[AkihiroSuda]

Can we avoid importing swarmkit v1 and v2 simultaneously?

[xeptore]

@AkihiroSuda, resolved in 760200f.

[AkihiroSuda]

Can we have integration tests?
https://github.com/moby/moby/tree/master/integration/secret

[thaJeztah]

I don't think this is the right thing to do.

While I can acknowledge the problem that docker secret is not currently supported for non-swarmkit containers, I don't think we should use this approach to provide it;

• docker secret was created to provide a secure way to pass secrets (not using env-vars, or plain-text files); storing the secret as plain text in the container's configuration means that these secrets leak (which is not the expectation from users when using such a feature)
• in the current implementation in this PR, the secret-data is stored in the container Config, which also lands in any image that may be created from the container (think of docker commit).

Storing the secret this way would mean it doesn't provide benefits over using an --env or mounting a file from the host inside the container.

We definitely should have a look at other options though (one option is to use the existing docker secret feature, but using an alternative backend for storing the secret, or enabling the swarm store by default.

[xeptore]

Thanks for your consideration @thaJeztah.

I don't think this is the right thing to do.

As I mentioned in the PR description, I totally agree that storing secret content as raw string in the container's configuration file is not the perfect so‍‍lution.

Storing the secret this way would mean it doesn't provide benefits over using an --env‍ ‍or mounting a file from the host inside the container.‍

But, I do not completely agree with this statement. I believe the concept of secrets has two main properties:

• They should be transmitted, and stored securely, as they might contain confidential data
• They should be configured to be mounted to the container with proper setup, i.e., destination file permissions and mode

This PR, although it has some security concerns around storing secrets, fixes the issue with the second point, which currently is not supported by Docker, as discussed in #40046.

Regarding the storage concern, as I realized, please correct me if I'm wrong, the current solution by Swarmkit is encrypting the snapshot of its raft state before being persisted, and decrypting the file during the startup process. To prevent adding the container's secrets into its Config property, I tried using boltdb in a similar solution to how the daemon stores information about volumes in <daemon_root>/volumes/metadata.db file, to <daemon_root>/secrets.db (see this commit), which simply stores a key-value pair of containerID->secrets. It gets loaded upon daemon startup, and closed/persisted at daemon shutdown phase. This solution has the benefit of not adding a new field to container Config field, and simplicity compared to moving toward Swarmkit in any way, but due to my lack of information about encryption/decryption, it actually stores secrets in a raw format. I'm also open to any further research or implementing a solution around dealing with secret storage, if you'd guide me.

[thaJeztah]

Swarm secrets are stored in swarm's raft store (which is encrypted), and (but it's been a while since I looked at the full implementation), it uses a memfs mount to prevent the mounted file from being written to disk.
For transmitting the data, we should also make sure that it doesn't end up in logs (for example, with the daemon in debug mode, API request calls may be logged).

That said; I don't think using the SwarmKit store for this is ideal; using the SwarmKit store means that secrets will always have to be "imported" into that store (which somewhat makes sense for swarm services, to distribute them, but for situations where a secure store is already available for secrets, means that the secret is duplicated); I know some work was done to allow swarm services to use an external store (see moby/swarmkit#2326), so possibly something like that should be explored as an option (and make it available for both "swarm" and "non-swarm" containers, allowing docker secret and docker config to be used for both).

[xeptore]

What do you think about using an encryption algorithm similar to what Swarmkit is doing to encrypt its raft snapshots, then using a memfs or ramfs mount typed storage with the same simple storage mechanism, e.g., boltdb, implementation for daemon's local container secrets, instead of making changes to Swarmkit?

[lonix1]

We've needed this since forever. Running a single-server stack - just to have access to secrets functionality - has proved to be very hard to maintain. This feature is sorely needed. I hope you can agree on all the underlying technical matters.

[xeptore]

I'm going to close this PR, as I did not see much interest, and did not receive enough useful feedback from maintainers after a while.

[lonix1]

Noooo. @thaJeztah Will your team reconsider this?

It's a major source of pain - lots of issues here and on StackOverflow show that to be the case.

[s4ke]

@lonix1

With the risk of sounding tone-deaf: Have you considered running docker swarm on that single node?

[lonix1]

@s4ke Thanks. That's what I've been doing for years, and it's a source of much friction. We need proper secrets in docker compose.

Surely @xeptore got the ball rolling on this... Why no interest?

[s4ke]

I don't think there is no interest, but I think right now the focus is on making 22.06 stable (I am only a community member, so please take this with a grain of salt).

[lonix1]

@xeptore Thanks for your efforts... I hope you'll reconsider opening and leaving this in the queue. 😄

[xeptore]

Sure @lonix1. I have some open questions both here and in Slack. I'll reopen this PR once I have answers to them.

Cheers

[lonix1]

Hey @xeptore, how does your PR compare to docker/compose#9386?

[xeptore]

@lonix1, as the title of docker/compose#9386 suggests, it's about build secrets. They are kind of build-time secrets, compared to the end goal of this PR, which is about (runtime mounted) secrets. AFAIK, build secret is a relatively new Docker Buildkit feature, which are the type of secrets you need during Docker image build process, e.g., GitHub access token to download some dependencies from GitHub's package registry during build time (npm install or bundle install for example), with the benefit of not leaving traces in the image history, and other minor features like permissions, etc. You can read more:

• https://docs.docker.com/develop/develop-images/build_enhancements/
• https://vsupalov.com/better-docker-build-secrets/
• https://render.com/docs/docker-secrets

[lonix1]

Thank you for those details. I'm still unfamiliar with the new "buildkit" stuff, but I had a feeling that it differs to what's being discussed here.

I think the approach in this PR is preferable. I'm saddened they didn't accept it.

[lonix1]

Hi again @xeptore... if this is still important to you, please upvote this new feature request, and/or add your opinions there.

@epopisces @marcoaureliocardoso I see you upvoted this issue. Please upvote that feature request and add your comments if any?