Migrate from environment variables to container labels

[rhansen]

There have been some recently reported issues (and probably old issues) involving environment variable collisions between nginx-proxy and a proxied container (#2144 (comment) and #2141). We should migrate away from environment variables on proxied containers and use labels instead.

We may also want to migrate the environment variables on the nginx-proxy container itself to labels, but the advantages are less clear. Originally posted in #1934 (comment) and moved here:

Advantages of environment variables for the nginx-proxy container:

• Less work: we don't need to migrate them to labels like we do with the environment variables on proxied containers.
• Having distinct ways of configuring nginx-proxy vs. the other containers might make configs more readable (environment variables are for the container's own use, labels are for use by a different container).
• Consistency: Each container's own configuration comes from environment variables. (Labels are for external consumption.)
• Less ambiguity: Does the com.github.nginx-proxy.nginx-proxy.foo label apply to proxied containers, to the nginx-proxy container, or to both? With environment variables on nginx-proxy and labels on other containers, the setting's role is clear. (Though we could use label namespaces, such as com.github.nginx-proxy.nginx-proxy.proxied.* for labels on proxied containers and com.github.nginx-proxy.nginx-proxy.proxy.* for settings on the nginx-proxy container itself.)
• Nested proxies: If nginx-proxy used labels instead of environment variables for its own settings, then it would be difficult to add support for proxying to another nginx-proxy container due to the likelihood of label collisions. (Namespacing could be used to avoid collisions but that might be more awkward than just using environment variables.)

Advantages of labels:

• Consistency: All nginx-proxy configuration is done via labels (and include files).
• Consistency: Setting com.github.nginx-proxy.nginx-proxy.foo on the nginx-proxy container could set the default value of the container-specific label of the same name.
• Less risk of collision with environment variables used by other code in the nginx-proxy container (nginx itself, a custom nginx module added by the user, the shell, docker-gen, Docker API libraries, logging libraries, etc.).
• Less pollution of the environment seen by various processes.
• It might be easier to prevent undesired disclosure of privacy- or security-sensitive values: Environment variables are normally copied to every spawned process, which makes it easier for an attacker to find a way to inspect the configuration. It should be easier to lock down access to the Docker socket to prevent such disclosure. (I haven't carefully studied the security implications so labels might actually increase the risk.)

[buchdag]

We may also want to migrate the environment variables on the nginx-proxy container itself to labels, but the advantages are less clear.

I gave a bit of thought to this and discussed it with some friends and colleagues last week. I strongly think that labels are not suited to store configuration properties meant for the process running inside the same container (ie replace environment variable on the nginx-proxy container). I don't think there is any mechanisms to access a container's label from inside itself short of exposing the Docker endpoint to the container and interacting with the API like docker-gen is doing, making them highly non standard for this use, and in our case a candidate only as a side effect.

Consistency: Each container's own configuration comes from environment variables. (Labels are for external consumption.)

💯

Regarding the more general move away from environment variable on the proxied containers, my biggest point of concern is the (easy) availability of labels on other Docker compatible container engines (I'm not certain they're easily available on AWS Elastic Container Service for instance).

[buchdag]

Another (theoretical) possibility for the nginx-proxy container would be to keep using environment variables but rename and correctly prefix them so that collisions are way less probable (NGXP_*, NGINX_PROXY_*, or something like that).

[buchdag]

I'm not certain they're easily available on AWS Elastic Container Service for instance

Just checked that: Docker labels were available on the old ECS management console, but aren't on the new one. My guess is that they will be added back eventually (but no idea when) and that they're still available when using JSON task definitions instead of the management console.