Passing config using environment variables

[femtotrader]

Hello,

I'm using Travis-CI for Continuous Integration in many projects.
I'm planning to use traitlets as a way to configure some tools (passing API_KEY, ...)

To be able to test my projects, I need to pass some settings using environnement variables (only for Continuous Integration) because it's necessary to "hide" (from Git repository) some config data (API_KEY, password...)

see http://docs.travis-ci.com/user/environment-variables/

Is it something which is possible with traitlets ?
Is it something which could be considered in traitlets ?

Kind regards

[minrk]

I use env variables to populate traitlets in a few projects (e.g.). I've thought about a way to support it officially in traitlets, but I haven't though of a way that ends up being better than what already works right now: a one-line default generator that reads a given env.

[ssanderson]

but I haven't though of a way that ends up being better than what already works right now: a one-line default generator that reads a given env.

What about just enshrining this common pattern as universally-supported behavior? For example:

class MyClass(Configurable):
    api_key = Unicode(config=True, envvar="MY_ENVVAR_NAME")

The expected behavior here would be that this behaves equivalently to:

class MyClass(Configurable):
    api_key = Unicode(config=True)
    def _api_key_default(self):
        return os.environ['MY_ENVVAR_NAME']

[minrk]

I think that should work.

[ankostis]

(I'm writting it here, because is a general discussion, not related to proposed PRs)

On #246, Carreau wrote:

[...] I think we should need to be extra careful for the precedence rule of ENV var vs config, vs command line argument as well [...]

I want to put my (light)weight on it: generally programs (e.g. gpg, git) receive configuration values from the following "sources", in that order (later ones win over previous):

• program defaults (trait.default_value, then dynamic_defaults),
• config-files,
• env-var,
• command-line arguments.
• values assigned by code [edit: of course...]

So far, command-line argument take precedence over config-files because they are merged on top of them.
But with the proposed changes (i.e. #157, #158, #246) I don't see env-vars taking precedence over config-files.

Am i correct to worry?

[Carreau]

This came up again in jupyter/nbconvert#2026 I've came up with this in the config file:

$ cat ~/.jupyter/jupyter_nbconvert_config.py
import os

for k, v in os.environ.items():
    if k.startswith("NBCONVERT"):
        # use __ instead of . as separator in env variable.
        _, klass, attribute = k.split("__")
        # c.Klass.attribute=v
        setattr(getattr(c, klass), attribute, v)

I believe we might be able to adapt it by pulling the app name in traitlets and have a default config loader that loads values from env variables.

My main concern is how could this be abused and could it be a security risk ? My thoughts is that it is more common to give write access to env variables to users than access to CLI flags.

[Carreau]

See #856 as a prototype of how this could work.

I'd appreciate reviews.