Clarify if env variables should take precedence to config provided by param

[BrynCooke]

If a config file is provided by param; should env variables take precedence? Env variables overriding discovered local or global config makes sense, but if config is passed in via param it is less clear.

config.yaml:

feature:
  enabled: false

bash:

export CONFIG_FEATURE_ENABLED=true
my-program --config config.yaml

In this case if I had an env variable set then it would invisibly override the config that I am passing in explicitly.
I can also see the opposite argument that env variables should always override config.

[tashian]

Hi, thanks for the note.

In my view, it really doesn't matter whether the config file is provided by a flag or loaded from some default location, the env variable settings should still take precedence over any config file. This is because env variables are often used for last-minute config overrides.

Now, if the flag were --feature-enabled false, then the flag would take precedence.

[tashian]

Also, one pattern we use at my company with the step command is to prefix all of our env variables with STEP_, to avoid name clashes with other software. And we allow all flags to be provided as env variables. So, STEP_CONTEXT=carl step ca health is the same as step ca health --context carl. We also allow default values to be set for any flags, which are loaded from a defaults.json file before each run (and are of the lowest precedence).

[foretspaisibles]

A common pattern for the precedence cascade of configuration sources is to

1. Take parameters from files stored in system locations like /etc, /usr/lib, etc.
2. Take parameters from files stored in user home directory ($XDG_HOME, dot files, etc.)
3. Take parameters from the environment variables.
4. Take parameters from the command line.

(Larger number wins.)

The rationale for this cascade is the ease of change or frequency of change for the source of the parameter. System files usually do not change, user files change as the user see fits but it would be inconvenient to edit a file before every single command run. Environment variables are usually set for a session and command line can change at every … line.

There is however a lot of variations and the most important one that I have seen, is when the application should enforce administrators settings and should not allow the user to override the settings in the system files. (example: sudo)

For a user tool, most users would be surprised if the cascade were not respected.

Some tools (AWS CLI) also have a stage 0, where authorization parameters are pulled from the computational context. (the magic aws private IP)

I once wrote a CLI library for OCaml (OCaml and Lisp are great languages!) implementing this cascade in parametrisable way, to have off-the shelf patterns:

So, for Unix filters the cascade is to use the Environment and then CLI parameters. No configuration files for filters.
https://github.com/foretspaisibles/gasoline/blob/master/application/gasoline_Plain_Filter.ml#L63

A user tool follows the cascade above. https://github.com/foretspaisibles/gasoline/blob/master/application/gasoline_Plain_UserTool.ml#L28

But a secure tool has the cascade modified by pulling the system files at the strongest position of the list — https://github.com/foretspaisibles/gasoline/blob/master/application/gasoline_Plain_SecureTool.ml#L32

For a dæmon or a service the cascade is as above but there is no user configuration. https://github.com/foretspaisibles/gasoline/blob/master/application/gasoline_Plain_Daemon.ml#L73