Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Environment variable override support #136

Open
Mechite opened this issue May 4, 2023 · 3 comments
Open

Environment variable override support #136

Mechite opened this issue May 4, 2023 · 3 comments

Comments

@Mechite
Copy link

Mechite commented May 4, 2023

Implementary notes
Should be available on builder classes, etc, as configuration options for loading configurations.

Expected behaviour
When a qualifier in the configuration, e.g. server.port is set as an environment variable (should be case insensitive), it should be the preferred value in the configuration when this feature is enabled. If the environment variable cannot be parsed correctly when requested, by an ObjectConverter or by a get* method, the value from the configuration file is used as a fallback.

Motivation
night-config provides a great way for efficiency in development environments, especially with auto updating, etc.
Large, scalable environments tend to prefer environment variables for configuration, e.g. Docker on k8s.

@Mechite
Copy link
Author

Mechite commented May 6, 2023

Extended proposal
Serialising from a pure environment variable is difficult as there are no primitives for lists and such. Therefore, I propose that we parse environment variables with a JSON-like syntax mixed with whatever configuration language the user might be using for the larger types.

  • Strings - Should be read if the value is pure text that can't be a number or boolean, i.e. server.identifier = Hello, world!
  • Numbers - Should be read if the value is serialisable as a number, i.e. server.port = 8080
  • Boolean - Should be read if the value is, well, true/false, i.e. server.production = true
  • Objects - Should be a supported value to set multiple environment variables in one block, parsed as whatever language the user happens to be using, i.e, given they are using JSON, example.settings = { port: 8080, identifier: "Hello, world!" }
  • Arrays - Should be parsed as whatever language the user happens to be using, i.e, given they are using JSON, server.seeds = [8080, 8090].

This is not trivial to parse strings like this but it would be a very useful feature. Arrays are likely going to be required to make environment variables a useful way to override configuration, but objects are quite useless if you can just make multiple qualifying environment variables, so do not need to be implemented. Either way, languages like TOML may it impossible to do things this way, so I'm really unsure how easy it will be to make it support the user's chosen markup, or if JSON should be the only option here.

@TheElectronWill
Copy link
Owner

yes, that would be a nice feature to offer! Thank you for the proposal.
We could take inspiration from clap here (also available in "derive" mode, which is analogous to NightConfig's annotations in terms of usage).

@Mechite
Copy link
Author

Mechite commented Jul 19, 2023

Alright so a little change to the proposal (I'm not familiar with clap and don't know if that has its own, better semantics for this)

Arrays in environment variables should not be parsed like that but instead should be lists separated with semicolons (;) as this is the idiomatic way to do it on all platforms (see PATH).

An array containing objects should be achievable by passing the users' configuration language in, i.e. this is what it would look like to pass an object in:
EXAMPLE={ name: "Mr Raffin", age: null }
and this is what it would look like to pass an array of objects:
EXAMPLE={ name: "Mr Raffin", age: null };{ name: "Mechite", age: 127 }

Given they are using a language that requires new lines, etc for this, escape codes can be used directly, and if the semicolon would normally be parsed by the language (a lot of these languages don't define start and EOF like JSON), it can be escaped too, i.e. with YAML:

EXAMPLE=name: Mr Raffin\nage: ~\;name: Mechite\nage: 127
which is basically just an escaped semicolon between these two YAML files:

name: Mr Raffin
age: ~
name: Mechite
age: 127

I feel like using objects/documents in environment variables is a very rare use case, but having support for it is ideal, and I think having a standard way of defining arrays is better simply because they are going to be used every often, e.g:
HOSTS=127.0.0.1:8080;172.17.0.1:8080

@TheElectronWill TheElectronWill self-assigned this Jan 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants