Configs files are hard to share across multiple environments

[silvin-lubecki]

A compose file can specify some config files, which are resolved inside the invocation images among the attachment files, (see #336 and #392 ). But those config files rarely match other environments for what they were defined.
A user consuming an app from a registry and who wants to install it on its own environment would need to override those config files inside the invocation image, but right now he/she can't.
I see two ways (cumulative) to handle that:
1/ Try to apply variable substitution on every/or specified attachments. It implies creating a new "variable substitution engine", as we re-use the compose one which only understands yaml files.
With this variable substitution extended to attachments, an app maintainer can add customization points directly at config file level and leverage docker app parameters files.

2/ Add a way to mount volumes and/or files in the invocation image, so a user can override a configuration file (or any other file) during the installation. It means we need to persist this file in the installation store, as we don't want the user to specify this mount again for an upgrade.

[thaJeztah]

Note that we allow templating in config files (docker swarmkit); wondering if that's also a solution to some of these cases.

IOW make the non-portable parts configurable through parameters

[ijc]

I think your option 1/ is by far the better option, despite the need to handling templating of parameters in attachments which as you say is going to be harder than with docker-compose.yaml since we cannot understand the semantics of the attachment files. I think that's fairly doable though.

I actually thought we already allowed templating there (at least for experimental=true builds) but I guess that was just a faulty memory.

I think allowing mounts into the invocation image (option 2/) is far too big of a hammer and breaks the encapsulation/manageability we are aiming for with apps.

[kinghuang]

I just ran into this limitation while converting an app from 0.6.0 to 0.8.0. I have an app that uses a config to take in a fluent-bit configuration file. With docker-app 0.6.0, I was able to install (deploy) the app with a parameter for the path of a config file outside of the app bundle to use. Now, I'm unable to pass in a config file outside the bundle.

Option 2 might work for this. The fluent-bit configuration file is an arbitrary list of stream processing steps. It wouldn't make sense to try to parameterize it.

[kinghuang]

Also, secrets. Trying out a different app originally built with docker-app 0.6.0. It loads up TLS certs as secrets from files. But, there's no way to pass those files in with docker-app 0.8.0.

[thaJeztah]

@chris-crone @silvin-lubecki ^

[scottyk537]

Another similar issue: Without the ability to reference parameters (attachments) files within an application pulled from a registry, it becomes very difficult to install older versions of an application to different target contexts when parameters are added or changed. I’m having to manually find and replace old parameter files from source control. When parameters change, they break older versions of the application.

[decentral1se]

1/ Try to apply variable substitution on every/or specified attachments. It implies creating a new "variable substitution engine", as we re-use the compose one which only understands yaml files.

That sounds like the way to go, no? I am thinking of Jinja2 here and things like:

# parameters.yml
---
with_configs: true
myservice:
  configs:
    - source: foo
      target: bar

# docker-compose.yml
---
{%- if with_configs -%}
configs: "${myservice.configs}"
{% endif %}

Which allows for conditional addition of config files, volumes and parametrisation of those via the parameters.yml file. I guess this is what is needed to account for real use cases and to make the app definitions generally useful in practice.

For example, in my staging environment I might use a bare docker socket for a volume but in my production setup I'll use a proxy with fine grained permissions. I don't want to have to package up two apps (apps-stage, apps-prod) to be docker app install'd, right?

I guess this leaks into a wider design discussion about how reusable these apps can be.