🐛 Bug Report: Environment APP_CONFIG_ variables including hyphens are not valid identifiers

[jvanalstyne-mdsol]

📜 Description

Setting an APP_CONFIG_ environment variable that includes hyphens in the name is not permitted by the shell. However, some backstage app config options use kebab case and include hyphens in the names, and are therefore unable to be overridden by APP_CONFIG_ environment variables.

👍 Expected behavior

Setting an APP_CONFIG_ env variable should have the same effect as setting the option in app-config.yaml. For example, if the following works in yaml:

backend:
  csp:
    upgrade-insecure-requests: false

I should be able to effectively do this (or similar) in my .env file to achieve the same effect*:

export APP_CONFIG_backend_csp_upgrade-insecure-requests="false"

*Given that shell variables generally only allow alphanumerics and underscores, alternative formatting may be necessary, such as using double underscores in place of hyphens:
APP_CONFIG_backend_csp_upgrade__insecure__requests

👎 Actual Behavior with Screenshots

In my particular case, including the following line in my local .env:

export APP_CONFIG_backend_csp_upgrade-insecure-requests="false"

will produce the following error on start:

.env: line 99: export: `APP_CONFIG_backend_csp_upgrade-insecure-requests="false"': not a valid identifier

The error itself seems to be caused by shell variables not accepting hyphens in the name, however, as mentioned above, this prevents a developer from using environment variables to configure some backstage app config options.

👟 Reproduction steps

1. Supply an APP_CONFIG_ variable in your .env that includes an option with a hyphen in its name, such as APP_CONFIG_backend_csp_upgrade-insecure-requests
2. Start backstage

📃 Provide the context for the Bug.

After upgrading from 1.24.2 to 1.26.5, our testing workflow has broken due to not being able to access a containerized instance of the service using insecure http. I found this workaround which seems to resolve the issue when modifying the app-config.yaml directly, but because of our testing workflows, we want to avoid having an app-config.local.yaml in the container, and would prefer to use an .env variable (which we would set in a docker-compose file).

🖥️ Your Environment

OS:   Darwin 23.4.0 - darwin/arm64
node: v18.19.1
yarn: 1.22.22
cli:  0.26.4 (installed)
backstage:  1.26.5

Dependencies:
  @backstage/app-defaults                                          1.5.4
  @backstage/backend-app-api                                       0.7.2
  @backstage/backend-common                                        0.21.7
  @backstage/backend-defaults                                      0.2.17
  @backstage/backend-dev-utils                                     0.1.4
  @backstage/backend-openapi-utils                                 0.1.10
  @backstage/backend-plugin-api                                    0.6.17
  @backstage/backend-tasks                                         0.5.22
  @backstage/backend-test-utils                                    0.3.7
  @backstage/catalog-client                                        1.6.4
  @backstage/catalog-model                                         1.4.5
  @backstage/cli-common                                            0.1.13
  @backstage/cli-node                                              0.2.5
  @backstage/cli                                                   0.26.4
  @backstage/config-loader                                         1.8.0
  @backstage/config                                                1.2.0
  @backstage/core-app-api                                          1.12.4
  @backstage/core-compat-api                                       0.2.4
  @backstage/core-components                                       0.13.10, 0.14.6
  @backstage/core-plugin-api                                       1.9.2
  @backstage/dev-utils                                             1.0.31
  @backstage/errors                                                1.2.4
  @backstage/eslint-plugin                                         0.1.7
  @backstage/frontend-plugin-api                                   0.6.4
  @backstage/integration-aws-node                                  0.1.12
  @backstage/integration-react                                     1.1.26
  @backstage/integration                                           1.10.0
  @backstage/plugin-api-docs                                       0.11.4
  @backstage/plugin-app-backend                                    0.3.65
  @backstage/plugin-app-node                                       0.1.17
  @backstage/plugin-auth-backend-module-atlassian-provider         0.1.9
  @backstage/plugin-auth-backend-module-aws-alb-provider           0.1.9
  @backstage/plugin-auth-backend-module-azure-easyauth-provider    0.1.0
  @backstage/plugin-auth-backend-module-bitbucket-provider         0.1.0
  @backstage/plugin-auth-backend-module-cloudflare-access-provider 0.1.0
  @backstage/plugin-auth-backend-module-gcp-iap-provider           0.2.12
  @backstage/plugin-auth-backend-module-github-provider            0.1.14
  @backstage/plugin-auth-backend-module-gitlab-provider            0.1.14
  @backstage/plugin-auth-backend-module-google-provider            0.1.14
  @backstage/plugin-auth-backend-module-microsoft-provider         0.1.12
  @backstage/plugin-auth-backend-module-oauth2-provider            0.1.14
  @backstage/plugin-auth-backend-module-oauth2-proxy-provider      0.1.10
  @backstage/plugin-auth-backend-module-oidc-provider              0.1.8
  @backstage/plugin-auth-backend-module-okta-provider              0.0.10
  @backstage/plugin-auth-backend                                   0.22.4
  @backstage/plugin-auth-node                                      0.4.12
  @backstage/plugin-auth-react                                     0.1.1
  @backstage/plugin-catalog-backend-module-aws                     0.3.12
  @backstage/plugin-catalog-backend-module-backstage-openapi       0.2.0
  @backstage/plugin-catalog-backend-module-github                  0.6.0
  @backstage/plugin-catalog-backend-module-openapi                 0.1.35
  @backstage/plugin-catalog-backend                                1.21.1
  @backstage/plugin-catalog-common                                 1.0.22
  @backstage/plugin-catalog-graph                                  0.4.4
  @backstage/plugin-catalog-node                                   1.11.1
  @backstage/plugin-catalog-react                                  1.11.3
  @backstage/plugin-catalog                                        1.19.0
  @backstage/plugin-events-node                                    0.3.3
  @backstage/plugin-home-react                                     0.1.12
  @backstage/plugin-home                                           0.7.3
  @backstage/plugin-kubernetes-common                              0.7.5
  @backstage/plugin-org                                            0.6.24
  @backstage/plugin-permission-backend                             0.5.41
  @backstage/plugin-permission-common                              0.7.13
  @backstage/plugin-permission-node                                0.7.28
  @backstage/plugin-permission-react                               0.4.22
  @backstage/plugin-proxy-backend                                  0.4.15
  @backstage/plugin-scaffolder-common                              1.5.1
  @backstage/plugin-scaffolder-node                                0.4.3
  @backstage/plugin-search-backend-module-catalog                  0.1.23
  @backstage/plugin-search-backend-module-elasticsearch            1.4.0
  @backstage/plugin-search-backend-module-techdocs                 0.1.22
  @backstage/plugin-search-backend-node                            1.2.21
  @backstage/plugin-search-backend                                 1.5.7
  @backstage/plugin-search-common                                  1.2.11
  @backstage/plugin-search-react                                   1.7.10
  @backstage/plugin-search                                         1.4.10
  @backstage/plugin-techdocs-backend                               1.10.4
  @backstage/plugin-techdocs-module-addons-contrib                 1.1.9
  @backstage/plugin-techdocs-node                                  1.12.3
  @backstage/plugin-techdocs-react                                 1.2.3
  @backstage/plugin-techdocs                                       1.10.4
  @backstage/plugin-user-settings                                  0.8.5
  @backstage/release-manifests                                     0.0.11
  @backstage/repo-tools                                            0.8.0
  @backstage/test-utils                                            1.5.4
  @backstage/theme                                                 0.4.4, 0.5.3
  @backstage/types                                                 1.1.1
  @backstage/version-bridge                                        1.0.8

👀 Have you spent some time to check if this bug has been raised before?

• I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• I have read the Code of Conduct

Are you willing to submit PR?

Yes I am willing to submit a PR!

[jvanalstyne-mdsol]

Not entirely unrelated, but my team is also very confused as to why this behavior started occurring after the upgrade... 😅

[hballangan-mdsol]

I suggest variable naming in app-config should stick to camelCase. Suggestion like using __ is too patchy

propose for new name can be upgradeInsecureRequests. Also would be good to enforce a case for other configuration as well to prevent issues from coming up.

[jvanalstyne-mdsol]

I suggest variable naming in app-config should stick to camelCase. Suggestion like using __ is too patchy

I would prefer camelCase, too, tbh, but I suggested double underscores as a convention to prevent from breaking existing configs, and also because I wasn't sure if it was possible/how to enforce that for plugins–they should also theoretically be able to have their config options overridden by env vars.

[vinzscam]

I suggest variable naming in app-config should stick to camelCase. Suggestion like using __ is too patchy

propose for new name can be upgradeInsecureRequests. Also would be good to enforce a case for other configuration as well to prevent issues from coming up.

yeah probably we should enforce the usage of camelCase in app-config in general.

[vinzscam]

thanks for reporting this @jvanalstyne-mdsol, opened #24723 which should fix your issue

[jvanalstyne-mdsol]

Thank you @vinzscam!!!