feat(env): load env vars at runtime with import-meta-env #428

billhimmelsbach · 2024-04-29T15:31:11Z

Allows env variables to be injected in sbl-frontend at runtime by using import-meta-env and a few other tweaks to nginx, which allows for a bunch of fun stuff:

configmaps can be used to set env vars (feat(frontend): add configmap regtech-deployments#15)
you can bounce a container to refresh env vars after modifying its configmap
you can run yarn docker-build-and-run to run just the frontend with env vars injected
no changes to local dev env work (keep using .env file like normal)
add sbl-frontend to sbl-project docker-compose.yaml (feat(docker): add sbl-frontend / profile to docker compose sbl-project#211)
alert users when they're missing env vars

There's a lot more that can be done here (see Future Improvements section), but this should be a good start. 🤞

This PR changes quite a few things, so I listed why I made each change in the section below (starting from the top of files changed list and working my way down) and included a bunch of code comments which I hope is helpful. 🤞

Changes

add .env to .dockerignore so that .env changes don't trigger image rebuilds
add .env.example for use by import-meta-env to define required env variables and to make it easy for new users to copy it and create their own .env files
modify .github/workflows/test.yml to add env variables to github workflows
modify .gitignore to ignore all .env files except the new .env.example
install new packages
- import-meta-env for injecting env vars, flexibility between .env and env variables being passed on the command line, allowing configmaps
  - includes: @import-meta-env/cli, @import-meta-env/unplugin, dotenv
- dotenv-cli for automating env var injection for yarn docker-run command
modify Dockerfile for env var injection
- add import-meta-env
- copy nginx config to allow for env var injection
- change CMD to ENTRYPOINT for env var injection and switching to non-root user
update env-guide with instructions on adding new env vars
update index.html with a placeholder for injecting env vars for prod builds / cache busting
add nginx-entrypoint.sh for env var injection and switching to non-root user
allow for env vars in nginx.conf to modify the CSP to add localhost when serving from docker locally
modify package.json
- add new packages (see above)
- add a few sbl-frontend only ways of running docker (in the future, we'll more often than not use sbl-project, see: feat(docker): add sbl-frontend / profile to docker compose sbl-project#211)
- modify docker-run to work now
  - removed the && open http://localhost:8080 to docker-run because it caused weird race conditions and made it hard to kill the running container
  - changed the port to 8085 to avoid using the same port as other containers
- add docker-build-and-run to combine build and run commands
modify axiosService.ts to change from a proxy based approach for base urls to per API axios instance approach. I talked with @shindigira about this: trying to replicate this kind of behavior in nginx causes some load balancer / upload issues, and causes some weirdness. This is a commonly used approach around these issues.
modify common.ts to include MAIL_BASE_URL
lint oidc.ts suggested removing as string now, since these env vars are already typed string
all API calls in /requests now use an axios instance for their API calls instead of relying on the proxy
modify vite.config.ts
- remove proxy
- add import-meta-env
- remove envPrefix since import-meta-env handles that now

Future improvements

How to test

If you're missing a variable in your .env file (see .env.example for complete list) do you see an error appear?

Are the github actions tests on this PR that now run on env vars passing?

Can you run yarn docker-build and yarn docker-run and see the website appear on http://localhost:8085/?

You may need to refresh the page after everything spins up, also see a future PR: #589
We'll mostly be using sbl-project to spin up the sbl-frontend container, but this is nice to have for standalone testing

Related PRs

meissadia

Awesome, really appreciate the documentation, and it works!

shindigira · 2024-05-29T18:52:14Z

Not critical but I think we can include https://import-meta-env.org/guide/tools/typescript.html to package.json with a script instead of relying on the manually created vite-env.d.ts

billhimmelsbach · 2024-05-29T20:43:56Z

Not critical but I think we can include https://import-meta-env.org/guide/tools/typescript.html to package.json with a script instead of relying on the manually created vite-env.d.ts

I ran this tool to generate that file @shindigira, so I'd be down to generate that file regularly / include it as a package.json script?

EDIT: I did rename it vite-env.d.ts, since that's what the vite docs recommended.

shindigira

Two small changes. Really awesome!

.github/workflows/test.yml

package.json

…nto 256-set-up-configmap

shindigira

Tested and verified:

missing environment variables will cause a terminal error
ensured all environment variables are used in local development
yarn start will prompt to install a typescript auto-generating package for the env variables
yarn start conditionally created the vite-env.d.ts

Let's add `sbl-frontend` to `sbl-project` 🥳 Closes #210 ## Changes - add `sbl-frontend` to `docker-compose.yml` (depends on cfpb/sbl-frontend#428) - add a `frontend` profile for easy startup (works with #213) - add a `frontend.local.env` with non-secret vars for local dev ## How to test this PR 1. Wait for cfpb/sbl-frontend#428 to get merged 2. Run `git checkout main && git pull" in the `sbl-frontend` repo to ensure you have the latest changes 3. Run `docker-compose --profile "*" up"` 4. Wait for things to spin up 5. Does navigating to http://localhost:8898/ result in seeing the home page? ## TODO - [x] merge cfpb/sbl-frontend#428 - [x] profiles: #212 which is closed by #213

Let's use a configmap for the `sbl-frontend` repo, since we can [use env vars at runtime](cfpb/sbl-frontend#428) now 🥳 Closes #14 ## Current behavior - env vars need to be baked into images in order for `sbl-frontend` to work ## Expected behavior - container env vars are defined by a configmap, templatized in the helm values.yaml, and then loaded at runtime by `import-meta-env` ## Additions - same changes as previous PR #7 for user-fi, but also includes `envFrom` - adds `configmap.yaml` - eval template in `deployment.yaml` - add configmap and `envFrom` to `values.yaml` ## Testing 1. Test in conjunction with the accompanying enterprise PR, see the testing instructions there ## TODOs - [x] depends on [this SBL Frontend PR to be merged first](cfpb/sbl-frontend#428)

billhimmelsbach added 30 commits April 29, 2024 08:30

feat(env): load env vars at runtime with import-meta-env

4d824c2

test(vitest): set tests to precompile .env

91aed5e

test(vitest): set tests to run in v18

846bc4e

chore(git): merge in latest main, resolve conflicts

5ea023e

chore(git): update to main

05520e4

deps(test): try rebuilding deps

ed25cb1

test(env): add default env vars for github actions

13a067e

test(env): change vite config, instead of command line args

ff7a13a

test(env): include empty sbl_logout_redirect_url

f3cbd9a

Merge branch 'main' into 256-set-up-configmap

6eb6f59

chore(deps): remove unnecessary deps changes

28627df

spike(api): temp add base_urls, nginx proxy later

8116e01

WIP

15e1a77

WIP

19aecc7

WIP

ada6c32

WIP

887a891

feat(nginx): dynamically add localhost to CSP

6113932

chore(linting): code cleanup and dockerignore

4e80619

chore(cleanup): code cleanup and comments

dcf8a01

fix(cleanup): remove dead code

13cab27

fix(Dockerfile): removed unused user

0eb48e0

fix(axios): remove proxy, use instances of axios

7e2d3d5

Merge branch 'main' into 256-set-up-configmap

43015ae

chore(cleanup): add missing \n, docs, remove dead code

5d5a4d2

docs(Dockerfile): add some comments; add missing quote

11cc002

chore(docs): cleanup the wording of the env guide

9bffa70

chore(common): replace missing as const

96ea89b

Merge branch 'main' into 256-set-up-configmap

0991790

fix(cypress): fix test runner, e2e mode

299a5b5

chore(docs): change package.json script name, env guide grammar

76e312c

docs(Dockerfile): add one more comment to existing cmd

98ae28e

This was referenced May 29, 2024

feat(docker): add sbl-frontend / profile to docker compose cfpb/sbl-project#211

Merged

[Task] Add configmap for sbl-frontend cfpb/regtech-deployments#14

Closed

feat(frontend): add configmap cfpb/regtech-deployments#15

Merged

billhimmelsbach added 4 commits May 29, 2024 05:14

docs(env-guide): add info on how to add env vars

6086b4c

chore(cleanup): edit comment, newline package.json

f7e7c88

docs(env-guide): small change to adding new env

c1e12a3

fix(docker-run): no race condition, easy shutdown

c8e5399

billhimmelsbach marked this pull request as ready for review May 29, 2024 14:36

billhimmelsbach requested review from meissadia and shindigira as code owners May 29, 2024 14:36

git(requests): add axios instances for new api calls from main

4b74114

billhimmelsbach changed the title ~~wip: feat(env): load env vars at runtime with import-meta-env~~ feat(env): load env vars at runtime with import-meta-env May 29, 2024

meissadia previously approved these changes May 29, 2024

View reviewed changes

Merge branch 'main' into 256-set-up-configmap

8eb4c78

shindigira reviewed May 29, 2024

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

package.json Outdated Show resolved Hide resolved

billhimmelsbach added 2 commits May 29, 2024 18:32

feat(cleanup): generate types, docs, env var fix

2e5d364

Merge branch '256-set-up-configmap' of github.com:cfpb/sbl-frontend i…

986b159

…nto 256-set-up-configmap

billhimmelsbach dismissed meissadia’s stale review via 986b159 May 30, 2024 01:33

shindigira self-requested a review May 30, 2024 01:44

shindigira approved these changes May 30, 2024

View reviewed changes

billhimmelsbach merged commit 21ba292 into main May 30, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(env): load env vars at runtime with import-meta-env #428

feat(env): load env vars at runtime with import-meta-env #428

billhimmelsbach commented Apr 29, 2024 •

edited

meissadia left a comment

shindigira commented May 29, 2024

billhimmelsbach commented May 29, 2024 •

edited

shindigira left a comment

shindigira left a comment

feat(env): load env vars at runtime with import-meta-env #428

feat(env): load env vars at runtime with import-meta-env #428

Conversation

billhimmelsbach commented Apr 29, 2024 • edited

Changes

Future improvements

How to test

Related PRs

meissadia left a comment

Choose a reason for hiding this comment

shindigira commented May 29, 2024

billhimmelsbach commented May 29, 2024 • edited

shindigira left a comment

Choose a reason for hiding this comment

shindigira left a comment

Choose a reason for hiding this comment

billhimmelsbach commented Apr 29, 2024 •

edited

billhimmelsbach commented May 29, 2024 •

edited