(Suggestion) A workaround for Vercel

[raflymln]

References:

• Variables loaded with dotenv-vault not prefixed with NEXT_ are undefined (server side)  #261
• Dotenv Vault not working on Vercel vercel/vercel#10607

Since Vercel doesn't allow dynamic env load for server env key, so i decided to make this workaround for other user to use. Here's I'm making an example on Next.js project.

1. Create .env.key file containing DOTENV_KEY
2. Create this script in /scripts/load-env.ts or any other place you wish

import { config } from "dotenv";

import { writeFile } from "fs/promises";

const main = async () => {
    const keyEnv = config({ path: ".env.key" });
    const DOTENV_KEY = keyEnv.parsed?.DOTENV_KEY || process.env.DOTENV_KEY; // If DOTENV_KEY not found in .env.key, get it from process.env

    if (!DOTENV_KEY) {
        throw new Error("No DOTENV_KEY found in .env.key or process.env");
    }

    const envs = config({ DOTENV_KEY }); // If DOTENV_KEY found, get the .env.vault, load it to get shared env variables

    if (!envs.parsed || Object.keys(envs.parsed).length === 0) {
        throw new Error("No .env.vault found or it is empty");
    }

    try {
        await writeFile(
            ".env",
            // eslint-disable-next-line prefer-template
            "# This file is generated by /scripts/load-env.ts\n" +
                "# DO NOT ATTEMPT TO EDIT THIS FILE\n" +
                Object.entries(envs.parsed)
                    .map(([key, value]) => `${key}=${value}`)
                    .join("\n")
        );

        console.log("Successfully loaded .env.vault to .env");
    } catch (error) {
        console.error("Failed to load .env.vault to .env", error);
    }
};

main();

4. Add env:load to your scripts and edit existing script like this

"scripts": {
    "dev": "pnpm env:load && start http://127.0.0.1:3000 && next dev",
    "build": "pnpm env:load && prisma generate && next build",
    "env:load": "ts-node ./scripts/load-env.ts"
}

💡 You can change pnpm with any other package manager you want

So basically, it loaded the content of the .env.vault to .env and then proceed to build the app with .env keys, i think this way is more safe and straightforward than prefixing the keys with NEXT_PUBLIC_ since the key prefixed with that can be bundled in the JavaScript file.

And also this helps executing cli that are depends on env vars like prisma db pull more easier, if you are collaborating with other devs, they just need to load the env first using pnpm env:load after pulling from the repo.

Let me know what you think. Thank you!

[dnishiyama]

Thank you so much for this, @raflymln! Just got it working in our project and could not have done it without your help. We have a monorepo setup so the one modification we had is to make sure the .env gets written to the directory that holds Nextjs. Seems obvious in retrospect 😂

[raflymln]

Your welcome @dnishiyama! Glad to hear it's working :D

[jelling]

A little random, but for anyone trying to use dotenv vault with vercel now: I couldn't get the .env.vault from the build input to copy into the output no matter what I tried.

Naturally, I assumed I was making some sort of mistake so I contacted Vercel and they just pointed me to this issue without addressing the underlying file copy issue. Digging into their issue repo, I found at least one issue where the post-build file copy process had a bug that omitted files. 🤷

Eventually, I just gave up and went back to using Docker. But there might be some additional weirdness in Vercel's deployment process.

[dnishiyama]

Our process is still working for us. We use a yarn or pnpm monorepo like this https://github.com/t3-oss/create-t3-turbo. What framework are you using? What is your build command?

Here is our file content:

import { readFileSync, writeFileSync } from 'fs'
import { config } from 'dotenv'

// Assumes it is called from root and run from tooling/env/src/load-env.ts

const main = () => {
  const keyEnv = config({ path: '../../.env.key' })
  // Use no default so that our dotenv calls don't go down the vault path
  const DOTENV_KEY = keyEnv.parsed?.DOTENV_KEY ?? process.env.DOTENV_KEY // If DOTENV_KEY not found in .env.key, get it from process.env

  if (!DOTENV_KEY) {
    let hasDotEnv = false
    try {
      hasDotEnv = !!readFileSync('../../.env')
    } catch {
      /* empty */
    }
    if (hasDotEnv) {
      console.log('Found .env file, skipping .env.vault load')
      return
    }

    throw new Error('No DOTENV_KEY found in .env.key or process.env')
  }

  const envs = config({ DOTENV_KEY, path: '../../.env.vault' }) // If DOTENV_KEY found, get the .env.vault, load it to get shared env variables

  if (!envs.parsed || Object.keys(envs.parsed).length === 0) {
    throw new Error('No .env.vault found or it is empty')
  }

  try {
    // This env is needed for the app
    writeFileSync(
      '../../apps/nextjs/.env',

      '# This file is generated by /scripts/load-env.ts\n' +
        '# DO NOT ATTEMPT TO EDIT THIS FILE\n' +
        Object.entries(envs.parsed)
          .map(([key, value]) => `${key}=${value}`)
          .join('\n'),
    )

    console.log('Successfully loaded .env.vault to .env')
  } catch (error) {
    console.error('Failed to load .env.vault to .env', error)
  }
}

main()

And turbo.json

{
  "$schema": "https://turborepo.org/schema.json",
  "globalDependencies": ["**/.env"],
  "pipeline": {
    "build": {
      "dependsOn": ["^build", "^db:deploy", "^db:generate", "env:load"],
      "outputs": [".next/**", ".expo/**"]
    },
    "ci": {
      "dependsOn": ["lint", "typecheck", "build"]
    },
    "clean": {
      "cache": false
    },
    "db:deploy": {
      "inputs": ["prisma/schema.prisma"],
      "cache": false
    },
    "db:generate": {
      "inputs": ["prisma/schema.prisma"],
      "cache": false
    },
    "db:push": {
      "inputs": ["prisma/schema.prisma"],
      "cache": false
    },
    "db:seed": {
      "inputs": ["prisma/schema.prisma"],
      "cache": false
    },
    "dev": {
      "persistent": true,
      "cache": false
    },
    "env:load": {},
    "format": {
      "outputs": ["node_modules/.cache/.prettiercache"],
      "outputMode": "new-only"
    },
    "lint": {
      "dependsOn": ["^topo"],
      "outputs": ["node_modules/.cache/.eslintcache"]
    },
    "topo": {
      "dependsOn": ["^topo"]
    },
    "typecheck": {
      "dependsOn": ["^topo"],
      "outputs": ["node_modules/.cache/tsbuildinfo.json"]
    }
  },
  "globalEnv": [
    "DATABASE_URL",
    "EXPO_ROUTER_APP_ROOT",
    "NEXTAUTH_SECRET",
    "NEXTAUTH_URL",
    "NEXT_PUBLIC_URL",
    "NODE_ENV",
    "SKIP_ENV_VALIDATION",
    "PORT",
    "VERCEL",
    "VERCEL_URL"
  ]
}

[multiplehats]

Just wanted to add my 2 cents, I'm using @dotenvx/dotenvx which is a successor (?) to dotenv. Honesty, these two packages confuse me, but anyway. If you use dotenvx, you don't have to traverse any paths such as ../../ you can just specify the root of you repo. In this example, my script is located in ./other/scripts/load-env.ts

import { readFileSync, writeFileSync } from 'fs';
import { config } from '@dotenvx/dotenvx';

// Assumes it is called from root and run from tooling/env/src/load-env.ts

const main = () => {
	const keyEnv = config({ path: './.env.key' });

	// Use no default so that our dotenv calls don't go down the vault path
	const DOTENV_KEY = keyEnv.parsed?.DOTENV_KEY ?? process.env.DOTENV_KEY; // If DOTENV_KEY not found in .env.key, get it from process.env

	if (!DOTENV_KEY) {
		let hasDotEnv = false;
		try {
			hasDotEnv = !!readFileSync('./.env');
		} catch {
			/* empty */
		}
		if (hasDotEnv) {
			console.log('Found .env file, skipping .env.vault load');
			return;
		}

		throw new Error('No DOTENV_KEY found in .env.key or process.env');
	}

	const envs = config({ DOTENV_KEY, path: './.env.vault' }); // If DOTENV_KEY found, get the .env.vault, load it to get shared env variables

	if (!envs.parsed || Object.keys(envs.parsed).length === 0) {
		throw new Error('No .env.vault found or it is empty');
	}

	try {
		// This env is needed for the app
		writeFileSync(
			'../../.env',

			'# This file is generated by /other/scripts/load-env.ts\n' +
				'# DO NOT ATTEMPT TO EDIT THIS FILE\n' +
				Object.entries(envs.parsed)
					.map(([key, value]) => `${key}=${value}`)
					.join('\n')
		);

		console.log('Successfully loaded .env.vault to .env');
	} catch (error) {
		console.error('Failed to load .env.vault to .env', error);
	}
};

main();