.env files in services/* directory breaks runtime start

[leorossi]

In a Platformatic Runtime, if thre is a .env file inside a service directory, seems like it is loaded in place of the root .env one.
This happens only if the service's platformatic.json file contains a reference to an env variable like

{
  "plugins": {
    "typescript": "{PLT_MY_SERVICE_TYPESCRIPT}"
  }
}

Repro:

$ node packages/create-platformatic/create-platformatic.mjs

Select Application and create the first (and only service)

Once finished ensure that the runtime starts correctly and stop it. Then

$ cd $RUNTIME_ROOT_DIR
$ touch services/YOUR_SERVICE_NAME/.env

Start again and you'll see that an error occurs, complaining that an env variable is missing.

The .env file inside the service should be either ignored or loaded together with the root one

[bobss24]

I can take a look at this, @leorossi

[leorossi]

Great, thanks!

[bobss24]

This is the screenshot:

[bobss24]

and this:

[bobss24]

@leorossi, what's the best way to debug this?

I am using VSCode. Are there platformatic-specific instructions somewhere? :)

[bobss24]

@leorossi, I am going to attempt attaching a proper debugger but I don't have the cycles, at the moment.

If you have any tips for me for setting up a Node.js debugging environment for plt - or if there are instructions elsewhere, pls let me know!

cc @mcollina!

[mcollina]

You can run https://github.com/platformatic/platformatic/blob/main/packages/cli/cli.js with --inspect --inspect-brk and then open chrome dev tools

[bobss24]

I believe the error comes from the ConfigManager.. here:

packages/config/lib/manager.js

I'll look deeper tomorrow! :)

[bobss24]

The problem is a worker thread starts looking for a .env file up the tree from the starting directory of the service (services/name-of-service) and when it finds the "empty" .env in the service's dir, it gives up looking elsewhere, thus causing the env vars in the project root directory's .env to not be resolved, causing the error!

One solution could be to read all the .envs and then combine them.?

[leorossi]

Or, since the worker thread runs only in runtimes, the .env file search should ignore the service's directory, so only the runtime root's one will be loaded.
I like also the idea of combining the envs, but it might be confusing for the user? WDYT @mcollina ?

[bobss24]

Or, since the worker thread runs only in runtimes, the .env file search should ignore the service's directory, so only the runtime root's one will be loaded. I like also the idea of combining the envs, but it might be confusing for the user? WDYT @mcollina ?

I like the idea here @leorossi .. however, if you see, the .env in the service's folder should get preference and the other .envs loaded later. But if we can do it, the root dir's .env can be the only one to be supported and we can safely ignore the service dir's .env What do you think?

It would be simpler, I think, for the user to know that the only .env to be read is the root dir's.. but it is an open question at this point what our strategy of parsing .envs should be.. 🤔🙂

I am in favor of supporting just 1 .env, that in the project's root dir...❤️ wdyt @mcollina ??

[mcollina]

Or, since the worker thread runs only in runtimes, the .env file search should ignore the service's directory, so only the runtime root's one will be loaded.
I like also the idea of combining the envs, but it might be confusing for the user? WDYT @mcollina ?

I agree with not loading those.

[bobss24]

I'll try to check in a fix tomorrow..🙂

[bobss24]

I think I am too new to tackle this.. 🙁!

[bobss24]

Hi @leorossi , I have tried and there's common logic for reading config -- that will need to be refactored, somewhat!

[bobss24]

Please guide Me..❤️ 🙏 @mcollina

[mcollina]

Unfortunately we have no time to give you step-by-step instructions.

[bobss24]

Unfortunately we have no time to give you step-by-step instructions.

Not asking for step-by-step instructions, @mcollina :)

Just looking for advice but never-mind then 🙂!

[bobss24]

This env-related code is hard-to-parse @mcollina :) . I'll refactor what I can 🙂!

[bobss24]

@leorossi, it can also be that we always default to the current working dir for the .env reading.

We can also make it an option in the configuration file for the service(for the user to specify), or have the runtime pass it to the services as they start up..this will likely not be trivial. Nor are the other options. We can::

1. Have the user specify the path to the .env file
2. Have the runtime make this decision and pass it through to the services as they start up
3. Always parse the CWD's .env and not search anywhere!

Let me know what we prefer!

@mcollina 👍

[mcollina]

1. no
2. yes
3. no

[bobss24]

1. no

2. yes

3. no

Thanks, @mcollina. I'll begin work on this tomorrow. Maybe I should pick up to 3 tasks & work on them in parallel!
🙂

[leorossi]

@bobss24 take a look at https://github.com/leorossi/dotenv-tool it's already imported in runtime package

[bobss24]

thx @leorossi! ❤️

I'll take a look when I am back from vacation! 🙂 ❤️

[bobss24]

I'm back.❤️ will take a look tomorrow, if all goes to plan.. :)

[bobss24]

@bobss24 take a look at https://github.com/leorossi/dotenv-tool it's already imported in runtime package

@leorossi thx for the suggestion here. I am not understanding how to use this. If we were collating the two .env's, I would have found your tool quite helpful for it. Here, the problem seems to be different wherein, instead of combining the two environment var config files, it stops looking for others. We have already decided to use the top-level .env in this case, ignoring what the service has in its directory!🍀