Use of TF_TOKEN_ Environment Variable for External Module Registry Access

[ben-balk-oc]

According to Terraform CLI documentation, version v1.2.0 and later support use of environment variable credentials to make requests to specific hostnames (https://developer.hashicorp.com/terraform/cli/config/config-file#environment-variable-credentials). We're hoping to migrate all of our workspaces from Terraform Cloud into Terrakube and it would be significantly easier if we had a way to pull modules from our current Terraform Cloud module registry while we convert to either a Terrakube registry or update our modules to use Github repo sources.

Should this work by simply setting an environment variable named TF_TOKEN_app_terraform_io in the workspace(s)? I can't seem to get that to work using a VCS workflow workspace regardless of which version I specify.
Example error from terraform plan:

Failed to retrieve available versions for module "core" (core.tf:1) from
app.terraform.io: error looking up module versions: 401 Unauthorized.

I have used a modified template to print the environment variables in my Terrakube shell environment and can see the appropriate variable for TF_TOKEN_app_terraform_io.

[alfespa17]

I never used the TF_TOKEN_app_terraform_io environment variable but if you add it to the workspace or as a global variable it will be injected when the job is running.

In the end Terrakube is just running the Terraform CLI and injecting the environment variables in a new process inside the container, which terraform version are you using? did you try running the plan/apply just with the terraform CLI to check if TF_TOKEN_app_terraform_io works correctly locally?

[ben-balk-oc]

I've confirmed that setting TF_TOKEN_app_terraform_io locally and running terraform init succeeds in downloading modules from the TF Cloud registry. Locally, I've tested with version 1.2.9.

In Terrakube, I have tried specifying versions 1.2.9, 1.3.10, 1.6.5 without luck in downloading modules.

[alfespa17]

Ok I will do some test, if putting the envirnoment variable does not work, we can add the app.terraform,io token to the internal credentials file, that is not like a big change it could be quite simple to add in the code

[alfespa17]

This is a bug we are not including environment variables when running terraform init internally.

Terrakube code set environment variables to empty before running the init

terrakube/executor/src/main/java/org/terrakube/executor/service/terraform/TerraformExecutorServiceImpl.java

Line 379 in a96747c

terraformProcessData.setTerraformEnvironmentVariables(new HashMap<>());

Because there is a validation in the terraform library that check that terraform init should not include environment variables, I dont remember why we add this validation to be honest.

https://github.com/AzBuilder/terraform-spring-boot/blob/452afdbf89db196c16618940a7c13156acbd810f/terraform-client/src/main/java/org/terrakube/terraform/TerraformClient.java#L101

https://github.com/AzBuilder/terraform-spring-boot/blob/452afdbf89db196c16618940a7c13156acbd810f/terraform-client/src/main/java/org/terrakube/terraform/TerraformClient.java#L228

[alfespa17]

I have created #631 to fix this issue.

[ben-balk-oc]

That is great news. Thank you!