terramate generate: nothing to do

[az-z]

Hello,
i have a git branch that is checked out using git worktree into a separate directory (WWWPRIV-65-iac-vm).
I added 2 files:
to disable integration with git - terramate.tm.hcl
to produce _provider.tf in each stack. - provider.tm.hcl

the log:

[az@fedora SOPRO]$ terramate generate 
Nothing to do, generated code is up to date

[az@fedora SOPRO]$ terramate generate  --disable-check-git-untracked --disable-check-git-uncommitted
Nothing to do, generated code is up to date

the project structure:

pwd
/home/az/repos/iac/WWWPRIV-65-iac-vm/Clients/SOPRO

where git root is at WWWPRIV-65-iac-vm

[az@fedora SOPRO]$ ls -l
total 17
 drwxrwx---. 1 root vboxsf    0 Jan  9 14:16 modules
-rwxrwx---. 1 root vboxsf  925 Jan 19 16:45 provider.tm.hcl
drwxrwx---. 1 root vboxsf    0 Jan 18 10:12 stack
-rwxrwx---. 1 root vboxsf   52 Jan 19 16:49 terramate.tm.hcl

cat provider.tm.hcl

generate_hcl "_provider.tf" {
  content {
            terraform {
              backend "remote" {
                organization = "XXX"
                workspaces {
                  name = "XXXX"
                }
              }

              required_providers {
               aws = {
                 source = "hashicorp/aws"
                 version = ">= 3.65.0"
               }
              }
            }

            provider "aws" {
              profile = var.aws_profile
              region =  var.aws_region
            }
}
}

[az@fedora SOPRO]$ cat ./terramate.tm.hcl 
#  no dependency on git
terramate {
  config {
  }
}

Question:
what am I missing ?

[az-z]

even if i remove the .git subdir the log is still the same.

[az-z]

i checked out the "live" project and in order to make TM to generate anything, i had to modify stacks/config.tm.hcl - as mentioned in the read.me.

which brings 2 questions:

1. is there a "dry run" mode for the generate
2. why placing provider.tm.hcl in the root did not trigger TM

Thank you.

[i4ki]

Hi @az-z

The Terramate does not support git worktree yet.
We discussed this use case internally and it seems it's just a small change to make Terramate work with worktree. We never used "worktree" so this use case wasn't on our radar.
We will implement this for next release.

Thanks for reporting it.

About your other questions:

1. is there a "dry run" mode for the generate
   Not yet.

2. why placing provider.tm.hcl in the root did not trigger TM
   The reason is that Terramate doesn't work with worktrees. The technical issue is that we ask git for the root of the project and it's returning a path inside the /.git/worktrees and then obviously there is not the checked out code.

[az-z]

Worktree - understood. Looking forward for the implementation. Do you have an eta? Generate - but I thought that I can disable the git integration by placing an empty configuration file?

…

On Fri, Jan 20, 2023, 06:56 伊亜希 ***@***.***> wrote: Hi @az-z <https://github.com/az-z> The Terramate does not support git worktree yet. We discussed this use case internally and it seems it's just a small change to make Terramate work with worktree. We never used "worktree" so this use case wasn't on our radar. We will implement this for next release. Thanks for reporting it. About your other questions: 1. is there a "dry run" mode for the generate Not yet. 2. why placing provider.tm.hcl in the root did not trigger TM The reason is that Terramate doesn't work with worktrees. The technical issue is that we ask git for the root of the project and it's returning a path inside the /.git/worktrees and then obviously there is not the checked out code. — Reply to this email directly, view it on GitHub <#811 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGHHXLRBPQHAPK4OECKUA3WTJ4N7ANCNFSM6AAAAAAUA3WBLA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[i4ki]

go install github.com/mineiros-io/terramate/cmd/terramate@i4k-fix-worktree

:-)

[az-z]

@i4ki , I got:

go install github.com/mineiros-io/terramate/cmd/terramate@i4k-fix-worktree
go: downloading github.com/mineiros-io/terramate v0.2.10-0.20230120133623-d3377d53f9b6
[az@fedora SOPRO]$ terramate --version
0.2.9
[az@fedora SOPRO]$ terramate generate 
2023-01-20T13:29:08-05:00 WRN ignoring root config error="root config found outside root dir" projectRoot=/mnt/Documents/repos/iac/WWWPRIV-65-iac-vm rootConfig=/mnt/Documents/repos/iac/WWWPRIV-65-iac-vm/Clients/SOPRO
2023-01-20T13:29:09-05:00 WRN error="/mnt/Documents/repos/iac/WWWPRIV-65-iac-vm/Clients/SOPRO/terramate.tm.hcl:2,1-10: `terramate` block is only allowed at the project root directory" action=TerramateParser.checkConfigSanity()
Nothing to do, generated code is up to date

I don't think this is right - it was installing terramate v0.2.10- but picked up 0.2.9

[az-z]

at the same time, it did produce the error output, which it didn't do before the upgrade...

[az-z]

Actually. I was too quick to jump to the incorrect conclusion.

The error makes sense.
Let me step back and explain:

I have one repo that covers a growing number of client specific iac implementations. Each "client" is a separate cloud account.

repo:
 | - Client1
 |       | - common
 |       |            |      - ec.tfvars  
 |       |            |      - main.tf
 |       | - stack
 |       |       - Env
 |       |            | - ec2
 |       |            |      - ec.tfvars  
 |       |            |      - main.tf
 |       |            |  - S3
 |       |            |      - s3.tfvars    
 |       |            |      - main.tf
......
 | - Client#
 |       | - common
 |       |            |      - ec.tfvars  
 |       |            |      - main.tf
 |       | - stack
.........

it is clear that common is just another stack. But what is important is that all work is done within a single "CLIENT#" stack from the TM's standpoint.

I do not have a solid usecase to cover all clients yet, since this is all is in active development phase. One day, i'll probably be able to create modules to cover all clients, but right now it is not the case.

What is the recommended way to leverage the TM ( to generate client specific "provider.tf", for instance) in this situation?

[i4ki]

Hi @az-z

For better isolation, we recommend a separate Terramate repo for each client, but if you do need/want to organize it this way, then have in mind that Terramate will treat the entire repository as a single project.

In the setup you showed, you can cd into the client (eg.: Client1) and execute Terramate from there and it will only execute stacks that could be reachable from that directory (it never goes up).

Note: the only exception is for the terramate generate command, which will generate for all stacks, always, independent of the directory from where it is ran.

So if you have the tree below:

.
├── client-1
│   ├── stack-1
│   │   └── stack.tm.hcl
│   └── stack-2
│       └── stack.tm.hcl
├── client-2
│   ├── stack-1
│   │   └── stack.tm.hcl
│   └── stack-2
│       └── stack.tm.hcl
└── terramate.tm.hcl

then:

$ cd client-1
$ terramate list
stack-1
stack-2
$ terramate run -- pwd
/mnt/example/client-1/stack-1
/mnt/example/client-1/stack-2

You can also tag the stacks with client names:

$ cd ..
$ cat client-1/stack-1/stack.tm.hcl
stack {
  tags        = ["client1"]
}
$ cat client-1/stack-2/stack.tm.hcl
stack {
  tags        = ["client1"]
}

Then you can execute even from the root dir but filter by clientX stacks.

$ terramate --tags client1 list
client-1/stack-1
client-1/stack-2

$ terramate --tags client1 run -- pwd
/mnt/example/client-1/stack-1
/mnt/example/client-1/stack-2

We may better support multiple projects in the same repository in the future but this is not a use case that we at Mineiros would recommend, then it's a low priority for us at this point.