Terramate using github's terraform provider

[jonatiao]

Hello Everyone,
Thx for the awesome tool. I just learned about it while researching and feel it's easier to use than terragrunt and terraform vanilla while using it inside CI/CD pipelines.

I wonder if anyone here had any experience to share regarding automating github's enterprise/organization as code. I would appreciate any advice that could be given to me regarding file structure and even if this is doable using terramate. I'm reaching you guys in this discussion tool because the examples I found are always related to environments and cloud providers.. so I'm not sure if the tool is suited for what I'm trying to achieve.

The idea is to automate the creation and configuration of Organizations, Teams, management of People, Repositories and so on. Everything that is possible using github's terraform provider:
https://registry.terraform.io/providers/integrations/github/latest/docs

We will have an github cloud Enterprise, which will have N Organizations and N repos inside each Organization. The interest in Terramate is because of Stacks and Globals and I would like to know if it can address issues like:

1. All Organizations must have repository fork disabled and I would like to address this globally instead of changing each Organization Settings in a specific file. Still, besides the global file it would be good if I could override it somehow in a Organization specific file.
   https://registry.terraform.io/providers/integrations/github/latest/docs/resources/organization_settings
2. If I change something globally, terramate would apply the change for all subsequent(sibling) structures.
   Organizations/global_configs.tm.hcl # global
   Organizations/My-Org/configs.tm.hcl # overrides some global configs

I came up with something like the bellow.(not considering yet terramate's own structure.. still playing around)

# Github Enterprise cloud

# An Enterprise has its own configurations but cannot be automated since there's no API for it
# An Enterprise contains several Organizations
EnterpriseA/
  # Each Organizations have it's own configs, Teams, People, Roles, Rulesets, Repositories and so on. There's no such thing as
  # "a config to rule em all", still, I would like to address this using terramate.
  # Like, define a configuration in the Organizations root that will be shared for each Org somehow.
  Organizations/
    # There's no such thing as global stuff for all organizations, so basically all files here will be merged into all(existing files) Orgs,
    # like, 'global_teams_and_people.tm.hcl' is supposed to be used to create the same Teams inside all Orgs.
    global_orgs_configs.tm.hcl # Configs shared by all Orgs, like, all Orgs will have repository forks disabled or all Orgs can only create private Repos.
    global_repos_configs.tm.hcl # Configs shared by all Repos inside all Orgs, like, all repos inside all Orgs will have a topic(labels) called "managed-by-terramate".
    global_teams_and_people.tm.hcl # Teams and its People that will exist in all Orgs. Like 'Automation-Team'.
    OrganizationA/
      # Some configs defined in the global files can be overriden by something specific definied in any file here.
      org_config.tm.hcl # Specific configs for this Org + merged 'global_orgs_configs.tm.hcl'
      teams_for_this_org.tm.hcl # A Team(and its People) that only exists in OrganizationA + merged 'global_teams_and_people.tm.hcl'. Like 'OrgA-Sec-Team'
      repos_general_configs.tm.hcl # Configs shared by all Repos inside OrganizationA, like, all Repos will have a topic(labels) called "TeamA-managed"
      Repositories/
        RepositoryA/
          repo_config.tm.hcl # Specific Repo config + merged 'repos_general_configs.tm.hcl'. Like the Topic "TeamA-managed" existing in the 'repos_general_configs.tm.hcl' plus the specific Topic for this repo 'Java'.
          team_config.tm.hcl # Specific Team(Java-Team) + merged 'global_teams_and_people.tm.hcl' + merged 'teams_for_this_org.tm.hcl' = 'Java-Team' + 'Automation-Team' + 'OrgA-Sec-Team'
        RepositoryB/
   
    My-OrganizationB/
      org_config.tm.hcl
      teams_for_this_org.tm.hcl
      repos_general_configs.tm.hcl
      Repositories/
        my-java-repo/
          repo_config.tm.hcl
          team_config.tm.hcl
        my-python-repo/
          repo_config.tm.hcl
          team_config.tm.hcl

Is this feasible using Terramate?
Any tips where I should start or a small example?

[soerenmartius]

Hi @jonatiao,

Thanks for reaching out. Funny coincidence. The first architecture we ever built on top of Terramate is called iac-github, which we use to manage our own GitHub organization. We don't use GitHub Enterprise and only manage a single organization but our approach matches quite a bit with what you want to achieve.

This is how our structure looks like currently:

.
├── README.md
├── mineiros_ghac.tm.hcl  # project-wide configuration
├── modules                        # vendored modules
├── stacks
│   ├── config_organization_blocked_users.tm.hcl
│   ├── config_organization_collaborators.tm.hcl
│   ├── config_organization_members.tm.hcl
│   ├── config_organization_owners.tm.hcl
│   ├── organization
│   │   ├── members
│   │   │   ├── _terramate_generated_backend.tf
│   │   │   ├── _terramate_generated_main.tf
│   │   │   ├── _terramate_generated_providers.tf
│   │   │   └── stack.tm.hcl
│   │   └── teams
│   │       ├── SomeTeam
│   │       │   ├── SomeNestedTeam
│   │       │   │   ├── _terramate_generated_backend.tf
│   │       │   │   ├── _terramate_generated_main.tf
│   │       │   │   ├── _terramate_generated_providers.tf
│   │       │   │   ├── config.tm.hcl
│   │       │   │   └── stack.tm.hcl
│   │       │   ├── AnotherNestedTeam
│   │       │   │   ├── _terramate_generated_backend.tf
│   │       │   │   ├── _terramate_generated_main.tf
│   │       │   │   ├── _terramate_generated_providers.tf
│   │       │   │   ├── config.tm.hcl
│   │       │   │   └── stack.tm.hcl
........
│   │       │   ├── _terramate_generated_backend.tf
│   │       │   ├── _terramate_generated_main.tf
│   │       │   ├── _terramate_generated_providers.tf
│   │       │   ├── config.tm.hcl
│   │       │   └── stack.tm.hcl
......
│   └── repositories
│       ├── config_repository_defaults.tf.tm.hcl
│       ├── engineering
│       │   ├── cli
│       │   │   ├── terramate
│       │   │   │   ├── _terramate_generated_backend.tf
│       │   │   │   ├── _terramate_generated_main.tf
│       │   │   │   ├── _terramate_generated_providers.tf
│       │   │   │   ├── config.tm.hcl
│       │   │   │   └── stack.tm.hcl
│       │   │   ├── terramate-action
│       │   │   │   ├── _terramate_generated_backend.tf
│       │   │   │   ├── _terramate_generated_main.tf
│       │   │   │   ├── _terramate_generated_providers.tf
│       │   │   │   ├── config.tm.hcl
│       │   │   │   └── stack.tm.hcl
.............
└── terramate.tm.hcl

Since we wanted to open-source this for some time anyway, I'd ask you to give me some time to align this internally. Perhaps we can make this happen sooner rather than later so you can use it for your organization instead of building from scratch (it also comes with pre-configured pipelines for GitHub Actions).

[jonatiao]

This would be awesome!
thx @soerenmartius ^^
I really appreciated.

[soerenmartius]

I'll keep you posted!