Enable running on Terraform commands from Docker container

[swanysimon]

Installing Terraform on CI systems can be a bit of a pain, and often I wish that I could keep a pure Python environment to run with. My solution was to run all the Terraform commands in a hashicorp/terraform Docker image. It was useful enough to me that I've copy-pasted it into a couple repos now and figured it was worth bringing up here as a feature request.

The simple implementation is pretty straightforward, but generalizing for all Terraform uses is potentially onerous. It's possible this request doesn't fit the general-purpose needs of this project because of this complexity.

My current solution relies on the docker-py module to manage a container and then shells out when running commands by subclassing TerraformTest. There's no reason it has to be designed this way; I just needed something to work at the time. It also relies on the module under test to be in a sub-directory of the module it may be consuming. All in all, the solution I have here is pretty bad but I figured it was worth including.

import docker
import os
import pytest
from pathlib import Path, PurePath


@pytest.fixture(scope="session")
def container(base_dir, repository_root, test_module, terraform_version):
    """Starts a Docker container in which Terraform commands can be run.

    This container is made to be able to run methods from the TerraformTest
    module, which involves mounting the temp directory. Some big assumptions are
    made with the container:

    - You are using AWS modules in your Terraform code
    - Your test module is in a sub directory of the repository root
    - Your test module depends on the module that lives at the repository root
    """
    client = docker.from_env()
    container_run_args = {
        "entrypoint": "/bin/sh",
        "user": f"{os.getuid()}:{os.getgid()}",
        "environment": {
            "AWS_REGION": os.environ.get("AWS_REGION", "us-east-1"),
        },
        "volumes": {
            repository_root: {"bind": base_dir, "mode": "rw,Z"},
            # The tftest module needs the temp directory to store plan output for parsing
            # into Python objects. It's easiest to just mount the directory than to mess
            # around with the class object
            tempfile.gettempdir(): {"bind": tempfile.gettempdir(), "mode": "rw,Z"},
        },
        "working_dir": os.path.join(
            base_dir, PurePath(test_module).relative_to(repository_root)
        ),
        "detach": True,
        "remove": True,
        "tty": True,
    }

    # Place AWS credentials in the Docker image
    if os.path.isdir(Path.home() / ".aws"):
        container_run_args["volumes"][Path.home() / ".aws"] = {
            "bind": "/.aws",
            "mode": "ro",
        }
    else:
        env = container_run_args["environment"]
        env["AWS_ACCESS_KEY_ID"] = os.environ["AWS_ACCESS_KEY_ID"]
        env["AWS_SECRET_ACCESS_KEY"] = os.environ["AWS_SECRET_ACCESS_KEY"]

    container = client.containers.run(
        f"hashicorp/terraform:{terraform_version}", **container_run_args
    )

    yield container

    container.kill(signal=9)

I then subclass TerraformTest to run all Terraform commands in the container:

from tftest import TerraformTest


class TerraformDockerTest(TerraformTest):
    # Base class: https://github.com/GoogleCloudPlatform/terraform-python-testing-helper/blob/v1.5.4/tftest.py
    def __init__(self, module_dir, root_module, container, base_dir):
        super(TerraformDockerTest, self).__init__(module_dir, basedir=base_dir)
        self.terraform = "docker"
        self.cmd = "exec"
        self.container = container

    def execute_command(self, cmd: str, *cmd_args: tuple):
        # This will always print "exec" as the command in the debug logging, unfortunately
        args = [self.container.name, "terraform", cmd, *cmd_args]
        return super().execute_command(self.cmd, *args)

Then creating a plan fixture is nearly the same as with the base class:

import pytest
import TerraformDockerTest


@pytest.fixture(scope="session")
def plan(base_dir, container, repository_root, test_module, variables):
    tf = TerraformDockerTest(test_module, repository_root, container, base_dir)
    tf.setup()
    return tf.plan(output=True, tf_vars=variables)

[ludoo]

I'm trying to wrap my head around this, and mostly failing. Lots of context switches during the day definitely don't help. :)

Would an extra class in the module similar to your TerraformDockerTest be enough to fit your needs? At a simple level, you just need to run Terraform via docker instead of a regular executable when invoking a test, right?

Or am I completely off the mark?

[swanysimon]

I think you're on the mark!

A class similar to the one I put above would be exactly what I'm looking for, but there are a couple things to consider for other users of the module, namely how do you know what parts of the user's environment need to be included in the container. I mostly work with AWS resources, so I wrote in the logic to get credentials into the container. It could be better to include all of the relevant credentials for all resource providers in the container.

I also have some assumptions I made with the volumes on the container: that the module under test was a subdirectory of all the modules that it was testing. It may be useful to think a little harder than I did about how volumes are mounted into the container.

I'm sure you'll come up with a more elegant solution to this than I did :)

[AgustinRamiroDiaz]

@swanysimon why not use containers all the way? Like having a container with python, terraform and this module into the image, and then running the tests inside the container