[Enhancement]: DockerClient Java equivalent

[christophwille]

Problem

I saw https://www.tomcools.be/post/june-2022-testcontainer-imagebuilder/ - this getDockerClient / commitCmd / pushImageCmd dance would be exactly what I wanted to do with a MSSQL Container - start, apply EF migration, seed some data, persist & push to registry for developers to pull.

Solution

I didn't see the functionality on .NET side (the obvious .DockerClient didn't exist), replicating the Java functionality would be great.

Benefit

Building container images on the fly using TestContainer.NET

Alternatives

tbh, haven't yet found a good one, eg via bacpac

Would you like to help contributing this enhancement?

No, in the sense that I don't know the internals of TestContainers.NET and likely wouldn't be able to contribute a PR.

[HofmeisterAn]

Indeed, Testcontainers for .NET does not expose the Docker client like Java does. docker-java is more mature and stable. Docker.DotNet does not contain all Docker Engine APIs and lacks various features compared to Java's implementation. However, in the future, I would like to support similar capabilities around the Docker client as Testcontainers for Java offers. I am currently experimenting with generating the client, or at least the model classes, from the OpenAPI spec.

That being said, there are two workarounds you can use in the meantime. Either way, you can use the image builder implementation to build an image first (the build cannot push the image, though). Or you can create your own Docker.DotNet client instances utilizing Testcontainers' auto-discovery mechanism and hope the client exposes the necessary APIs already.

using var dockerClientConfiguration = TestcontainersSettings.OS.DockerEndpointAuthConfig.GetDockerClientConfiguration(ResourceReaper.DefaultSessionId);
using var dockerClient = dockerClientConfiguration.CreateClient();
await dockerClient.Images.CommitContainerChangesAsync(new CommitContainerChangesParameters());

[christophwille]

I gave it a try - commit + push (commit only means after Testcontainer exits the image is gone too):

    using var dockerClientConfiguration = TestcontainersSettings.OS.DockerEndpointAuthConfig.GetDockerClientConfiguration(ResourceReaper.DefaultSessionId);
    using var dockerClient = dockerClientConfiguration.CreateClient();

    await dockerClient.Images.CommitContainerChangesAsync(new Docker.DotNet.Models.CommitContainerChangesParameters
    {
        ContainerID = msSqlContainer.Id,
        RepositoryName = "chris/test",
        Tag = "guesswhat",
    });

    await dockerClient.Images.PushImageAsync("chris/test:guesswhat", new Docker.DotNet.Models.ImagePushParameters(),
        new Docker.DotNet.Models.AuthConfig(), new DockerPushProgress());

// via https://medium.com/the-aws-coder/pushing-a-local-docker-image-to-amazon-ecr-repository-using-net-5-c-and-aws-ecr-sdk-d5d5ce7c338a
class DockerPushProgress : IProgress<JSONMessage>
{
    public void Report(JSONMessage value)
    {
        if (value.Progress == null)
            Console.WriteLine($"Progress {value.Status}");
        else
            Console.WriteLine($"Progress. Status {value.Status}, ID {value.ID}, Current {value.Progress.Current}, Total {value.Progress.Total}");
    }
}

[HofmeisterAn]

after Testcontainer exits the image is gone too

Please make sure the new image does not contain the resource reaper session label, respectively set it to Guid.Empty. Otherwise, Ryuk will clean it up. If that works, we can think about extending the IContainer interface. Note that this is only necessary to keep the image among different test sessions (runs).

Edit: This builder configuration labels the resources and tells Ryuk to track and clean them up. If the label gets inherited, you need to somehow override the Guid as mentioned above.

[christophwille]

Even if I use Guid.Empty instead of ResourceReaper.DefaultSessionId on line 1 the things go away. Or are the labels in other places too (unfamiliar with this concept). I want the container to go away, but the committed image should stay.

[HofmeisterAn]

Even if I use Guid.Empty instead of ResourceReaper.DefaultSessionId on line 1 the things go away.

You need to override the resource label. Each resource, such as a container, image, network, or volume, is labeled to track it. I believe that when you commit a container, the new image inherits its labels.

Please try disabling the cleanup for the container and check if this also keeps the image.

_ = new MsSqlBuilder().WithCleanUp(false).Build();

[christophwille]

I just made the alternative I outlined in the initial comment work: I use Testcontainers to create the database via EF migrations, then export via DacFx

    var dac = new DacServices(connectionString);
    dac.ExportBacpac("export.bacpac", "mydb");

Now, I use the approach from https://www.kenmuse.com/blog/devops-sql-server-dacpac-docker (replacing dacpac with bacpac) with a few minor changes (ie the link to sqlpackage is no longer current, and USER root is needed for apt-get) to generate a new Docker image. Bonus of this approach: smaller image.