Testcontainers with azurite container (podman on windows)

[gb-nb]

Hello,

I'm trying to write a test which has a dependency on azure storage, so I was experimenting with using the azurite image since that would make it easy to reproduce the condition(s) both locally and in our CI/CD pipelines.
I had some trouble with getting testcontainers to work with podman. But setting TESTCONTAINERS_RYUK_DISABLED=true on the environment seemed to work for now. I'm using podman on Windows 10.

I managed to get the Azurite container working with the method below. (the test containers azurite package did not work for me).
It works in the sense that it spins up. The only thing is I can't reach it when doing a request.

container = new ContainerBuilder()
                    .WithImage("mcr.microsoft.com/azure-storage/azurite")
                    .WithPortBinding(10000, true)
                    .WithPortBinding(10001, true)
                    .WithPortBinding(10002, true)
                    .WithOutputConsumer(this.consumer)
                    .WithWaitStrategy(Wait.ForUnixContainer()
                      .UntilMessageIsLogged("listening"))
                    .Build();

If I start a container with the same latest azurite image from podman desktop manually, I can successfully make a request.

What I've seen inspecting the deployment is that the main difference is that testcontainers deploys it with hostip 0.0.0.0.
On the left is the test containers deployment, on the right my manual deployment from podman interface.

Does someone know what the culprit might be? and if I can make it visible what commands / request are being done to podman? so that I can rule out any difference(s).

[HofmeisterAn]

I managed to get the Azurite container working with the method below. (the test containers azurite package did not work for me).
It works in the sense that it spins up. The only thing is I can't reach it when doing a request.

Which hostname do you use to connect to the container? Using localhost (or any static value) is not recommended because, depending on the Docker host and operating system, the address varies.

What I've seen inspecting the deployment is that the main difference is that testcontainers deploys it with hostip 0.0.0.0.
On the left is the test containers deployment, on the right my manual deployment from podman interface.

Docker has a known issue where it assigns different random host ports to the IPv4 and IPv6 addresses. To avoid encountering this issue, Testcontainers binds the port to the IPv4 address. By using the container's Hostname property, you ensure that you resolve the correct IP address based on your Docker host and the assigned random port. This relates to the following issues (features):

• [Enhancement]: Resolve port bindings according to IPv4 and IPv6 #825
• Different port number are getting mapped on ipv4 and ipv6 for same expose port moby/moby#42442
• ListContainersAsync response contains inconsistent public host ports dotnet/Docker.DotNet#565
• Different port number are getting mapped on ipv4 and ipv6 for same expose port moby/moby#42442 (comment)

To resolve the correct port for localhost (whether it's IPv4 or IPv6), we need to implement #825.

However, when using Testcontainers, it is always recommended to use the Hostname property to resolve the correct Docker host address. If you follow the best practices, your configuration should work. Always utilize _container.Hostname and _container.GetMappedPublicPort(containerPort) to obtain the correct endpoint.

[HofmeisterAn]

The configuration looks fine, I do not see any obvious issue. I would expect connecting to _container.GetConnectionString() to work. Can you add the inspect result of podman run -p 10000 -p 10001 -p 10002 mcr.microsoft.com/azure-storage/azurite azurite -l /data --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 too? Since, you are saying it is working running it manually, maybe we can compare the diff of both results and figure out what goes wrong. Sorry, I cannot help more, but I do not have a Podman configuration here.

[gb-nb]

I have run the exact command manually, which works with both random host port and exact mapping.

Here is the podman inspect output

[
     {
          "Id": "1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad",
          "Created": "2023-10-06T10:56:01.057276698+02:00",
          "Path": "docker-entrypoint.sh",
          "Args": [
               "azurite",
               "-l",
               "/data",
               "--blobHost",
               "0.0.0.0",
               "--queueHost",
               "0.0.0.0",
               "--tableHost",
               "0.0.0.0"
          ],
          "State": {
               "OciVersion": "1.0.2-dev",
               "Status": "running",
               "Running": true,
               "Paused": false,
               "Restarting": false,
               "OOMKilled": false,
               "Dead": false,
               "Pid": 6274,
               "ConmonPid": 6272,
               "ExitCode": 0,
               "Error": "",
               "StartedAt": "2023-10-06T10:56:01.349382299+02:00",
               "FinishedAt": "0001-01-01T00:00:00Z",
               "Health": {
                    "Status": "",
                    "FailingStreak": 0,
                    "Log": null
               },
               "CgroupPath": "/libpod_parent/libpod-1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad",
               "CheckpointedAt": "0001-01-01T00:00:00Z",
               "RestoredAt": "0001-01-01T00:00:00Z"
          },
          "Image": "8e896c90652a94722612eb7a7b28f5dbce3a334696ba81715a013f97e91d96a1",
          "ImageDigest": "sha256:f3e3a7eb821d7ca50a2c8f64cee2e2f759096dd69191e5065a385da46d1252c6",
          "ImageName": "mcr.microsoft.com/azure-storage/azurite:latest",
          "Rootfs": "",
          "Pod": "",
          "ResolvConfPath": "/run/containers/storage/overlay-containers/1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad/userdata/resolv.conf",
          "HostnamePath": "/run/containers/storage/overlay-containers/1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad/userdata/hostname",
          "HostsPath": "/run/containers/storage/overlay-containers/1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad/userdata/hosts",
          "StaticDir": "/var/lib/containers/storage/overlay-containers/1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad/userdata",
          "OCIConfigPath": "/var/lib/containers/storage/overlay-containers/1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad/userdata/config.json",
          "OCIRuntime": "crun",
          "ConmonPidFile": "/run/containers/storage/overlay-containers/1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad/userdata/conmon.pid",
          "PidFile": "/run/containers/storage/overlay-containers/1d778df499ef93843673ce644e147878997cf4afea4085060270186cd01ca4ad/userdata/pidfile",
          "Name": "condescending_mahavira",
          "RestartCount": 0,
          "Driver": "overlay",
          "MountLabel": "",
          "ProcessLabel": "",
          "AppArmorProfile": "",
          "EffectiveCaps": [
               "CAP_CHOWN",
               "CAP_DAC_OVERRIDE",
               "CAP_FOWNER",
               "CAP_FSETID",
               "CAP_KILL",
               "CAP_NET_BIND_SERVICE",
               "CAP_SETFCAP",
               "CAP_SETGID",
               "CAP_SETPCAP",
               "CAP_SETUID",
               "CAP_SYS_CHROOT"
          ],
          "BoundingCaps": [
               "CAP_CHOWN",
               "CAP_DAC_OVERRIDE",
               "CAP_FOWNER",
               "CAP_FSETID",
               "CAP_KILL",
               "CAP_NET_BIND_SERVICE",
               "CAP_SETFCAP",
               "CAP_SETGID",
               "CAP_SETPCAP",
               "CAP_SETUID",
               "CAP_SYS_CHROOT"
          ],
          "ExecIDs": [],
          "GraphDriver": {
               "Name": "overlay",
               "Data": {
                    "LowerDir": "/var/lib/containers/storage/overlay/685e6d7dc2f19164979ff3afce77526127e6171c94a9c7a18cdb99b5e939af1a/diff:/var/lib/containers/storage/overlay/ad228f1a30a5850bb83a6b5159d0394e7c3ad395cc787569ab46d82fe9095a38/diff:/var/lib/containers/storage/overlay/ffe5858a3eaf6c0d66623fc25959769504ecf79aba0f8fef443bc16e2f61a709/diff:/var/lib/containers/storage/overlay/79b4d7c4857868811d25cbf8dc5f874e85e622fa7f1053c08e0c69ed19cfa34d/diff:/var/lib/containers/storage/overlay/6e022bb2493c2647f04a9dc88770ac77b9d1da15e5f81576ae4c35785b1859d8/diff:/var/lib/containers/storage/overlay/8ab3f5a456b2a985ad1398d7159d781eeb7b35995a45bc35348bb706bebaa153/diff:/var/lib/containers/storage/overlay/6a38f7f43124527d98b6581a7270f51c5aa9bd24f419caff5b1f004b7c99d4a0/diff:/var/lib/containers/storage/overlay/f1417ff83b319fbdae6dd9cd6d8c9c88002dcd75ecf6ec201c8c6894681cf2b5/diff",
                    "MergedDir": "/var/lib/containers/storage/overlay/341b5b19af4e4a259ee4c4c0373b37d73bf2275ff37fbe5f44d8530cd9204224/merged",
                    "UpperDir": "/var/lib/containers/storage/overlay/341b5b19af4e4a259ee4c4c0373b37d73bf2275ff37fbe5f44d8530cd9204224/diff",
                    "WorkDir": "/var/lib/containers/storage/overlay/341b5b19af4e4a259ee4c4c0373b37d73bf2275ff37fbe5f44d8530cd9204224/work"
               }
          },
          "Mounts": [
               {
                    "Type": "volume",
                    "Name": "f98cbfb0747071592114aeb0f876157d5ce37bbd66b96177fab54a88801d30d0",
                    "Source": "/var/lib/containers/storage/volumes/f98cbfb0747071592114aeb0f876157d5ce37bbd66b96177fab54a88801d30d0/_data",
                    "Destination": "/data",
                    "Driver": "local",
                    "Mode": "",
                    "Options": [
                         "nodev",
                         "exec",
                         "nosuid",
                         "rbind"
                    ],
                    "RW": true,
                    "Propagation": "rprivate"
               }
          ],
          "Dependencies": [],
          "NetworkSettings": {
               "EndpointID": "",
               "Gateway": "10.88.0.1",
               "IPAddress": "10.88.0.49",
               "IPPrefixLen": 16,
               "IPv6Gateway": "",
               "GlobalIPv6Address": "",
               "GlobalIPv6PrefixLen": 0,
               "MacAddress": "3e:89:d6:3d:12:e4",
               "Bridge": "",
               "SandboxID": "",
               "HairpinMode": false,
               "LinkLocalIPv6Address": "",
               "LinkLocalIPv6PrefixLen": 0,
               "Ports": {
                    "10000/tcp": [
                         {
                              "HostIp": "",
                              "HostPort": "35235"
                         }
                    ],
                    "10001/tcp": [
                         {
                              "HostIp": "",
                              "HostPort": "36427"
                         }
                    ],
                    "10002/tcp": [
                         {
                              "HostIp": "",
                              "HostPort": "46647"
                         }
                    ]
               },
               "SandboxKey": "/run/netns/netns-49888d2f-bafe-77a0-1298-d530a680360a",
               "Networks": {
                    "podman": {
                         "EndpointID": "",
                         "Gateway": "10.88.0.1",
                         "IPAddress": "10.88.0.49",
                         "IPPrefixLen": 16,
                         "IPv6Gateway": "",
                         "GlobalIPv6Address": "",
                         "GlobalIPv6PrefixLen": 0,
                         "MacAddress": "3e:89:d6:3d:12:e4",
                         "NetworkID": "podman",
                         "DriverOpts": null,
                         "IPAMConfig": null,
                         "Links": null,
                         "Aliases": [
                              "1d778df499ef"
                         ]
                    }
               }
          },
          "Namespace": "",
          "IsInfra": false,
          "IsService": false,
          "KubeExitCodePropagation": "",
          "lockNumber": 0,
          "Config": {
               "Hostname": "1d778df499ef",
               "Domainname": "",
               "User": "",
               "AttachStdin": false,
               "AttachStdout": false,
               "AttachStderr": false,
               "Tty": false,
               "OpenStdin": false,
               "StdinOnce": false,
               "Env": [
                    "YARN_VERSION=1.22.19",
                    "NODE_ENV=production",
                    "NODE_VERSION=14.21.3",
                    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                    "TERM=xterm",
                    "container=podman",
                    "HOME=/root",
                    "HOSTNAME=1d778df499ef"
               ],
               "Cmd": [
                    "azurite",
                    "-l",
                    "/data",
                    "--blobHost",
                    "0.0.0.0",
                    "--queueHost",
                    "0.0.0.0",
                    "--tableHost",
                    "0.0.0.0"
               ],
               "Image": "mcr.microsoft.com/azure-storage/azurite:latest",
               "Volumes": null,
               "WorkingDir": "/opt/azurite",
               "Entrypoint": "docker-entrypoint.sh",
               "OnBuild": null,
               "Labels": null,
               "Annotations": {
                    "io.container.manager": "libpod",
                    "io.kubernetes.cri-o.Created": "2023-10-06T10:56:01.057276698+02:00",
                    "io.podman.annotations.autoremove": "FALSE",
                    "io.podman.annotations.init": "FALSE",
                    "io.podman.annotations.privileged": "FALSE",
                    "io.podman.annotations.publish-all": "FALSE",
                    "org.opencontainers.image.stopSignal": "15"
               },
               "StopSignal": 15,
               "HealthcheckOnFailureAction": "none",
               "CreateCommand": [
                    "C:\\Program Files\\RedHat\\Podman\\podman.exe",
                    "run",
                    "-p",
                    "10000",
                    "-p",
                    "10001",
                    "-p",
                    "10002",
                    "mcr.microsoft.com/azure-storage/azurite",
                    "azurite",
                    "-l",
                    "/data",
                    "--blobHost",
                    "0.0.0.0",
                    "--queueHost",
                    "0.0.0.0",
                    "--tableHost",
                    "0.0.0.0"
               ],
               "Umask": "0022",
               "Timeout": 0,
               "StopTimeout": 10,
               "Passwd": true,
               "sdNotifyMode": "container"
          },
          "HostConfig": {
               "Binds": [
                    "f98cbfb0747071592114aeb0f876157d5ce37bbd66b96177fab54a88801d30d0:/data:rprivate,rw,nodev,exec,nosuid,rbind"
               ],
               "CgroupManager": "cgroupfs",
               "CgroupMode": "host",
               "ContainerIDFile": "",
               "LogConfig": {
                    "Type": "journald",
                    "Config": null,
                    "Path": "",
                    "Tag": "",
                    "Size": "0B"
               },
               "NetworkMode": "bridge",
               "PortBindings": {
                    "10000/tcp": [
                         {
                              "HostIp": "",
                              "HostPort": "35235"
                         }
                    ],
                    "10001/tcp": [
                         {
                              "HostIp": "",
                              "HostPort": "36427"
                         }
                    ],
                    "10002/tcp": [
                         {
                              "HostIp": "",
                              "HostPort": "46647"
                         }
                    ]
               },
               "RestartPolicy": {
                    "Name": "",
                    "MaximumRetryCount": 0
               },
               "AutoRemove": false,
               "VolumeDriver": "",
               "VolumesFrom": null,
               "CapAdd": [],
               "CapDrop": [
                    "CAP_AUDIT_WRITE",
                    "CAP_MKNOD",
                    "CAP_NET_RAW"
               ],
               "Dns": [],
               "DnsOptions": [],
               "DnsSearch": [],
               "ExtraHosts": [],
               "GroupAdd": [],
               "IpcMode": "shareable",
               "Cgroup": "",
               "Cgroups": "default",
               "Links": null,
               "OomScoreAdj": 0,
               "PidMode": "private",
               "Privileged": false,
               "PublishAllPorts": false,
               "ReadonlyRootfs": false,
               "SecurityOpt": [],
               "Tmpfs": {},
               "UTSMode": "private",
               "UsernsMode": "",
               "ShmSize": 65536000,
               "Runtime": "oci",
               "ConsoleSize": [
                    0,
                    0
               ],
               "Isolation": "",
               "CpuShares": 0,
               "Memory": 0,
               "NanoCpus": 0,
               "CgroupParent": "",
               "BlkioWeight": 0,
               "BlkioWeightDevice": null,
               "BlkioDeviceReadBps": null,
               "BlkioDeviceWriteBps": null,
               "BlkioDeviceReadIOps": null,
               "BlkioDeviceWriteIOps": null,
               "CpuPeriod": 0,
               "CpuQuota": 0,
               "CpuRealtimePeriod": 0,
               "CpuRealtimeRuntime": 0,
               "CpusetCpus": "",
               "CpusetMems": "",
               "Devices": [],
               "DiskQuota": 0,
               "KernelMemory": 0,
               "MemoryReservation": 0,
               "MemorySwap": 0,
               "MemorySwappiness": 0,
               "OomKillDisable": false,
               "PidsLimit": 2048,
               "Ulimits": [
                    {
                         "Name": "RLIMIT_NOFILE",
                         "Soft": 1048576,
                         "Hard": 1048576
                    },
                    {
                         "Name": "RLIMIT_NPROC",
                         "Soft": 32768,
                         "Hard": 32768
                    }
               ],
               "CpuCount": 0,
               "CpuPercent": 0,
               "IOMaximumIOps": 0,
               "IOMaximumBandwidth": 0,
               "CgroupConf": null
          }
     }
]

[HofmeisterAn]

Besides the HostIp the configuration is almost identical... Sorry, dunno why the connection to 127.0.0.1 fails. According to this (containers/podman#17780 (comment)) comment, I would expect it to work. Maybe you can ask our friends at Podman what goes wrong. If we add support for #825, we probably can bind "" instead of 0.0.0.0 again.

[gb-nb]

I've asked for some help in the podman repo, the issue was resolved by switching the podman machine to rootless :)

See the discussion here for more info:
containers/podman#20331