Skip to content

eriksjolund/podman-networking-docs

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits

LICENSE

LICENSE

 
 

README.md

README.md

 
 

Repository files navigation

podman-networking-docs

This guide is about how to configure networking when using rootless Podman.

Inbound TCP/UDP connections

Overview

Listening TCP/UDP sockets

method source address preserved native perfomance support for binding to specific network device minimum port number
socket activation (systemd user service) ✔️ ✔️ ✔️ ip_unprivileged_port_start
socket activation (systemd system service with User=) ✔️ ✔️ ✔️ 0
pasta ✔️ ✔️ ip_unprivileged_port_start
slirp4netns + port_handler=slirp4netns ✔️ ip_unprivileged_port_start
slirp4netns + port_handler=rootlesskit ip_unprivileged_port_start
host ✔️ ✔️ ✔️ ip_unprivileged_port_start

Source address preserved

Example:

If the source address is preserved in the incoming TCP connection, then nginx is able to see the IP address of host2 (192.0.2.10) where the curl request is run.

flowchart LR
    curl-->nginx["nginx container"]
    subgraph host1
    nginx
    end
    subgraph host2 ["host2 ip=192.0.2.10"]
    curl
    end

nginx logs the HTTP request as coming from 192.0.2.10

192.0.2.10 - - [15/Jun/2023:07:41:18 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"

If the source address is not preserved, then nginx sees another source address in the TCP connection. For example, if the nginx container is run with slirp4netns + port_handler=rootlesskit

podman run --network=slirp4netns:port_handler=rootlesskit \
	   --publish 8080:80 \
	   --rm \
	   ghcr.io/nginxinc/nginx:latest

nginx logs the HTTP request as coming from 10.0.2.2

10.0.2.2 - - [15/Jun/2023:07:41:18 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"

example: socket activation (systemd user service) - source address preserved

Click me

This example uses two computers

  • host1.example.com (for running the nginx web server)
  • host2.example.com (for running curl)
  1. On host1 create user test 
    sudo useradd test
  2. Open an interactive shell session for the user test 
    sudo machinectl shell test@
  3. Create directories 
    mkdir -p ~/.config/containers/systemd
mkdir -p ~/.config/systemd/user
  4. Create the file /home/test/.config/containers/systemd/nginx.container containing 
    [Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Environment="NGINX=3;"

[Install]
WantedBy=default.target
  5. Create the file /home/test/.config/systemd/user/nginx.socket containing 
    [Unit]
Description=nginx socket

[Socket]
ListenStream=0.0.0.0:8080

[Install]
WantedBy=default.target
  6. Reload the systemd user manager 
    systemctl --user daemon-reload
  7. Pull the container image 
    podman pull ghcr.io/nginxinc/nginx-unprivileged:latest
  8. Start the socket 
    systemctl --user start nginx.socket
  9. Test the nginx web server by accessing it from host2
    1. Log in to host2
    2. Run curl 
      curl host1.example.com:8080
    3. Log out from host2
  10. Check the logs in the container mynginx 
    podman logs mynginx 2> /dev/null | grep "GET /"
    The output should look something like 
    192.0.2.10 - - [15/Jun/2023:07:41:18 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"
    nginx logged the source address of the TCP connection to be 192.0.2.10 which matches the IP address of host2.example.com. Conclusion: the source address was preserved.

A side-note: If the feature request https://trac.nginx.org/nginx/ticket/237 gets implemented, the Environment="NGINX=3;" could be removed. This example makes use of the fact that "nginx includes an undocumented, internal socket-passing mechanism" quote from https://freedesktop.org/wiki/Software/systemd/DaemonSocketActivation/

example: pasta - source address preserved

Click me

This example uses two computers

  • host1.example.com (for running the nginx web server)
  • host2.example.com (for running curl)
  1. On host1 create user test 
    sudo useradd test
  2. Open an interactive shell session for the user test 
    sudo machinectl shell test@
  3. Create directories 
    mkdir -p ~/.config/containers/systemd
  4. Create the file /home/test/.config/containers/systemd/nginx.container containing 
    [Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=pasta
PublishPort=0.0.0.0:8080:8080

[Install]
WantedBy=default.target
  5. Reload the systemd user manager 
    systemctl --user daemon-reload
  6. Pull the container image 
    podman pull ghcr.io/nginxinc/nginx-unprivileged:latest
  7. Start the service 
    systemctl --user start nginx.service
  8. Test the nginx web server by accessing it from host2
    1. Log in to host2
    2. Run curl 
      curl host1.example.com:8080
    3. Log out from host2
  9. Check the logs in the container mynginx 
    podman logs mynginx 2> /dev/null | grep "GET /"
    The output should look something like 
    192.0.2.10 - - [15/Jun/2023:07:55:03 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"
    nginx logged the source address of the TCP connection to be 192.0.2.10 which matches the IP address of host2.example.com. Conclusion: the source address is preserved.

example: slirp4netns + port_handler=slirp4netns - source address preserved

Click me

Follow the same steps as

example: pasta - source address preserved

but replace Network=pasta with Network=slirp4netns:port_handler=slirp4netns.

In other words, replace step 4 with

  1. Create the file /home/test/.config/containers/systemd/nginx.container containing 
    [Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=slirp4netns:port_handler=slirp4netns
PublishPort=0.0.0.0:8080:8080

[Install]
WantedBy=default.target

example: slirp4netns + port_handler=rootlesskit - source address not preserved

Click me

Follow the same steps as

example: pasta - source address preserved

but replace Network=pasta with Network=slirp4netns:port_handler=rootlesskit.

In other words, replace step 4 with

  1. Create the file /home/test/.config/containers/systemd/nginx.container containing 
    [Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=slirp4netns:port_handler=rootlesskit
PublishPort=0.0.0.0:8080:8080

[Install]
WantedBy=default.target

At step 9 you will see that the source address is not preserved. Instead of 192.0.2.10 (IP address for host1.example.com), nginx instead logs the IP address 10.0.2.100.

podman logs mynginx 2> /dev/null | grep "GET /"

The output should look something like

10.0.2.100 - - [15/Jun/2023:07:55:03 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/8.1.1" "-"

example: host - source address preserved

Click me

Follow the same steps as

example: pasta - source address preserved

but remove the line PublishPort=0.0.0.0:8080:8080 and replace Network=pasta with Network=host.

In other words, replace step 4 with

  1. Create the file /home/test/.config/containers/systemd/nginx.container containing 
    [Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=host

[Install]
WantedBy=default.target

Performance

method native perfomance
socket activation (systemd user service) ✔️
socket activation (systemd system service) ✔️
pasta
slirp4netns + port_handler=slirp4netns
slirp4netns + port_handler=rootlesskit
host ✔️

Best performance has

  • socket activation (systemd user service)
  • socket activation (systemd system service)
  • host

where there is no slowdown compared to running directly on the host.

The other methods ordered from fastest to slowest:

  1. pasta
  2. slirp4netns + port_handler=rootlesskit
  3. slirp4netns + port_handler=slirp4netns

Support for binding to specific network device

method support for binding to specific network device
socket activation (systemd user service) ✔️
socket activation (systemd system service) ✔️
pasta ✔️
slirp4netns + port_handler=slirp4netns
slirp4netns + port_handler=rootlesskit
host ✔️

examples

example: socket activation (systemd user service) - bind to specific network device

Click me

Specify the network device to bind to with the systemd directive BindToDevice in the socket unit file.

For example, to bind to the ethernet interface eth0, add the line

BindToDevice=eth0

The socket unit file could look like this

[Unit]
Description=example socket

[Socket]
ListenStream=0.0.0.0:8080
BindToDevice=eth0

[Install]
WantedBy=default.target

example: pasta - bind to specific network device

Click me

To publish the TCP port 8080 and bind the listening socket to the ethernet interface eth0 use the configuration lines

Network=pasta:-t,0.0.0.0%eth0/8080
PublishPorts=0.0.0.0:8080:8080

under the [Container] section in the container file.

For example the file /home/test/.config/containers/systemd/nginx.container containing

[Container]
Image=ghcr.io/nginxinc/nginx-unprivileged:latest
ContainerName=mynginx
Network=pasta:-t,0.0.0.0%eth0/8080
PublishPort=0.0.0.0:8080:8080

[Install]
WantedBy=default.target

If you want to publish an UDP port instead of a TCP port, replace -t with -u above.

Configure ip_unprivileged_port_start

Read the current setting

$ cat /proc/sys/net/ipv4/ip_unprivileged_port_start
1024

To set a new value (for example 443), create the file /etc/sysctl.d/99-mysettings.conf with the contents:

net.ipv4.ip_unprivileged_port_start=443

and reload the configuration

sudo sysctl --system

The setting is system-wide so changing it impacts all users on the system.

Giving this privilege to all users on the computer might not be what you want because often you already know which systemd service should be listening on a privileged port. If the software supports socket activation, an alternative is to set up a systemd system service with User=. For details, see the section Socket activation (systemd system service with User=)

Outbound TCP/UDP connections

Outbound TCP/UDP connections to the internet

An example of an outbound TCP/UDP connection to the internet is when a container downloads a file from a web server on the internet.

method native perfomance
pasta
slirp4netns
host ✔️

Outbound TCP/UDP connections to the host's localhost

An example of an outbound TCP/UDP connection to the host's localhost is when a container downloads a file from a web server on the host that listens on 127.0.0.1:80.

method outbound TCP/UDP connection to the host's localhost allowed by default
pasta
slirp4netns
host ✔️

Connecting to the host's localhost is not enabled by default for pasta and slirp4netns due to security reasons. See network mode host as to why access to the host's localhost is considered insecure.

To allow curl in a container to connect to a web server on the host that listens on 127.0.0.1:80,

for pasta add the option --map-gw

podman run --rm \
           --network=pasta:--map-gw \
           registry.fedoraproject.org/fedora curl localhost 10.0.2.2:80

and for slirp4netns add the option slirp4netns:allow_host_loopback=true

podman run --rm \
           --network=slirp4netns:allow_host_loopback=true \
	   registry.fedoraproject.org/fedora curl localhost 10.0.2.2:80

Connecting to Unix socket on the host

method description
systemd directive OpenFile= The executed command in the container inherits a file descriptor to an already opened file.
bind-mount, (--volume ./dir:/dir:Z ) Standard way

The systemd directive OpenFile= was introduced in systemd 253 (released February 2023).

See also https://github.com/eriksjolund/podman-OpenFile

Valid method combinations

The methods

  • pasta
  • slirp4netns + port_handler=rootlesskit
  • slirp4netns + port_handler=slirp4netns
  • host

are mutually exclusive.

Socket activation can be combined with the other methods.

Description of the different methods

Socket activation (systemd user service)

This method can only be used for container images that has software that supports socket activation.

Socket activation of a systemd user service is set up by creating two files

  • ~/.config/systemd/user/example.socket

and either a Quadlet file

  • ~/.config/containers/systemd/example.container

or a service unit

  • ~/.config/systemd/user/example.service

See Socket activation

Socket activation (systemd system service with User=)

Systemd system service (User=) and socket activation makes it possible for rootless Podman to use privileged ports.

For details of how to use socket-actived nginx, see for instance Example 3, Example 4, Example 5, Example 6 in the repo https://github.com/eriksjolund/podman-nginx-socket-activation

⚠️ How well this solution works is currently unknown. What are the pros and cons? Will it work for other software than nginx? More testing is needed.

There is a Podman feature request for adding Podman support for User= in systemd system services. The feature request was moved into a GitHub discussion.

Pasta

Pasta is enabled by default if no --network option is provided to podman run. Pasta is generally the better choice because it is often faster and has more features than slirp4netns.

On RPM-based systems the executable pasta is in the passt RPM package.

Show the RPM package for the executable /usr/bin/pasta

$ rpm -qf --queryformat "%{NAME}\n" /usr/bin/pasta
passt

The RPM package passt-selinux contains the SELinux configuration for pasta.

To install pasta on Fedora run

$ sudo dnf install -y passt passt-selinux

See the --network option. See also the pasta web page https://passt.top/

Show the default rootlessNetworkCmd

Pasta is the default rootlessNetworkCmd since Podman 5.0.0 (released March 2024).

To show the rootlessNetworkCmd that is configured to be used by default, run

podman info -f '{{.Host.RootlessNetworkCmd}}'

If jq is installed on the computer, then the same result is produced with

podman info -f json | jq -r .host.rootlessNetworkCmd

If podman info does not support the field RootlessNetworkCmd, then it's possible to find out the information by running

podman run -d --rm -p 12345 docker.io/library/alpine sleep 300

and observing if the helper process is pasta or slirp4netns.

For details:

Click me
  1. Set the shell variable user to a username that is not in use. 
    user=mytestuser
  2. Create the new user 
    sudo useradd $user
  3. Open a shell for the new user 
    sudo machinectl shell --uid $user
  4. Verify that no pasta processes are running as the new user. 
    pgrep -u $USER pasta -l
    The command should not list any processes.
  5. Verify that no slirp4netns processes are running as the new user. 
    pgrep -u $USER slirp4netns -l
    The command should not list any processes.
  6. Run container 
    podman run -d --rm -p 12345 docker.io/library/alpine sleep 300
    (12345 is just an arbitrary container port number)
  7. Check if there are any pasta processes running as the new user. 
    pgrep -u $USER pasta -l
    If the command lists any processes, then pasta is detected as being the default.
  8. Check if there are any slirp4netns processes running as the new user. 
    pgrep -u $USER slirp4netns -l
    If the command lists any processes, then slirp4netns is detected as being the default.
  9. Exit the shell 
    exit
  10. Optional step: Delete the newly created user

Slirp4netns

Slirp4netns is similar to Pasta but is slower and has less functionality. Slirp4netns was the default rootlessNetworkCmd before Podman 5.0.0 (released March 2024).

The two port forwarding modes allowed with slirp4netns are described in https://news.ycombinator.com/item?id=33255771

See the --network option.

Host

⚠️ Using --network=host is considered insecure.

Quote from podman run man page: "The host mode gives the container full access to local system services such as D-bus and is therefore considered insecure".

See also the article [CVE-2020–15257] Don’t use --net=host . Don’t use spec.hostNetwork that explains why running containers in the host network namespace is insecure.

Network backends

Check which network backend is in use

$ podman info --format {{.Host.NetworkBackend}}
netavark

CNI

The network backend CNI (Container Network Interface) was removed in Podman 5.0.0. The reasons for replacing CNI with Netavark are described in the article Podman 4.0's new network stack: What you need to know.

Netavark

Netavark is the default network backend.

Example Create a network and run an nginx container

Create the network mynyet

$ podman network create mynet

Start the container docker.io/library/nginx and let it be connected to the network mynet

$ podman run -d -q --network mynet docker.io/library/nginx
19f812cfbb43c022529b84bb9914cda2b16e55ef09c0bc8e937afddfc803f812

Check the IP address

$ podman container inspect -l --format "{{(index .NetworkSettings.Networks \"mynet\").IPAddress}}"
10.89.0.2

Try to fetch a web page from nginx

$ curl --max-time 3 10.89.0.2
curl: (28) Connection timed out after 3000 milliseconds

result: curl was not able to connect to the web server

Join the rootless network namespace used for netavark networking before running the curl command

$ podman unshare --rootless-netns curl --max-time 3 10.89.0.2 | head -4
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

result: curl fetched the web page.

Capture network traffic

The pasta option --pcap enables capturing of network traffic.

Example

Capture a curl request with pasta to the file myfile.pcap. Use tshark to analyse the file myfile.pcap.

  1. Fetch web page from http://podman.io with curl 
    podman run \
   --rm \
   --network=pasta:--pcap,myfile.pcap \
   docker.io/library/fedora curl http://podman.io
    pasta is configured to capture network traffic to the file myfile.pcap

Build tshark container image

  1. Create directory 
    mkdir ctr
  2. Create the file ctr/Containerfile with the contents 
    FROM docker.io/library/fedora
RUN dnf install -y tshark && dnf clean all
  3. Build container image tshark 
    podman build -t tshark ctr/

Show HTTP host and HTTP method in HTTP requests

  1. Use tshark to analyse the file myfile.pcap. 
    podman run \
   --rm
   -v ./myfile.pcap:/mnt/myfile.pcap:Z,ro \
   --user 65534:65534 \
   --userns keep-id:uid=65534,gid=65534 \
   localhost/tshark \
     tshark \
       -r /mnt/myfile.pcap \
       -T fields \
       -e http.host \
       -e http.request.method \
       -Y http | sort -u
    The command prints the following output 
         
podman.io    GET

Show the destination address of IP packets.

  1. Use tshark to analyse the file myfile.pcap. 
    podman run \
   --rm
   -v ./myfile.pcap:/mnt/myfile.pcap:Z,ro \
   --user 65534:65534 \
   --userns keep-id:uid=65534,gid=65534 \
   localhost/tshark \
     tshark \
       -r /mnt/myfile.pcap \
       -T fields \
       -e ip.dst | sort -u
    The command prints the following output 
    
10.0.2.15
169.254.0.1
185.199.110.153
  2. Look up DNS A record of podman.io 
    host -t a podman.io
    The command prints the following output 
    podman.io has address 185.199.110.153
podman.io has address 185.199.111.153
podman.io has address 185.199.108.153
podman.io has address 185.199.109.153
    The IP address 185.199.110.153 is also seen in the tshark output in step 1.

About

rootless Podman networking documentation with examples

Topics

documentation networking examples systemd socket-activation pasta podman slirp4netns systemd-run

Resources

Readme

License

Apache-2.0 license
Activity

Stars

22 stars

Watchers

3 watching

Forks

0 forks
Report repository