Allow per-hostname ports

[dajester2013]

The current template allows for separate hostnames, but they all use the same port for the upstream.

I want to proxy the sonatype/nexus3 container, which starts the docker registry on a separate port from the standard service (8082 for docker vs 8081 for maven).

I need to be able to configure two hostnames for the proxy, with two separate upstream ports.

I would think that you since you can do -e VIRTUAL_HOST=host1,host2, you should also be able to do -e VIRTUAL_PORT=8081,8082

[dajester2013]

I have a workaround - I modified the template and mounted it as a volume when starting the proxy container.

workaround template:

{{ $CurrentContainer := where $ "ID" .Docker.CurrentContainerID | first }}

{{ $external_http_port := coalesce $.Env.HTTP_PORT "80" }}
{{ $external_https_port := coalesce $.Env.HTTPS_PORT "443" }}

{{ define "upstream" }}
	{{ if .Address }}
		{{/* If we got the containers from swarm and this container's port is published to host, use host IP:PORT */}}
		{{ if and .Container.Node.ID .Address.HostPort }}
			# {{ .Container.Node.Name }}/{{ .Container.Name }}
			server {{ .Container.Node.Address.IP }}:{{ .Address.HostPort }};
		{{/* If there is no swarm node or the port is not published on host, use container's IP:PORT */}}
		{{ else if .Network }}
			# {{ .Container.Name }}
			server {{ .Network.IP }}:{{ .Address.Port }};
		{{ end }}
	{{ else if .Network }}
		# {{ .Container.Name }}
		{{ if .Network.IP }}
			server {{ .Network.IP }} down;
		{{ else }}
			server 127.0.0.1 down;
		{{ end }}
	{{ end }}

{{ end }}

{{ define "ssl_policy" }}
	{{ if eq .ssl_policy "Mozilla-Modern" }}
		ssl_protocols TLSv1.3;
		{{/* nginx currently lacks ability to choose ciphers in TLS 1.3 in configuration, see https://trac.nginx.org/nginx/ticket/1529 /*}}
		{{/* a possible workaround can be modify /etc/ssl/openssl.cnf to change it globally (see https://trac.nginx.org/nginx/ticket/1529#comment:12 ) /*}}
		{{/* explicitly set ngnix default value in order to allow single servers to override the global http value */}}
		ssl_ciphers HIGH:!aNULL:!MD5;
		ssl_prefer_server_ciphers off;
	{{ else if eq .ssl_policy "Mozilla-Intermediate" }}
		ssl_protocols TLSv1.2 TLSv1.3;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384';
		ssl_prefer_server_ciphers off;
	{{ else if eq .ssl_policy "Mozilla-Old" }}
		ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA';
		ssl_prefer_server_ciphers on;
	{{ else if eq .ssl_policy "AWS-TLS-1-2-2017-01" }}
		ssl_protocols TLSv1.2 TLSv1.3;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:AES128-GCM-SHA256:AES128-SHA256:AES256-GCM-SHA384:AES256-SHA256';
		ssl_prefer_server_ciphers on;
	{{ else if eq .ssl_policy "AWS-TLS-1-1-2017-01" }}
		ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA';
		ssl_prefer_server_ciphers on;
	{{ else if eq .ssl_policy "AWS-2016-08" }}
		ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA';
		ssl_prefer_server_ciphers on;
	{{ else if eq .ssl_policy "AWS-2015-05" }}
		ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA:DES-CBC3-SHA';
		ssl_prefer_server_ciphers on;
	{{ else if eq .ssl_policy "AWS-2015-03" }}
		ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA:DHE-DSS-AES128-SHA:DES-CBC3-SHA';
		ssl_prefer_server_ciphers on;
	{{ else if eq .ssl_policy "AWS-2015-02" }}
		ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA:DHE-DSS-AES128-SHA';
		ssl_prefer_server_ciphers on;
	{{ end }}
{{ end }}

# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
# scheme used to connect to this server
map $http_x_forwarded_proto $proxy_x_forwarded_proto {
  default $http_x_forwarded_proto;
  ''      $scheme;
}

# If we receive X-Forwarded-Port, pass it through; otherwise, pass along the
# server port the client connected to
map $http_x_forwarded_port $proxy_x_forwarded_port {
  default $http_x_forwarded_port;
  ''      $server_port;
}

# If we receive Upgrade, set Connection to "upgrade"; otherwise, delete any
# Connection header that may have been passed to this server
map $http_upgrade $proxy_connection {
  default upgrade;
  '' close;
}

# Apply fix for very long server names
server_names_hash_bucket_size 128;

# Default dhparam
{{ if (exists "/etc/nginx/dhparam/dhparam.pem") }}
ssl_dhparam /etc/nginx/dhparam/dhparam.pem;
{{ end }}

# Set appropriate X-Forwarded-Ssl header
map $scheme $proxy_x_forwarded_ssl {
  default off;
  https on;
}

gzip_types text/plain text/css application/javascript application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

log_format vhost '$host $remote_addr - $remote_user [$time_local] '
                 '"$request" $status $body_bytes_sent '
                 '"$http_referer" "$http_user_agent"';

access_log off;

{{/* Get the SSL_POLICY defined by this container, falling back to "Mozilla-Intermediate" */}}
{{ $ssl_policy := or ($.Env.SSL_POLICY) "Mozilla-Intermediate" }}
{{ template "ssl_policy" (dict "ssl_policy" $ssl_policy) }}

{{ if $.Env.RESOLVERS }}
resolver {{ $.Env.RESOLVERS }};
{{ end }}

{{ if (exists "/etc/nginx/proxy.conf") }}
include /etc/nginx/proxy.conf;
{{ else }}
# HTTP 1.1 support
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
proxy_set_header X-Forwarded-Ssl $proxy_x_forwarded_ssl;
proxy_set_header X-Forwarded-Port $proxy_x_forwarded_port;

# Mitigate httpoxy attack (see README for details)
proxy_set_header Proxy "";
{{ end }}

{{ $access_log := (or (and (not $.Env.DISABLE_ACCESS_LOGS) "access_log /var/log/nginx/access.log vhost;") "") }}

{{ $enable_ipv6 := eq (or ($.Env.ENABLE_IPV6) "") "true" }}
server {
	server_name _; # This is just an invalid value which will never trigger on a real hostname.
	listen {{ $external_http_port }};
	{{ if $enable_ipv6 }}
	listen [::]:{{ $external_http_port }};
	{{ end }}
	{{ $access_log }}
	return 503;
}

{{ if (and (exists "/etc/nginx/certs/default.crt") (exists "/etc/nginx/certs/default.key")) }}
server {
	server_name _; # This is just an invalid value which will never trigger on a real hostname.
	listen {{ $external_https_port }} ssl http2;
	{{ if $enable_ipv6 }}
	listen [::]:{{ $external_https_port }} ssl http2;
	{{ end }}
	{{ $access_log }}
	return 503;

	ssl_session_cache shared:SSL:50m;
	ssl_session_tickets off;
	ssl_certificate /etc/nginx/certs/default.crt;
	ssl_certificate_key /etc/nginx/certs/default.key;
}
{{ end }}

{{ range $host, $containers := groupByMulti $ "Env.VIRTUAL_HOST" "," }}

{{ $host := trim $host }}
{{ $is_regexp := hasPrefix "~" $host }}
{{ $upstream_name := when $is_regexp (sha1 $host) $host }}

# {{ $host }}
upstream {{ $upstream_name }} {

{{ range $container := $containers }}
	{{ $addrLen := len $container.Addresses }}

	{{ range $knownNetwork := $CurrentContainer.Networks }}
		{{ range $containerNetwork := $container.Networks }}
			{{ if (and (ne $containerNetwork.Name "ingress") (or (eq $knownNetwork.Name $containerNetwork.Name) (eq $knownNetwork.Name "host"))) }}
				## Can be connected with "{{ $containerNetwork.Name }}" network


				{{ $chosts := split $container.Env.VIRTUAL_HOST "," }}
                                {{ $cports := split $container.Env.VIRTUAL_PORT "," }}

				{{/* if multiple virtual_host and virtual_ports specified for the container */}}
				{{ if and ( gt (len $chosts) 1 )  ( eq (len $chosts) (len $cports) ) }}
				{{ range $chix, $chost := $chosts }}
				{{ if eq $chost $host }}
					{{ $port := index $cports $chix }}
					{{ $address := coalesce (where $container.Addresses "Port" $port | first) (dict "Port" $port) }}

					{{ template "upstream" (dict "Container" $container "Address" $address "Network" $containerNetwork ) }}
				{{ end }}
				{{ end }}	
				{{/* If only 1 port exposed, use that */}}
				{{ else if eq $addrLen 1 }}
					{{ $address := index $container.Addresses 0 }}
					{{ template "upstream" (dict "Container" $container "Address" $address "Network" $containerNetwork) }}
				{{/* If more than one port exposed, use the one matching VIRTUAL_PORT env var, falling back to standard web port 80 */}}
				{{ else }}
					{{ $port := coalesce $container.Env.VIRTUAL_PORT "80" }}
					{{ $address := where $container.Addresses "Port" $port | first }}
					{{ template "upstream" (dict "Container" $container "Address" $address "Network" $containerNetwork) }}
				{{ end }}
			{{ else }}
				# Cannot connect to network of this container
				server 127.0.0.1 down;
			{{ end }}
		{{ end }}
	{{ end }}
{{ end }}
}

{{ $default_host := or ($.Env.DEFAULT_HOST) "" }}
{{ $default_server := index (dict $host "" $default_host "default_server") $host }}

{{/* Get the VIRTUAL_PROTO defined by containers w/ the same vhost, falling back to "http" */}}
{{ $proto := trim (or (first (groupByKeys $containers "Env.VIRTUAL_PROTO")) "http") }}

{{/* Get the NETWORK_ACCESS defined by containers w/ the same vhost, falling back to "external" */}}
{{ $network_tag := or (first (groupByKeys $containers "Env.NETWORK_ACCESS")) "external" }}

{{/* Get the HTTPS_METHOD defined by containers w/ the same vhost, falling back to "redirect" */}}
{{ $https_method := or (first (groupByKeys $containers "Env.HTTPS_METHOD")) (or $.Env.HTTPS_METHOD "redirect") }}

{{/* Get the SSL_POLICY defined by containers w/ the same vhost, falling back to empty string (use default) */}}
{{ $ssl_policy := or (first (groupByKeys $containers "Env.SSL_POLICY")) "" }}

{{/* Get the HSTS defined by containers w/ the same vhost, falling back to "max-age=31536000" */}}
{{ $hsts := or (first (groupByKeys $containers "Env.HSTS")) (or $.Env.HSTS "max-age=31536000") }}

{{/* Get the VIRTUAL_ROOT By containers w/ use fastcgi root */}}
{{ $vhost_root := or (first (groupByKeys $containers "Env.VIRTUAL_ROOT")) "/var/www/public" }}


{{/* Get the first cert name defined by containers w/ the same vhost */}}
{{ $certName := (first (groupByKeys $containers "Env.CERT_NAME")) }}

{{/* Get the best matching cert  by name for the vhost. */}}
{{ $vhostCert := (closest (dir "/etc/nginx/certs") (printf "%s.crt" $host))}}

{{/* vhostCert is actually a filename so remove any suffixes since they are added later */}}
{{ $vhostCert := trimSuffix ".crt" $vhostCert }}
{{ $vhostCert := trimSuffix ".key" $vhostCert }}

{{/* Use the cert specified on the container or fallback to the best vhost match */}}
{{ $cert := (coalesce $certName $vhostCert) }}

{{ $is_https := (and (ne $https_method "nohttps") (ne $cert "") (exists (printf "/etc/nginx/certs/%s.crt" $cert)) (exists (printf "/etc/nginx/certs/%s.key" $cert))) }}

{{ if $is_https }}

{{ if eq $https_method "redirect" }}
server {
	server_name {{ $host }};
	listen {{ $external_http_port }} {{ $default_server }};
	{{ if $enable_ipv6 }}
	listen [::]:{{ $external_http_port }} {{ $default_server }};
	{{ end }}
	{{ $access_log }}
	
	# Do not HTTPS redirect Let'sEncrypt ACME challenge
	location /.well-known/acme-challenge/ {
		auth_basic off;
		allow all;
		root /usr/share/nginx/html;
		try_files $uri =404;
		break;
	}
	
	location / {
		return 301 https://$host$request_uri;
	}
}
{{ end }}

server {
	server_name {{ $host }};
	listen {{ $external_https_port }} ssl http2 {{ $default_server }};
	{{ if $enable_ipv6 }}
	listen [::]:{{ $external_https_port }} ssl http2 {{ $default_server }};
	{{ end }}
	{{ $access_log }}

	{{ if eq $network_tag "internal" }}
	# Only allow traffic from internal clients
	include /etc/nginx/network_internal.conf;
	{{ end }}

	{{ template "ssl_policy" (dict "ssl_policy" $ssl_policy) }}

	ssl_session_timeout 5m;
	ssl_session_cache shared:SSL:50m;
	ssl_session_tickets off;

	ssl_certificate /etc/nginx/certs/{{ (printf "%s.crt" $cert) }};
	ssl_certificate_key /etc/nginx/certs/{{ (printf "%s.key" $cert) }};

	{{ if (exists (printf "/etc/nginx/certs/%s.dhparam.pem" $cert)) }}
	ssl_dhparam {{ printf "/etc/nginx/certs/%s.dhparam.pem" $cert }};
	{{ end }}

	{{ if (exists (printf "/etc/nginx/certs/%s.chain.pem" $cert)) }}
	ssl_stapling on;
	ssl_stapling_verify on;
	ssl_trusted_certificate {{ printf "/etc/nginx/certs/%s.chain.pem" $cert }};
	{{ end }}

	{{ if (not (or (eq $https_method "noredirect") (eq $hsts "off"))) }}
	add_header Strict-Transport-Security "{{ trim $hsts }}" always;
	{{ end }}

	{{ if (exists (printf "/etc/nginx/vhost.d/%s" $host)) }}
	include {{ printf "/etc/nginx/vhost.d/%s" $host }};
	{{ else if (exists "/etc/nginx/vhost.d/default") }}
	include /etc/nginx/vhost.d/default;
	{{ end }}

	location / {
		{{ if eq $proto "uwsgi" }}
		include uwsgi_params;
		uwsgi_pass {{ trim $proto }}://{{ trim $upstream_name }};
		{{ else if eq $proto "fastcgi" }}
		root   {{ trim $vhost_root }};
		include fastcgi_params;
		fastcgi_pass {{ trim $upstream_name }};
		{{ else if eq $proto "grpc" }}
		grpc_pass {{ trim $proto }}://{{ trim $upstream_name }};
		{{ else }}
		proxy_pass {{ trim $proto }}://{{ trim $upstream_name }};
		{{ end }}

		{{ if (exists (printf "/etc/nginx/htpasswd/%s" $host)) }}
		auth_basic	"Restricted {{ $host }}";
		auth_basic_user_file	{{ (printf "/etc/nginx/htpasswd/%s" $host) }};
		{{ end }}
		{{ if (exists (printf "/etc/nginx/vhost.d/%s_location" $host)) }}
		include {{ printf "/etc/nginx/vhost.d/%s_location" $host}};
		{{ else if (exists "/etc/nginx/vhost.d/default_location") }}
		include /etc/nginx/vhost.d/default_location;
		{{ end }}
	}
}

{{ end }}

{{ if or (not $is_https) (eq $https_method "noredirect") }}

server {
	server_name {{ $host }};
	listen {{ $external_http_port }} {{ $default_server }};
	{{ if $enable_ipv6 }}
	listen [::]:80 {{ $default_server }};
	{{ end }}
	{{ $access_log }}

	{{ if eq $network_tag "internal" }}
	# Only allow traffic from internal clients
	include /etc/nginx/network_internal.conf;
	{{ end }}

	{{ if (exists (printf "/etc/nginx/vhost.d/%s" $host)) }}
	include {{ printf "/etc/nginx/vhost.d/%s" $host }};
	{{ else if (exists "/etc/nginx/vhost.d/default") }}
	include /etc/nginx/vhost.d/default;
	{{ end }}

	location / {
		{{ if eq $proto "uwsgi" }}
		include uwsgi_params;
		uwsgi_pass {{ trim $proto }}://{{ trim $upstream_name }};
		{{ else if eq $proto "fastcgi" }}
		root   {{ trim $vhost_root }};
		include fastcgi_params;
		fastcgi_pass {{ trim $upstream_name }};
		{{ else if eq $proto "grpc" }}
		grpc_pass {{ trim $proto }}://{{ trim $upstream_name }};
		{{ else }}
		proxy_pass {{ trim $proto }}://{{ trim $upstream_name }};
		{{ end }}
		{{ if (exists (printf "/etc/nginx/htpasswd/%s" $host)) }}
		auth_basic	"Restricted {{ $host }}";
		auth_basic_user_file	{{ (printf "/etc/nginx/htpasswd/%s" $host) }};
		{{ end }}
		{{ if (exists (printf "/etc/nginx/vhost.d/%s_location" $host)) }}
		include {{ printf "/etc/nginx/vhost.d/%s_location" $host}};
		{{ else if (exists "/etc/nginx/vhost.d/default_location") }}
		include /etc/nginx/vhost.d/default_location;
		{{ end }}
	}
}

{{ if (and (not $is_https) (exists "/etc/nginx/certs/default.crt") (exists "/etc/nginx/certs/default.key")) }}
server {
	server_name {{ $host }};
	listen {{ $external_https_port }} ssl http2 {{ $default_server }};
	{{ if $enable_ipv6 }}
	listen [::]:{{ $external_https_port }} ssl http2 {{ $default_server }};
	{{ end }}
	{{ $access_log }}
	return 500;

	ssl_certificate /etc/nginx/certs/default.crt;
	ssl_certificate_key /etc/nginx/certs/default.key;
}
{{ end }}

{{ end }}
{{ end }}

[maxnoe]

This would be great, some services expose multiple ports that need to each have their own virtualhost assigned to.

[maxkratz]

This is also interesting for MinIO S3, because the service exposes a management interface on a separate port: https://docs.min.io/docs/minio-docker-quickstart-guide.html

[tkw1536]

This issue looks like a duplicate of #1463.

For a workaround, see #1463 (comment).

[pini-gh]

I'm changing my mind about this topic.

For services exposing more that one port it should be possible to declare only one host and have nginx-proxy set one upstream block per port. The routing can then be handled depending on the location.

To this end I use a new environment variable VIRTUAL_MULTIPORT with this syntax:

VIRTUAL_MULTIPORT = port, { ",",  port };
port = <virtual_port> [ ":",<virtual_ path> [ ":", <virtual_dest> ]]

<virtual_port>, <virtual_path>, and <virtual_dest> accept the same values than VIRTUAL_{PORT,PATH,DEST}. VIRTUAL_MULTIPORT must not be used along VIRTUAL_{PORT,PATH,DEST} for the same service.

Example:

VIRTUAL_MULTIPORT: "9220:~ ^/(admin|fonts?|images|webmin)/,10901,20901:/ws2p,30901:/gva/playground"

The above example will create four upstream blocks:

# multiport.example.com:10901/
upstream multiport.example.com-10901 {
        ## Can be connected with "docker-gen-bridge" network
        # blah
        server 172.28.0.5:10901;
}
# multiport.example.com:20901/ws2p
upstream multiport.example.com-5c7ebef820fe004e45e3af1d0c47971594d028b2-20901 {
        ## Can be connected with "docker-gen-bridge" network
        # blah
        server 172.28.0.5:20901;
}
# multiport.example.com:30901/gva/playground
upstream multiport.example.com-1f02ce2421b17d828edaabfc3014360891bb0be3-30901 {
        ## Can be connected with "docker-gen-bridge" network
        # blah
        server 172.28.0.5:30901;
}
# multiport.example.com:9220~ ^/(admin|fonts?|images|webmin)/
upstream multiport.example.com-cae8bfc2ea1fe6bb6fda08727ab065e8fed98aa2-9220 {
        ## Can be connected with "docker-gen-bridge" network
        # blah
        server 172.28.0.5:9220;
}

And four location blocs for the related server multiport.example.com:

server {
        server_name multiport.example.com;
        listen 443 ssl http2  ;
        access_log /var/log/nginx/access.log vhost;
        ssl_session_timeout 5m;
        ssl_session_cache shared:SSL:50m;
        ssl_session_tickets off;
        ssl_certificate /etc/nginx/certs/default.crt;
        ssl_certificate_key /etc/nginx/certs/default.key;
        add_header Strict-Transport-Security "max-age=31536000" always;
        location / {
                proxy_pass http://multiport.example.com-10901;
        }
        location /ws2p {
                proxy_pass http://multiport.example.com-5c7ebef820fe004e45e3af1d0c47971594d028b2-20901;
                include /etc/nginx/vhost.d/multiport.example.com_5c7ebef820fe004e45e3af1d0c47971594d028b2_location;
        }
        location /gva/playground {
                proxy_pass http://multiport.example.com-1f02ce2421b17d828edaabfc3014360891bb0be3-30901;
        }
        location ~ ^/(admin|fonts?|images|webmin)/ {
                proxy_pass http://multiport.example.com-cae8bfc2ea1fe6bb6fda08727ab065e8fed98aa2-9220;
                include /etc/nginx/vhost.d/multiport.example.com_cae8bfc2ea1fe6bb6fda08727ab065e8fed98aa2_location;
        }
}

As with the VIRTUAL_PATH it is possible to define per path location configuration files.

This fulfill my needs for the few multiport services I host.

Method	Pros	Cons
Per hostname ports		Must declare a fake host for each added port Pollute the nginx configuration with fake servers Cannot work without setting up custom nginx location files
VIRTUAL_MULTIPORT	Default routing through generated location blocs Simplest configurations work without custom location files	Yet another environment variable

@buchdag Would you consider a MR provided I write some documentation and test cases?

[pini-gh]

I've found a way to use multiport syntax with VIRTUAL_PORT while keeping compatibility with legacy syntax.
This way there is no more need to introduce a new variable.

[rhansen]

@pini-gh It sounds like you are proposing support for different container ports for different location blocks, but all virtual hosts are configured identically. I think this bug is originally about having different virtual hosts proxy to different ports in a single application container. It would be good to come up with a scheme that is general enough to support both. For example, it should be possible to configure nginx-proxy like this:

• https://foo.example/a/ proxies to http://app:80/xyz/
• https://foo.example/b/ proxies to http://app:81/b/
• https://bar.example/ proxies to https://app:4430/

If there was no need to support regular expressions, then I think the above example could be expressed like:

VIRTUAL_HOST: "https://foo.example/a/=>/xyz/,https://foo.example/b/=>81,https://bar.example/=>https:4430”

The above is a comma-separated list, where the left side of the => of each entry describes handled URLs and the right side describes how requests for those URLs are passed to the application container (default protocol is http, default IP is the container's IP if it has only one, default port depends on the protocol, and default destpath is the empty string).

Thoughts?

[pini-gh]

@rhansen:

I think this bug is originally about having different virtual hosts proxy to different ports in a single application container.

You're right. But the main problem people express in this thread is dealing with containers exposing more than one port as you can see in these quotes:

I want to proxy the sonatype/nexus3 container, which starts the docker registry on a separate port from the standard service (8082 for docker vs 8081 for maven).

This would be great, some services expose multiple ports that need to each have their own virtualhost assigned to.

This is also interesting for MinIO S3, because the service exposes a management interface on a separate port

Routing to the different ports via dedicated paths seems good enough a solution to me.

I understand one may prefer using different virtual hosts for different ports of the same container. I plan to address this issue as well, but I'd like not to mix it with the current PR.

[pini-gh]

Thinking about it, it is easy to mix both solutions:

• Multiport syntax for VIRTUAL_PORT
• Per host port syntax for VIRTUAL_HOST

For virtual hosts defined with a dedicated port (say app.example.com:1234), the variable VIRTUAL_PORT is ignored and the dedicated port is used as its virtual port.

Say we have a container with:

VIRTUAL_HOST: "web.nginx-proxy.tld, web1.nginx-proxy.tld:84"
VIRTUAL_PORT: "80,81:/81:/,82:/82:/port,83:~ ^/[8][3]"

It would generate something like this for virtual host web1.nginx-proxy.tld:

# web1.nginx-proxy.tld
upstream web1.nginx-proxy.tld_84 {
        ## Can be connected with "bridge" network
        # test_multiport_syntax_web_1
        server 172.17.0.2:84;
}
server {
        server_name web1.nginx-proxy.tld;
        listen 80 ;
        access_log /var/log/nginx/access.log vhost;
        location / {
                proxy_pass http://web1.nginx-proxy.tld_84;
        }
}

[rhansen]

But the main problem people express in this thread is dealing with containers exposing more than one port as you can see in these quotes:

I agree that there's a need to handle containers that expose multiple ports. I'm just calling attention to how those multiple ports are handled. I interpret the comments as calling for separate virtual hosts for each upstream port, not separate paths (of a common virtual host) for each upstream port:

I need to be able to configure two hostnames for the proxy, with two separate upstream ports.

multiple ports that need to each have their own virtualhost assigned to.

PR #2130 only supports a single virtual host (or multiple identical virtual hosts) for all upstream ports, demuxed by path. That is not sufficient for the original bug report.

Routing to the different ports via dedicated paths seems good enough a solution to me.

I disagree that it is good enough. Many applications have a hard-coded path scheme that does not support namespacing by path. For example, some users must do https://foo.example/ and https://bar.example/ for two upstream ports, not https://example/foo/ and https://example/bar/.

I do think that path-based routing (like #2130) is useful, just not as useful as host-based routing. Regardless of relative importance, the solution for path-based routing should not make it awkward to solve the host-based routing problem, and vice-versa. That's why I would like to find a more general syntax than the syntax in #2130.

[pini-gh]

See my last answer just above yours.

[buchdag]

You're correct and that's already what we do for other features like keepalive (take the config value from the first container in the list), but without a warning.

[pini-gh]

From each container we need to obtain:

* container_ip
* virtual_port
* virtual_dest

We might want to handle several ports for the same container:

* container_ip
* map:
    key: virtual_port
    value: virtual_dest

[pini-gh]

Thinking again about it and testing, I've came up with this data structure:

vhosts (map):
- key: virtual_host
- value (struct):
  - paths (map):
    - key: virtual_path
    - value (struct):
      - dest: virtual_dest (the first in the row)
      - containers (map):
        - key: container_id
        - value (struct):
          - port:  virtual_port
          - dest: virtual_dest
          - ip: container_ip
          - ip_debug: debug text from the 'container_ip' template

Of course some data is omitted, but this structure reflects what we want, as I understand it.

[buchdag]

YAML parsing functions added to docker-gen and docker-gen version updated in nginx-proxy.

[pini-gh]

@buchdag Please see the pini-yaml branch on my repo where i've just pushed a first try at implementing VIRTUAL_HOST_YAML. What do you think?

[buchdag]

@pini-gh sorry I had limited time during April.

I just took a quick glance at your branch and it look really promising, nice use of some sprig features.

I'll take another look at it in an IDE tonight or tomorrow.

The only real question I have at the moment is do we really name the variable VIRTUAL_HOST_YAML ? VIRTUAL_HOST_MULTIPORT maybe 🤔 ?

[buchdag]

Based on your branch's nginx.tmpl, I managed to remove a few excess stuff, most notably the Has_virtual_paths and Multiport_syntax that do not seem to serve a real purpose, along with the dest in the return values of the get_path_info template (we already set dest in each respective VIRTUAL_HOST(*) loop).

The rest is code shuttled around to have both loop be as similar to the other as possible.

This change passes tests.

--- nginx.pinigh.tmpl	2024-05-01 22:41:12.000000000 +0200
+++ nginx.tmpl	2024-05-01 22:42:06.000000000 +0200
@@ -346,12 +346,9 @@
      * The provided dot dict is expected to have the following entries:
      *   - "Containers": List of container's RuntimeContainer struct.
      *   - "Upstream_name"
-     *   - "Has_virtual_paths": boolean
-     *   - "Multiport_syntax": boolean
      *   - "Path"
      *
      * The return values will be added to the dot dict with keys:
-     * - "dest"
      * - "proto"
      * - "network_tag"
      * - "upstream"
@@ -368,22 +365,13 @@
     {{- $keepalive := coalesce (first (keys (groupByLabel $.Containers "com.github.nginx-proxy.nginx-proxy.keepalive"))) "disabled" }}
 
     {{- $upstream := $.Upstream_name }}
-    {{- $dest := "" }}
-    {{- if $.Has_virtual_paths }}
+    {{- if (not (eq $.Path "/")) }}
         {{- $sum := sha1 $.Path }}
         {{- $upstream = printf "%s-%s" $upstream $sum }}
-        {{- $dest = or (first (groupByKeys $.Containers "Env.VIRTUAL_DEST")) "" }}
-    {{- end }}
-    {{- if $.Multiport_syntax }}
-        {{- if (not (eq $.Path "/")) }}
-            {{- $sum := sha1 $.Path }}
-            {{- $upstream = printf "%s-%s" $upstream $sum }}
-        {{- end }}
     {{- end }}
     {{- $_ := set $ "proto" $proto }}
     {{- $_ := set $ "network_tag" $network_tag }}
     {{- $_ := set $ "upstream" $upstream }}
-    {{- $_ := set $ "dest" $dest }}
     {{- $_ := set $ "loadbalance" $loadbalance }}
     {{- $_ := set $ "keepalive" $keepalive }}
 {{- end }}
@@ -512,9 +500,11 @@
     {{- range $hostname, $vhost := (fromYaml $vhosts_yaml) }}
         {{- $vhost_data := when (hasKey $globals.vhosts $hostname) (get $globals.vhosts $hostname) (dict) }}
         {{- $paths := coalesce $vhost_data.paths (dict) }}
+        
         {{- if (empty $vhost) }}
             {{ $vhost = dict "/" (dict) }}
         {{- end }}
+
         {{- range $path, $vpath := $vhost }}
             {{- $dest := coalesce $vpath.dest "" }}
             {{- $port := when (hasKey $vpath "port") (toString $vpath.port) "default" }}
@@ -532,18 +522,18 @@
         {{- $_ := set $vhost_data "paths" $paths }}
         {{- $is_regexp := hasPrefix "~" $hostname }}
         {{- $_ := set $vhost_data "upstream_name" (when (or $is_regexp $globals.sha1_upstream_name) (sha1 $hostname) $hostname) }}
-        {{- $_ := set $vhost_data "has_virtual_paths" false }}
-        {{- $_ := set $vhost_data "multiport_syntax" true }}
         {{- $_ := set $globals.vhosts $hostname $vhost_data }}
     {{- end }}
 {{- end }}
 
 {{- range $hostname, $containers := groupByMulti $globals.containers "Env.VIRTUAL_HOST" "," }}
+    {{- /* Ignore containers with VIRTUAL_HOST set to the empty string. */}}
     {{- $hostname = trim $hostname }}
     {{- if not $hostname }}
-        {{- /* Ignore containers with VIRTUAL_HOST set to the empty string. */}}
         {{- continue }}
     {{- end }}
+
+    {{/* Drop containers with VIRTUAL_HOST_YAML set */}}
     {{- range $_, $containers_to_drop := groupBy $containers "Env.VIRTUAL_HOST_YAML" }}
         {{- range $container := $containers_to_drop }}
             {{- $containers = without $containers $container }}
@@ -552,36 +542,32 @@
     {{- if (eq (len $containers) 0) }}
         {{- continue }}
     {{- end }}
-    {{- $vhost_data := when (hasKey $globals.vhosts $hostname) (get $globals.vhosts $hostname) (dict) }}
 
-    {{- $is_regexp := hasPrefix "~" $hostname }}
-    {{- $upstream_name := when (or $is_regexp $globals.sha1_upstream_name) (sha1 $hostname) $hostname }}
-
-    {{- $has_virtual_paths := false }}
+    {{- $vhost_data := when (hasKey $globals.vhosts $hostname) (get $globals.vhosts $hostname) (dict) }}
     {{- $paths := coalesce $vhost_data.paths (dict) }}
+
     {{- $tmp_paths := groupBy $containers "Env.VIRTUAL_PATH" }}
-    {{- $has_virtual_paths = gt (len $tmp_paths) 0}}
-    {{- if not $has_virtual_paths }}
+    {{- if not (gt (len $tmp_paths) 0) }}
         {{- $tmp_paths = dict "/" $containers }}
     {{- end }}
+
     {{- range $path, $containers := $tmp_paths }}
+        {{- $dest := or (first (groupByKeys $containers "Env.VIRTUAL_DEST")) "" }}
+        {{- $port := "legacy" }}
         {{- $path_data := when (hasKey $paths $path) (get $paths $path) (dict) }}
         {{- $path_ports := when (hasKey $path_data "ports") (get $path_data "ports") (dict) }}
-        {{- $port := "legacy" }}
         {{- $path_port_containers := when (hasKey $path_ports $port) (get $path_ports $port) (list) }}
         {{- $path_port_containers = concat $path_port_containers $containers }}
-
         {{- $_ := set $path_ports $port $path_port_containers }}
         {{- $_ := set $path_data "ports" $path_ports }}
         {{- if (not (hasKey $path_data "dest")) }}
-            {{- $_ := set $path_data "dest" (or (first (groupByKeys $containers "Env.VIRTUAL_DEST")) "") }}
+            {{- $_ := set $path_data "dest" $dest }}
         {{- end }}
         {{- $_ := set $paths $path $path_data }}
     {{- end }}
     {{- $_ := set $vhost_data "paths" $paths }}
-    {{- $_ := set $vhost_data "upstream_name" $upstream_name }}
-    {{- $_ := set $vhost_data "has_virtual_paths" $has_virtual_paths }}
-    {{- $_ := set $vhost_data "multiport_syntax" false }}
+    {{- $is_regexp := hasPrefix "~" $hostname }}
+    {{- $_ := set $vhost_data "upstream_name" (when (or $is_regexp $globals.sha1_upstream_name) (sha1 $hostname) $hostname) }}
     {{- $_ := set $globals.vhosts $hostname $vhost_data }}
 {{- end }}
 
@@ -593,11 +579,13 @@
             {{ $vpath_containers = concat $vpath_containers $vport_containers }}
         {{- end }}
 
-        {{- $args := dict "Containers" $vpath_containers "Path" $path "Upstream_name" $vhost_data.upstream_name "Has_virtual_paths" $vhost_data.has_virtual_paths "Multiport_syntax" $vhost_data.multiport_syntax }}
+        {{- $args := (dict 
+            "Containers" $vpath_containers
+            "Path" $path
+            "Upstream_name" $vhost_data.upstream_name
+        )}}
+
         {{- template "get_path_info" $args }}
-        {{- if $vhost_data.has_virtual_paths }}
-            {{- $_ := set $vpath_data "dest" $args.dest }}
-        {{- end }}
         {{- $_ := set $vpath_data "proto" $args.proto }}
         {{- $_ := set $vpath_data "network_tag" $args.network_tag }}
         {{- $_ := set $vpath_data "upstream" $args.upstream }}

[buchdag]

I pushed your commit, my refactor and a basic test of the feature to the multiport-support branch.

[pini-gh]

Based on your branch's nginx.tmpl, I managed to remove a few excess stuff, most notably the Has_virtual_paths and Multiport_syntax that do not seem to serve a real purpose, along with the dest in the return values of the get_path_info template (we already set dest in each respective VIRTUAL_HOST(*) loop).

My bad. Obviously I missed some bits when cleaning up my work for this branch.

[buchdag]

@pini-gh I added documentation and opened a Draft PR : #2434

I think we can now finish the work on the PR.