Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: duplicate listen options for [::]:443 #5550

Closed
hackuun opened this issue Feb 7, 2018 · 35 comments
Closed

Error: duplicate listen options for [::]:443 #5550

hackuun opened this issue Feb 7, 2018 · 35 comments

Comments

@hackuun
Copy link

hackuun commented Feb 7, 2018

My operating system is (include version):

Ubuntu 16.04

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

sudo apt-get install python-certbot-nginx

I ran this command and it produced this output:

nginx: [emerg] duplicate listen options for [::]:443 in /etc/nginx/sites-enabled/example.online:29

Certbot's behavior differed from what I expected because:

It should be no errors

Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:

server {
  listen 80;
  listen [::]:80;

  server_name example.online;

  root /home/example/deploy;
  index index.html;

  location / {
    try_files $uri $uri/ =404;
  }
}

server {
  listen 80;
  listen [::]:80;
  server_name www.example.online;
  return 301 $scheme://example.online$request_uri;
}

@neoadventist
Copy link

@iamdubx did you figure this out? I'm having the same issue.

@quen2404
Copy link

quen2404 commented Feb 9, 2018

Same issue... It works for my default site, but not for custom sub domain

@varunyellina
Copy link

varunyellina commented Feb 9, 2018

I'm having the same issue.

I do have a config to catch multiple domains though.

server {
  listen 80;
  listen [::]:80;

  root /home/primarydomain/public;
  index index.html index.htm;

  server_name domain1.com *.domain1.com domain2.com *.domain2.com domain3.com *.domain3.com domain4.com *.domain4.com;

  return 302 $scheme://primarydomain.com$request_uri;

  access_log /var/log/nginx/others.access.log;
  error_log /var/log/nginx/others.error.log;

  location / {
    try_files $uri $uri/ /index.html =404;
  }
}

I get nginx: [emerg] duplicate listen options for [::]:443 in /etc/nginx/sites-enabled/others:19 for this config.

OS: Ubuntu 16.04. Any help?

@amattn
Copy link

amattn commented Feb 9, 2018

Same problem.

I ran the command: certbot --redirect --nginx -d readacted.com -d www.redacted.com

my original conf file looks like:

    server {
        server_name redacted.com;
        location / {
        root   /home/redacted/www;
        index  index.html;
      }
    }

    server {
        listen 80;
        listen [::]:80;
        server_name www.redacted.com;
        return 301 $scheme://redacted.com$request_uri;
    }

according to /var/log/letsencrypt/letsencrypt.log I see certbot is trying to do this:

    server {
        server_name redacted.com;
        location / {
        root   /home/redacted/www;
        index  index.html;
      }

        listen [::]:443 ssl ipv6only=on; # managed by Certbot
        listen 443 ssl; # managed by Certbot
        ssl_certificate /etc/letsencrypt/live/redacted.com-0001/fullchain.pem; # managed by Certbot
        ssl_certificate_key /etc/letsencrypt/live/redacted.com-0001/privkey.pem; # managed by Certbot
        include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    }

    server {
        listen 80;
        listen [::]:80;
        server_name www.redacted.com;
        return 301 $scheme://redacted.com$request_uri;

        listen [::]:443 ssl ipv6only=on; # managed by Certbot
        listen 443 ssl; # managed by Certbot
        ssl_certificate /etc/letsencrypt/live/www.redacted.com/fullchain.pem; # managed by Certbot
        ssl_certificate_key /etc/letsencrypt/live/www.redacted.com/privkey.pem; # managed by Certbot
        include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    }

nginx complains that on the line listen [::]:443 ssl ipv6only=on; # managed by Certbot

the actual error message:

nginx: [emerg] duplicate listen options for [::]:443 in /etc/nginx/sites-enabled/redacted.com:23

a quick google brought up this page from 2010:

http://www.serverphorums.com/read.php?5,203912

which suggests that nginx gets confused due to some internal implementation detail.

I'm not an nginx expert, but I've tested that the following seems to work:

    server {
        server_name redacted.com;
        location / {
        root   /home/redacted/www;
        index  index.html;
      }

        listen [::]:443 ssl ipv6only=on; # managed by Certbot
        listen 443 ssl; # managed by Certbot
        ssl_certificate /etc/letsencrypt/live/redacted.com-0001/fullchain.pem; # managed by Certbot
        ssl_certificate_key /etc/letsencrypt/live/redacted.com-0001/privkey.pem; # managed by Certbot
        include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    }

    server {
        listen 80;
        listen [::]:80;
        server_name www.redacted.com;
        return 301 http://redacted.com$request_uri;

        listen [::]:443; # manually changed
        ssl on;  #manually changed
        listen 443 ssl; # managed by Certbot
        ssl_certificate /etc/letsencrypt/live/www.redacted.com/fullchain.pem; # managed by Certbot
        ssl_certificate_key /etc/letsencrypt/live/www.redacted.com/privkey.pem; # managed by Certbot
        include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    }

I'd love a better solution that this workaround ideally...

@schoen
Copy link
Contributor

schoen commented Feb 13, 2018

@ohemorange, do you know if we have an existing issue tracking this? It feels familiar to me but I don't remember whether or not it's something that we've looked into before.

@ohemorange
Copy link
Contributor

I have never seen this before. Looks like they fixed the original bug, except when you're using IPv6. And since we just launched IPv6 support, that's why people are hitting this. The solution above will work; I'll see if there's a reason this hasn't been fixed in Nginx for IPv6 yet.

@ohemorange
Copy link
Contributor

Actually, you don't even have to do the ssl on change -- removing ipv6only=on from either or both fixes the problem.

@joohoi, we probably want to fix this by either removing ipv6only=on completely or only putting it in once per unique address line that we add. Do you have a sense of what would be best here?

@rushilsrivastava
Copy link

Having the same issue here. Everything was fine for my first domain. The second domain started having these issues.

@joohoi
Copy link
Member

joohoi commented Feb 14, 2018

It looks like Certbot is unable to detect ipv6only directive completely for some reason. Removing that would fix the issue for most of the users. This can cause some issues with really old Nginx versions, as the behavior of ipv6only and the defaults have changed over the time.

@attie-argentum
Copy link

attie-argentum commented Feb 14, 2018

Apologies for the nasty patch, but this fixed it for me, hopefully there will be a proper fix soon!

--- /usr/lib/python3/dist-packages/certbot_nginx/configurator.py.orig	2018-02-14 18:38:30.380863045 +0000
+++ /usr/lib/python3/dist-packages/certbot_nginx/configurator.py	2018-02-14 18:38:01.501018553 +0000
@@ -507,10 +507,10 @@ class NginxConfigurator(common.Installer
                           '[::]:{0}'.format(self.config.tls_sni_01_port),
                           ' ',
                           'ssl']
-            if not ipv6info[1]:
-                # ipv6only=on is absent in global config
-                ipv6_block.append(' ')
-                ipv6_block.append('ipv6only=on')
+            #if not ipv6info[1]:
+            #    # ipv6only=on is absent in global config
+            #    ipv6_block.append(' ')
+            #    ipv6_block.append('ipv6only=on')
 
         if vhost.ipv4_enabled():
             ipv4_block = ['\n    ',
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial
$ nginx -V
nginx version: nginx/1.10.3 (Ubuntu)
built with OpenSSL 1.0.2g  1 Mar 2016
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-stream --with-stream_ssl_module --with-mail --with-mail_ssl_module --with-threads
$ apt show python-certbot-nginx
Package: python-certbot-nginx
Version: 0.21.1-1+ubuntu16.04.1+certbot+1
Priority: optional
Section: oldlibs
Maintainer: Debian Let's Encrypt <letsencrypt-devel@lists.alioth.debian.org>
Installed-Size: 9,216 B
Depends: python3-certbot-nginx
Download-Size: 2,470 B
APT-Manual-Installed: yes
APT-Sources: http://ppa.launchpad.net/certbot/certbot/ubuntu xenial/main amd64 Packages
Description: transitional dummy package
 This is a transitional dummy package for the migration of certbot
 from python2 to python3.  It can be safely removed.

@lt100
Copy link

lt100 commented Feb 23, 2018

Same issue. Will this be addressed?

@joohoi
Copy link
Member

joohoi commented Feb 23, 2018

Sorry for the wall of text, but here we go.

To shed some light on the issue:
ipv6only option is used to be able to handle multiple listen statements per socket. Unfortunately it can only be used once in the server configuration for the socket. So Nginx will fail to start in case of:

server {
    ...
    server_name first.example.org;
    listen [::]:80 ipv6only=on;
    listen 80;
} 
server {
    ...
    server_name second.example.org;
    listen [::]:80 ipv6only=on;
    listen 80;
}

With the recent versions of Nginx this problem does not exist if the ipv6only setting is omitted completely, as the default value of the variable is ipv6only=on. So the following is valid and working configuration in Nginx versions >= 1.3.4:

server {
    ...
    server_name first.example.org;
    listen [::]:80;
    listen 80;
} 
server {
    ...
    server_name second.example.org;
    listen [::]:80;
    listen 80;
}

However the default value of ipv6only variable in Nginx versions prior to 1.3.4 was ipv6only=off, so older versions will fail with the following configuration:

server {
    ...
    server_name first.example.org;
    listen [::]:80;
    listen 80;
} 

Currently the situation with distribution packaging is that the only distribution that ships with older version of Nginx would be Debian Wheezy (Debian 7), which ships version 1.2.1 of Nginx from the default repositories.

If we would remove the ipv6only detection and setting completely from Certbot, this would break for all users on Debian Wheezy. Fortunately EOL date for Wheezy is set for May 2018, so we're close to being able to remove this additional complexity from Certbot code altogether.

The current functionality of Certbot is parsing the complete Nginx configuration, detecting ipv6only=on setting already present in one of the server{} blocks and omitting addition of it if so. If however the value was not found, Certbot would then add it. This issue boils down to Certbot not being able to detect this already existing variable from some block of the users current configuration, and hence tries to add it to the server{} block that's being configured.

To make decision of the route to fix this issue, we'd need a complete example configuration where Certbot fails in the way described above to be able to improve the detection of an already existing ipv6only=onvariable if we decide to fix it that way instead of removing this functionality altogether.

@caspervoogt
Copy link

Thanks for the patch; that worked for me. FWIW, I'm on Ubuntu 17.

@chilion
Copy link

chilion commented Feb 27, 2018

I had to remove all the

listen [::]:80;
listen 80;

To make it work

@davidkartuzinski
Copy link

davidkartuzinski commented Feb 28, 2018

https://github.com/chilion - thanks! removing:

listen [::]:80;
listen 80;

worked for me as well.

I have two domains on one Ubuntu server. The first one worked no problem. Then I got the error as above. Your solution worked for me. I just installed nginx on a fresh server with fresh everything.

Thank you.

@jevets
Copy link

jevets commented Feb 28, 2018

removing listen [::]:80 but leaving listen 80; worked for me for installation on non-default domain

@oscarzhou
Copy link

I comment listen [::]:443 in the subdomain setting, then it works. Is it okay

martinklepsch added a commit to cljdoc/cljdoc that referenced this issue Oct 21, 2018
also multiple occurences cause problems with `nginx -t`, see:
certbot/certbot#5550 (comment)
@JFossey
Copy link

JFossey commented Apr 11, 2019

I just bumped into this issue. Any body trying to make sense of the the different listen directives and ipv6only.

I highly recommend this blog post, until I found this article I was not sure what do with all the different advice I was finding on the web.

https://stefanchrist.eu/blog/2015_01_21/Using%20ipv6only%20in%20Nginx.xhtml

This quote from the blog post was the light-bulb moment for me.

The parameter is different from e.g. the ssl flag. The flag ssl can be used in multiple server contexts and be switch on and off as you wish. The flag ipv6only can only be set once per port (and address). Only a single listen directive my contain the parameter and it will be valid for all server contexts using this port. If you use it twice, the nginx daemon won't start and will write the following error messages to his error log

@BonBonSlick
Copy link

BonBonSlick commented Jul 28, 2019

Still exists, after purge python, reinstall this error thrown. Error somewhere in certbot

Commenting this line solves error, but creates other issues

server {
        listen  443 ssl http2;
#        listen [::]:443 ssl http2 ipv6only=on;


@sakaruk
Copy link

sakaruk commented Sep 2, 2019

In case of multiple domains

Instead of
listen [::]:443 ssl http2 ipv6only=on;

Use
listen example.com:443 ssl http2 ipv6only=on;

@souvik-networth-corp
Copy link

Omit the listen directive in all your server blocks.

@AlexTies
Copy link

AlexTies commented Nov 18, 2019

This error shows up when there are two server blocks listening on the same domain with the same port.
Check all your configuration files in the sites-available folder for the duplicate listener. In my case certbot created duplicate a listener for 443 in the default file.

@bmw
Copy link
Member

bmw commented Nov 20, 2019

If you can provide configuration files to reproduce this with an up-to-date version of Certbot, I'd be interested to see them.

@LF-DevJourney
Copy link

nginx: [emerg] a duplicate default server for

@Znuff
Copy link

Znuff commented Jun 10, 2020

For the adventurers that may hit this ticket in the future via a lonesome search, and they can't figure why this happens when they don't have ipv6only=on anywhere else.

You will get the same error/issue if you have reuseport in your config.

@ohemorange
Copy link
Contributor

I will admit, I am confused. According to the nginx documentation, there's a number of parameters for listen, but only ipv6only specifies "It can only be set once on start." Is this line just missing from the remaining parameters? Is it system-dependent? I'm starting to think that fixing this behavior in upstream might be the best course of action; it seems silly to only allow these options to be set once, anyway.

@Znuff
Copy link

Znuff commented Jun 10, 2020

I'm unfortunately not an expert in linux sockets, so I can't form a proper opinion why those options can only be set once, but I'm sure there's a reason.

Maybe this post helps: https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/

What I do know is that, just like ipv6only, reuseport can also only be set once per a specific port (so only one listener can have it). Why this incidentally conflicts (for the lack of a better word) with ipv6only, I have no clue.

Still, I feel like adding ipv6only=on when running certbot is a bit futile.

It's no longer needed since nginx 1.3.4, which has been released in 2012, and it's technically EOL.

At very least, maybe there should be a version check and only add if it nginx < 1.3.4 before adding it.

@ohemorange
Copy link
Contributor

We don't set it in Certbot. When we create a server block, we copy some directives from the existing default server block, or other template server block, including the listen directive along with its options. This lets Certbot work even if Nginx is behind a proxy or other type of port forwarding. We explicitly delete ipv6only=on from the duplicated server block because the documentation indicates it can only be used once.

Ideally, we would do the same for all options that we know can't be duplicated in this way, but leave other options that the user might specifically want all their server blocks to have. To do that, we have to know which options are repeatable, which the documentation does not seem to indicate, and which we only seem to discover through people coming to us on issues like this one.

@OverStruck
Copy link

Thanks @joohoi
Your explanation & solution worked for me on Ubuntu 20 with nginx version: 1.18.0

@maitrungduc1410
Copy link

I have 2 VPS: 1 is Ubuntu runs Nginx 1.10 works fine, the other is Centos runs Nginx 1.16 and has this error. Weird

@heylix
Copy link

heylix commented Mar 16, 2021

I also ran into that issue and think that autogenerated stuff like this should be prevented by adding an nginx -t check before writing the configuration. Automated backups somewhere would also be a nice touch…

@hak33m16
Copy link

hak33m16 commented Aug 7, 2021

Looks like this is still an issue. This comment was the solution for me: #5550 (comment)

What's interesting is that I didn't hit this until I added the 4th site to my /etc/nginx/sites-available folder. I have two domains running out of the same box, and each of those have their own dev.<site>.com sites-available config as well

I have zero issues with the first domain that I added the dev. subdomain for, but the second is what kicked this off

Versions:

$> lsb_release -a
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.2 LTS
Release:        20.04
Codename:       focal

$> nginx -V
nginx version: nginx/1.18.0 (Ubuntu)
built with OpenSSL 1.1.1f  31 Mar 2020

$> certbot --version
certbot 0.40.0

Edit: Uhhhh, wow, did not realize my version of certbot was so far behind since this was a new machine and I blindly followed the DigitalOcean certbot setup guide here: https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04

Looks like that does not guarantee I get the latest version

@hardikhari96
Copy link

hardikhari96 commented Sep 19, 2021

I solved issue ,

It's occur when you trying to install certificate with certbot.

So what happens, certbot automatically add its own listen 443; so if your script had already that ,it will throw error that duplicate listen 443

@oDinZu
Copy link

oDinZu commented May 19, 2022

I am having the same issue.

It started today after using certbot for a new domain.

Now, all my configs don't work and the server won't start now.

Has anyone solved this issue? I had no problems until today and have used certbot before without this happening.

I have never used ipv6only with my configs; this error happened without having ipv6only added.

@schevenin
Copy link

Sorry for the wall of text, but here we go.

To shed some light on the issue: ipv6only option is used to be able to handle multiple listen statements per socket. Unfortunately it can only be used once in the server configuration for the socket. So Nginx will fail to start in case of:

server {
    ...
    server_name first.example.org;
    listen [::]:80 ipv6only=on;
    listen 80;
} 
server {
    ...
    server_name second.example.org;
    listen [::]:80 ipv6only=on;
    listen 80;
}

With the recent versions of Nginx this problem does not exist if the ipv6only setting is omitted completely, as the default value of the variable is ipv6only=on. So the following is valid and working configuration in Nginx versions >= 1.3.4:

server {
    ...
    server_name first.example.org;
    listen [::]:80;
    listen 80;
} 
server {
    ...
    server_name second.example.org;
    listen [::]:80;
    listen 80;
}

However the default value of ipv6only variable in Nginx versions prior to 1.3.4 was ipv6only=off, so older versions will fail with the following configuration:

server {
    ...
    server_name first.example.org;
    listen [::]:80;
    listen 80;
} 

Currently the situation with distribution packaging is that the only distribution that ships with older version of Nginx would be Debian Wheezy (Debian 7), which ships version 1.2.1 of Nginx from the default repositories.

If we would remove the ipv6only detection and setting completely from Certbot, this would break for all users on Debian Wheezy. Fortunately EOL date for Wheezy is set for May 2018, so we're close to being able to remove this additional complexity from Certbot code altogether.

The current functionality of Certbot is parsing the complete Nginx configuration, detecting ipv6only=on setting already present in one of the server{} blocks and omitting addition of it if so. If however the value was not found, Certbot would then add it. This issue boils down to Certbot not being able to detect this already existing variable from some block of the users current configuration, and hence tries to add it to the server{} block that's being configured.

To make decision of the route to fix this issue, we'd need a complete example configuration where Certbot fails in the way described above to be able to improve the detection of an already existing ipv6only=onvariable if we decide to fix it that way instead of removing this functionality altogether.

Removing ipv6only=on; fixed the issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests