unable to run certbot --nginx - UnicodeDecodeError: 'ascii' #5236

john-gold · 2017-11-12T15:22:06Z

My operating system is (include version):

ArchLinux (Kernel 4.13.11-1)

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

pacman -S cerbot-nginx

I ran this command and it produced this output:

cerbot --nginx

Certbot's behavior differed from what I expected because:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
An unexpected error occurred:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 330: ordinal not in range(128)
Please see the logfiles in /var/log/letsencrypt for more details.

Here is a Certbot log showing the issue (if available):

2017-11-12 15:15:18,497:DEBUG:certbot.main:certbot version: 0.19.0
2017-11-12 15:15:18,499:DEBUG:certbot.main:Arguments: ['--nginx']
2017-11-12 15:15:18,500:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2017-11-12 15:15:18,529:DEBUG:certbot.log:Root logging level set at 20
2017-11-12 15:15:18,531:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2017-11-12 15:15:18,532:DEBUG:certbot.plugins.selection:Requested authenticator nginx and installer nginx
2017-11-12 15:15:18,557:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.19.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3.6/site-packages/certbot/main.py", line 861, in main
    return config.func(config, plugins)
  File "/usr/lib/python3.6/site-packages/certbot/main.py", line 685, in run
    installer, authenticator = plug_sel.choose_configurator_plugins(config, plugins, "run")
  File "/usr/lib/python3.6/site-packages/certbot/plugins/selection.py", line 189, in choose_configurator_plugins
    authenticator = installer = pick_configurator(config, req_inst, plugins)
  File "/usr/lib/python3.6/site-packages/certbot/plugins/selection.py", line 25, in pick_configurator
    (interfaces.IAuthenticator, interfaces.IInstaller))
  File "/usr/lib/python3.6/site-packages/certbot/plugins/selection.py", line 77, in pick_plugin
    verified.prepare()
  File "/usr/lib/python3.6/site-packages/certbot/plugins/disco.py", line 248, in prepare
    return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
  File "/usr/lib/python3.6/site-packages/certbot/plugins/disco.py", line 248, in <listcomp>
    return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
  File "/usr/lib/python3.6/site-packages/certbot/plugins/disco.py", line 130, in prepare
    self._initialized.prepare()
  File "/usr/lib/python3.6/site-packages/certbot_nginx/configurator.py", line 156, in prepare
    self.parser = parser.NginxParser(self.conf('server-root'))
  File "/usr/lib/python3.6/site-packages/certbot_nginx/parser.py", line 35, in __init__
    self.load()
  File "/usr/lib/python3.6/site-packages/certbot_nginx/parser.py", line 42, in load
    self._parse_recursively(self.config_root)
  File "/usr/lib/python3.6/site-packages/certbot_nginx/parser.py", line 53, in _parse_recursively
    trees = self._parse_files(filepath)
  File "/usr/lib/python3.6/site-packages/certbot_nginx/parser.py", line 203, in _parse_files
    parsed = nginxparser.load(_file)
  File "/usr/lib/python3.6/site-packages/certbot_nginx/nginxparser.py", line 122, in load
    return loads(_file.read())
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 330: ordinal not in range(128)
2017-11-12 15:15:18,560:ERROR:certbot.log:An unexpected error occurred:

Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:

user http;
worker_processes auto;
worker_cpu_affinity auto;
pcre_jit on;

events {
    worker_connections 2048;
}


http {
    include mime.types;
    default_type application/octet-stream;
    sendfile on;
    tcp_nopush on;
    aio threads;
    server_tokens off; # Security: Disables nginx version in error messages and in the “Server” response header field.
    charset utf-8; # Force usage of UTF-8
    index index.php index.html index.htm;
    server {
      listen 80;
      listen [::]:80;
      server_name domain.tld;
      root /usr/share/nginx/html;
      location / {
        index index.htm index.html;
      }

      # ACME challenge
      location ^~ /.well-known/acme-challenge/ {
        default_type "text/plain";
        root /var/lib/letsencrypt;
      }
    }
}

The text was updated successfully, but these errors were encountered:

joohoi · 2017-11-13T11:16:39Z

We will look into this, as it is certainly something that we should handle, but to work around this, replace the double quotes in comment of server_tokens directive: “Server”

shiroorg · 2018-05-05T12:29:29Z

@joohoi

2018-05-05 12:28:44,181:DEBUG:certbot.main:certbot version: 0.22.2
2018-05-05 12:28:44,181:DEBUG:certbot.main:Arguments: ['--nginx']
2018-05-05 12:28:44,181:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2018-05-05 12:28:44,193:DEBUG:certbot.log:Root logging level set at 20
2018-05-05 12:28:44,193:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2018-05-05 12:28:44,194:DEBUG:certbot.plugins.selection:Requested authenticator nginx and installer nginx
2018-05-05 12:28:44,343:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File "/usr/bin/certbot", line 11, in
load_entry_point('certbot==0.22.2', 'console_scripts', 'certbot')()
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1266, in main
return config.func(config, plugins)
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1018, in run
installer, authenticator = plug_sel.choose_configurator_plugins(config, plugins, "run")
File "/usr/lib/python3/dist-packages/certbot/plugins/selection.py", line 189, in choose_configurator_plugins
authenticator = installer = pick_configurator(config, req_inst, plugins)
File "/usr/lib/python3/dist-packages/certbot/plugins/selection.py", line 25, in pick_configurator
(interfaces.IAuthenticator, interfaces.IInstaller))
File "/usr/lib/python3/dist-packages/certbot/plugins/selection.py", line 77, in pick_plugin
verified.prepare()
File "/usr/lib/python3/dist-packages/certbot/plugins/disco.py", line 245, in prepare
return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
File "/usr/lib/python3/dist-packages/certbot/plugins/disco.py", line 245, in
return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
File "/usr/lib/python3/dist-packages/certbot/plugins/disco.py", line 126, in prepare
self._initialized.prepare()
File "/usr/lib/python3/dist-packages/certbot_nginx/configurator.py", line 137, in prepare
self.parser = parser.NginxParser(self.conf('server-root'))
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 38, in init
self.load()
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 45, in load
self._parse_recursively(self.config_root)
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 66, in _parse_recursively
self._parse_recursively(subentry[1])
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 56, in _parse_recursively
trees = self._parse_files(filepath)
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 206, in _parse_files
parsed = nginxparser.load(_file)
File "/usr/lib/python3/dist-packages/certbot_nginx/nginxparser.py", line 123, in load
return loads(_file.read())
File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position 31: ordinal not in range(128)
2018-05-05 12:28:44,346:ERROR:certbot.log:An unexpected error occurred:

mardiros · 2018-07-05T20:45:40Z

For instance, the mime type files contains this lines:

application/vnd.geocube+xml                     g3 g³;

g³ will make raise the UnicodeDecodeError

> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(62)_parse_recursively()
-> elif entry[0] == ['http'] or entry[0] == ['server']:
(Pdb) l
 57             for tree in trees:
 58                 for entry in tree:
 59                     if _is_include_directive(entry):
 60                         # Parse the top-level included file
 61                         self._parse_recursively(entry[1])
 62  ->                 elif entry[0] == ['http'] or entry[0] == ['server']:
 63                         # Look for includes in the top-level 'http'/'server' context
 64                         for subentry in entry[1]:
 65                             if _is_include_directive(subentry):
 66                                 self._parse_recursively(subentry[1])
 67                             elif entry[0] == ['http'] and subentry[0] == ['server']:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(64)_parse_recursively()
-> for subentry in entry[1]:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(65)_parse_recursively()
-> if _is_include_directive(subentry):
(Pdb) subentry
['include', 'mime.types']
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(66)_parse_recursively()
-> self._parse_recursively(subentry[1])
(Pdb) s
--Call--
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(47)_parse_recursively()
-> def _parse_recursively(self, filepath):
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(55)_parse_recursively()
-> filepath = self.abs_path(filepath)
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(56)_parse_recursively()
-> trees = self._parse_files(filepath)
(Pdb) s
--Call--
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(189)_parse_files()
-> def _parse_files(self, filepath, override=False):
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(198)_parse_files()
-> files = glob.glob(filepath) # nginx on unix calls glob(3) for this
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(201)_parse_files()
-> trees = []
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(202)_parse_files()
-> for item in files:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(203)_parse_files()
-> if item in self.parsed and not override:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(205)_parse_files()
-> try:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(206)_parse_files()
-> with open(item) as _file:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(207)_parse_files()
-> parsed = nginxparser.load(_file)
(Pdb) s
--Call--
> /usr/lib/python3.6/site-packages/certbot_nginx/nginxparser.py(115)load()
-> def load(_file):
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/nginxparser.py(123)load()
-> return loads(_file.read())
(Pdb) loads
<function loads at 0x7faacac85048>
(Pdb) s
--Call--
> /usr/lib/python3.6/encodings/ascii.py(25)decode()
-> def decode(self, input, final=False):
(Pdb) --KeyboardInterrupt--
(Pdb) n
> /usr/lib/python3.6/encodings/ascii.py(26)decode()
-> return codecs.ascii_decode(input, self.errors)[0]
(Pdb) n
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 10453: ordinal not in range(128)
> /usr/lib/python3.6/encodings/ascii.py(26)decode()
-> return codecs.ascii_decode(input, self.errors)[0]

joohoi · 2018-07-06T10:26:04Z

Thanks for the additional information!

TommyZG · 2018-09-02T12:59:53Z

After reading a comment on another thread by user @egberts, I ran the following command:

grep -r -P '[^\x00-\x7f]' /etc/apache2 /etc/letsencrypt /etc/nginx

That command found the offending character "´" in one .conf file in the comment. After removing it (you can edit comments as you wish) and reloading nginx, everything worked again.

Fretice · 2018-09-07T09:36:37Z

@TommyZG can u tell me change which file ?

Fretice · 2018-09-13T01:13:38Z

i found that when i use certbot-auto in windows remote, it throw me this error ,but when i use it in linux remote ,it works great

Finesse · 2018-09-15T05:57:27Z

I had the same problem. I've found the following workaround (on Ubuntu):

Run in a console:

sudo locale-gen "en_US.UTF-8"
sudo dpkg-reconfigure locales

Add line LC_ALL="en_US.UTF-8" to the file /etc/default/locale.
Reconnect SSH.

After doing it, running cerbot --nginx successes.

da3020 · 2018-09-18T12:30:18Z

Hi! The reason is that if you have some non ascii letters in nginx config (even in comments!!!) it will not work.

ohemorange · 2018-09-18T19:25:08Z

Closing in favor of duplicate #5337, which has a PR at #5341.

TommyZG · 2018-10-29T20:30:53Z

@TommyZG can u tell me change which file ?

It was in one of my .conf files. You have others. It was in the comment line.

A34 · 2019-04-25T14:50:13Z

I got a similar issue, it was in /etc/nginx/conf.d/default.conf line 13 to 17:

# 1) limit_req  ^`^s to limit the rate of requests from one IP
# 2) limit_conn  ^`^s to limit the number of connections from one IP
# 3) client_body_timeout  ^`^s to close the connections with slow body
# 4) client_header_timeout  ^`^s to close the connections with slow headers
# 5) send_timeout  ^`^s If the client does not receive anything within this time, the connection is closed.

I removed those comments and it worked.

Thanks @egberts & @TommyZG for the grep tip.

Kassan424kh · 2019-08-20T08:08:01Z

Hy ✌🏻,
you should only using other terminal.
I have the same problem while used mac terminal, than i used windows bash to get it works.

Bill0412 · 2020-01-22T08:25:22Z

I had the same problem. I've found the following workaround (on Ubuntu):
Run in a console:
sudo locale-gen "en_US.UTF-8"
sudo dpkg-reconfigure locales
Add line LC_ALL="en_US.UTF-8" to the file /etc/default/locale.

Reconnect SSH.
After doing it, running cerbot --nginx successes.

Thanks, it worked for me.

mauryakrishna · 2021-01-04T19:20:23Z

After reading a comment on another thread by user @egberts, I ran the following command:

grep -r -P '[^\x00-\x7f]' /etc/apache2 /etc/letsencrypt /etc/nginx

That command found the offending character "´" in one .conf file in the comment. After removing it (you can edit comments as you wish) and reloading nginx, everything worked again.

Running the grep command as mentioned above did find a unwanted character in my nginx .conf file. Replacing that with the valid character does resolve the issue.

Thanks a lot.

dsignr · 2021-04-29T05:12:11Z

After reading a comment on another thread by user @egberts, I ran the following command:

grep -r -P '[^\x00-\x7f]' /etc/apache2 /etc/letsencrypt /etc/nginx

That command found the offending character "´" in one .conf file in the comment. After removing it (you can edit comments as you wish) and reloading nginx, everything worked again.

Three years later and this is still a life saver. THANK YOU!

fandigunawan · 2022-02-03T04:41:57Z

After reading a comment on another thread by user @egberts, I ran the following command:

grep -r -P '[^\x00-\x7f]' /etc/apache2 /etc/letsencrypt /etc/nginx

That command found the offending character "´" in one .conf file in the comment. After removing it (you can edit comments as you wish) and reloading nginx, everything worked again.

This is a life saving, thanks

marcelomborges · 2023-09-13T14:52:20Z

After reading a comment on another thread by user @egberts, I ran the following command:

grep -r -P '[^\x00-\x7f]' /etc/apache2 /etc/letsencrypt /etc/nginx

That command found the offending character "´" in one .conf file in the comment. After removing it (you can edit comments as you wish) and reloading nginx, everything worked again.

After 5 years, still helping

joohoi added area: nginx bug labels Nov 13, 2017

SwartzCr mentioned this issue Dec 19, 2017

Nginx plugin crashes when non-ascii configuration file read #5337

Closed

This was referenced Jan 29, 2018

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 2: ordinal not in range(128) #5509

Closed

webroot plugin can't handle unicode webroot paths #5512

Closed

bmw mentioned this issue Mar 2, 2018

UnicodeDecodeError: 'utf-8' codec can't decode #5646

Closed

joohoi added the area: encoding/type error label Jul 6, 2018

ohemorange closed this as completed Sep 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unable to run certbot --nginx - UnicodeDecodeError: 'ascii' #5236

unable to run certbot --nginx - UnicodeDecodeError: 'ascii' #5236

john-gold commented Nov 12, 2017

joohoi commented Nov 13, 2017

shiroorg commented May 5, 2018

mardiros commented Jul 5, 2018

joohoi commented Jul 6, 2018

TommyZG commented Sep 2, 2018

Fretice commented Sep 7, 2018

Fretice commented Sep 13, 2018

Finesse commented Sep 15, 2018 •

edited

da3020 commented Sep 18, 2018

ohemorange commented Sep 18, 2018

TommyZG commented Oct 29, 2018

A34 commented Apr 25, 2019 •

edited

Kassan424kh commented Aug 20, 2019

Bill0412 commented Jan 22, 2020

mauryakrishna commented Jan 4, 2021 •

edited

dsignr commented Apr 29, 2021

fandigunawan commented Feb 3, 2022

marcelomborges commented Sep 13, 2023

unable to run certbot --nginx - UnicodeDecodeError: 'ascii' #5236

unable to run certbot --nginx - UnicodeDecodeError: 'ascii' #5236

Comments

john-gold commented Nov 12, 2017

My operating system is (include version):

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

I ran this command and it produced this output:

Certbot's behavior differed from what I expected because:

Here is a Certbot log showing the issue (if available):

Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:

joohoi commented Nov 13, 2017

shiroorg commented May 5, 2018

mardiros commented Jul 5, 2018

joohoi commented Jul 6, 2018

TommyZG commented Sep 2, 2018

Fretice commented Sep 7, 2018

Fretice commented Sep 13, 2018

Finesse commented Sep 15, 2018 • edited

da3020 commented Sep 18, 2018

ohemorange commented Sep 18, 2018

TommyZG commented Oct 29, 2018

A34 commented Apr 25, 2019 • edited

Kassan424kh commented Aug 20, 2019

Bill0412 commented Jan 22, 2020

mauryakrishna commented Jan 4, 2021 • edited

dsignr commented Apr 29, 2021

fandigunawan commented Feb 3, 2022

marcelomborges commented Sep 13, 2023

Finesse commented Sep 15, 2018 •

edited

A34 commented Apr 25, 2019 •

edited

mauryakrishna commented Jan 4, 2021 •

edited