[Nginx] Server Block Not Found in External Nginx Config Files with External global HTTP Block

[vanviethieuanh]

My operating system is (include version):

Linux - Debian 5.10.209-2 (2024-01-31) x86_64 GNU/Linux

I installed Certbot with:

snap

I ran this command:

sudo certbot --nginx -d example.com -d www.example.com # I replaced with my domain

and it produced this output:

Certbot could not find a block to include challenges in /etc/nginx/nginx.conf.

Certbot's behavior differed from what I expected because:

I expect it can find my include where I put by http block (in another file, not 'nginx.conf') and continue finding for the server block.

Here is a Certbot log showing the issue (if available):

DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 88, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot_nginx/_internal/configurator.py", line 1218, in perform
    http_response = http_doer.perform()
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot_nginx/_internal/http_01.py", line 64, in perform
    self._mod_config()
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot_nginx/_internal/http_01.py", line 131, in _mod_config
    raise errors.MisconfigurationError(
certbot.errors.MisconfigurationError: Certbot could not find a block to include challenges in /etc/nginx/nginx.conf.

DEBUG:certbot._internal.error_handler:Calling registered functions
INFO:certbot._internal.auth_handler:Cleaning up challenges
DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/snap/certbot/3700/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/_internal/main.py", line 1894, in main
    return config.func(config, plugins)
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/_internal/main.py", line 1450, in run
    new_lineage = _get_and_save_cert(le_client, config, domains,
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/_internal/main.py", line 143, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/_internal/client.py", line 517, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 88, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot_nginx/_internal/configurator.py", line 1218, in perform
    http_response = http_doer.perform()
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot_nginx/_internal/http_01.py", line 64, in perform
    self._mod_config()
  File "/snap/certbot/3700/lib/python3.8/site-packages/certbot_nginx/_internal/http_01.py", line 131, in _mod_config
    raise errors.MisconfigurationError(
certbot.errors.MisconfigurationError: Certbot could not find a block to include challenges in /etc/nginx/nginx.conf.

Here is the relevant nginx server block or Apache virtual host for the domain I am configuring:

worker_processes 1;

events {
    worker_connections 1024;
}

include /etc/nginx/conf.d/http.conf;

As I see:

In certbot/certbot-nginx/certbot_nginx/_internal/http_01.py:L87C1-L93C22 the code will try to find a http block in nginx.conf for start.

This will fail if I put my global http block in an external file like this case.

Admittedly, this scenario is somewhat uncommon, but it warrants consideration for enhancing the parsing and detection capabilities to handle all possible cases. Perhaps a gentle warning for users could be beneficial.

Thanks! You all are awesome!