LetsEncrypt ACME redirect issue fixes #2881

[EDIflyer]

Evolution of #2038 to escape regex sequence (as per #2038 (comment)) and rebased against latest develop branch.

[nginxproxymanagerci]

Docker Image for build 1 is available on DockerHub as jc21/nginx-proxy-manager:github-pr-3121

Note: ensure you backup your NPM instance before testing this PR image! Especially if this PR contains database changes.

[xipox]

i don't know why NPM isn't starting when it's trying to migrate the DBs
`

app_1 | - /data/nginx/stream/17.conf
app_1 | - /data/nginx/stream/13.conf
app_1 | - /data/nginx/stream/7.conf
app_1 | - /data/nginx/stream/2.conf
app_1 | - /data/nginx/stream/3.conf
app_1 | ❯ Docker secrets ...
app_1 |
app_1 | -------------------------------------
app_1 | _ _ ____ __ __
app_1 | | \ | | _ | / |
app_1 | | | | |) | |/| |
app_1 | | |\ | __/| | | |
app_1 | || _|| || |_|
app_1 | -------------------------------------
app_1 | User: npm PUID:0 ID:0 GROUP:0
app_1 | Group: npm PGID:0 ID:0
app_1 | -------------------------------------
app_1 |
app_1 | ❯ Starting nginx ...
app_1 | ❯ Starting backend ...
app_1 | [8/14/2023] [11:52:05 PM] [Global ] › ℹ info Using Sqlite: /data/database.sqlite
app_1 | [8/14/2023] [11:52:06 PM] [Migrate ] › ℹ info Current database version: none
app_1 | [8/14/2023] [11:58:34 PM] [Global ] › ✖ error Command failed: . /opt/certbot/bin/activate && pip install --no-cache-dir certbot-dns-cloudflare==$(certbot --version | grep -Eo '0-9+') cloudflare && deactivate
app_1 | WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b240>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b518>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b6a0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b7f0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b940>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | ERROR: Could not find a version that satisfies the requirement certbot-dns-cloudflare==2.5.0 (from versions: none)
app_1 | ERROR: No matching distribution found for certbot-dns-cloudflare==2.5.0
app_1 |
app_1 | [8/14/2023] [11:58:35 PM] [Migrate ] › ℹ info Current database version: none

`

[EDIflyer]

@jc21 any chance of merging this in? 🙂

[JDENredden]

This fixed my installation. Please merge.

[EDIflyer]

Thanks for approving @etabarestx 🙂 Do you know if a new release is upcoming with this and other PRs included?

[KaeTuuN]

Manually applied this fix via CLI and can confirm it works!

If anyone else doesn't want to wait for the merge, here is how to do it manually:

1. Open a shell as root in the docker container (varies, depending on your setup)
2. cd /etc/nginx/conf.d/include/ <- less typing
3. mv force-ssl.conf force-ssl.conf.bak <- don't change anything without a backup!
4. Copy the following codeblock:

cat <<'EOF' >force-ssl.conf     
# Since force-ssl.conf has now moved to the server section it overrides
# the LetsEncrypt config which is inside a location section
# Set FORCE variable in first two if tests pass and action in the third
set $FORCE "";
if ($scheme = "http") {
        set $FORCE 'H';
}
if ($request_uri !~ "^\/.well-known\/acme-challenge\/(.*)") {
        set $FORCE "${FORCE}D";
}
# If we are http and outside the LetsEncrypt directories redirect to https via 301
if ($FORCE = HD) {
        return 301 https://$host$request_uri;
}

5. Close your shell.
6. Restart the container.
7. REMINDER: This change will get lost if you update your Docker Image!

[evansharp]

Bumping to keep this alive.

[smibrandon]

I found a fix for my issue: allocating more storage space.

Running NPM in a Proxmox CT (no docker at all), and happened to catch that it was at 96% of its storage. I gave it some extra, and boom. Worked!

[Blogshot]

This has been an ongoing issue since 2022. How is this not merged yet?

As it stands, NPM will block renewals. I don't want to compromise security (-> disable "Force SSL") to enable renewals.

[tristanXme]

/bump please merge this!

[Guiorgy]

Applied this manually, but couldn't validate if it worked, since too many failed authorizations recently 😅.
Will update the comment later.
Edit: Didn't work for me, looks like I have a different problem.

[HarryVasanth]

This fixed my installation after manually applying this fix.
@jc21 could you please merge this when you have time? Thank you.