proposal: x/crypto/acme/autocert: Manager should support DNS-01 verification

[winteraz]

What did you do?

I've tried to setup autocert behind a firewall.

What did you expect to see?

https working flawlessly (using letsencrypt infrastructure)

What did you see instead?

Verification failed due the firewall.

I believe dns-01 should be built into Manager. It could have a function (i.e. SetTXT) field which if mutated is used by the Manager to set the TXT records required for the DNS verification.

[bradfitz]

Who terminates your TLS? What's your setup look like?

We're really trying to keep the autocert package simple and move any complexity into the acme package. Generally our answer for people with non-standard setups is to have them use the acme package directly, rather than try to make the autocert package be all things to all people.

If we did support dns-01, I'd rather it be automatic. What do your firewall allow in & out?

[bradfitz]

/cc @x1ddos

[winteraz]

A function in Manager(i.e. type DNSUpdater func(k, v)error // updates DNS TXT records with key/name k and value k should suffice and allow the setup to be fully automatic.

Depending on the DNS provider the client/developer will provide the appropriate DNSUpdater function.
The changes would be minimal and I believe all of them should be in the verify method. The dns-01 verification seems already in developed in the acme package. https://github.com/golang/crypto/blob/master/acme/autocert/autocert.go#L491
I don't think it will make it significantly more complex than it already is.

[bradfitz]

@winteraz, I'm not asking about solutions. We try to describe & understand the problem before jumping to solutions. Could you answer my questions above?

[winteraz]

My server has all incoming traffic restricted to a limited set of IPs so letsencrypt can't access it for the tls-sni challenge verification. The outgoing traffic has no restriction. I used to allow incoming traffic temporarily(as long as it took to set-up the SSL) but this requires manual intervention and defeats the purpose of ACME.

[winteraz]

This may not be the most popular setup but firewalls are not that uncommon. The main issue is that Letsencrypt doesn't advertise their IP addresses and they actually forbid whitelist practices. A simple google "Letsencrypt firewall" may prove the issue is quite common and their response is to use dns-01.

[winteraz]

It's also worth to note that the upcoming wildcard certificates will be available using only dns-01

We intend to support wildcard certificates in January 2018 as part of the ACMEv2 endpoint. Wildcard issuance will require base domain validation using DNS-01 challenges.

https://letsencrypt.org/docs/faq/

[x1ddos]

The changes would be minimal and I believe all of them should be in the verify method. The dns-01 verification seems already in developed in the acme package. https://github.com/golang/crypto/blob/master/acme/autocert/autocert.go#L491
I don't think it will make it significantly more complex than it already is.

I actually doubt the changes would be minimal. The tls-sni verification is almost instant, whereas anything DNS related may take hours. It's quite a different flow, i.e. out-of-band.

[winteraz]

@x1ddos below are the changes required. I've been using the fork for several days. I still stand by my statement that the changes are minimal. I'm using route53 and the DNS verification is almost instant(i.e. takes few seconds).
golang/crypto@master...winteraz:master

[x1ddos]

Given the "tls-sni-xx" is probably gone for good, "http-01" in #21890 will become the default. Maybe we should reconsider and also add "dns-01" in case something happens to "http-01" too.

[rusenask]

Hello, I have been looking into this as well. I agree with @x1ddos that DNS challenge might be too slow for some (might be the most) of the cases, but this recent issue with tls-sni demonstrated that we need to have multiple options as a lot of applications are now helpless.

[mpx]

I think it's a little early to say tls-sni is "probably gone for good", it's going to take a while before anyone knows how this is going to work out.

There are active discussions looking at how tls-sni might be fixed. Given the benefits of using SNI, I suspect it's more likely there will be a replacement -- how long it takes is another question.

[x1ddos]

I think it's a little early to say tls-sni is "probably gone for good"

From https://community.letsencrypt.org/t/2018-01-11-update-regarding-acme-tls-sni-and-shared-hosting-infrastructure/50188:

The ACME TLS-SNI-01 validation method will remain disabled permanently for new accounts by default. Since the same problems apply to TLS-SNI-02, TLS-SNI-02 will remain disabled in our upcoming ACMEv2 API endpoint.

Mitigations for Existing TLS-SNI Users

Our recommendation for users is to begin a migration to the HTTP-01 or DNS-01 validation methods. We are working to provide a reasonable amount of migration time for as many users as possible, while maintaining our commitment to security.