Get ACMEv2 support in Certbot

[ohemorange]

Step 2 of #5365

Certbot core

• Delete lines from certbot.util.enforce_domain_sanity that error on wildcards. We should still have code somewhere that errors if a wildcard is requested from a v1 endpoint because according to the spec "Wildcard domain names (with "*" as the first label) MUST NOT be included in authorization requests" and we should provide a clean error message to the user.
• Improve error message if a wildcard is requested with an authenticator that can't do DNS challenges.
• Don't make cert names with wildcards by default.
• Fix --allow-subset-of-names for ACMEv2. This could be done in certbot.client.obtain_certificate, by having it repeatedly create a CSR, request a new_order, and perform authorizations until all authorizations pass. If any fail, we should create a new CSR with only the domains that succeeded. The order/authorizations would then be passed to obtain_certificate_from_csr.
• Need to show link to TOS before calling register/new_account on the client from the acme module when ACMEv2 is used. Doing this requires being able to get TOS from ACMEv2 directory, ideally without making the user of the ACME library reach into the directory object themselves. We also need to skip this code for ACMEv1 and not try and show TOS again after registering for ACMEv2. (Suggests acme change)
• Call new_account/register based on ACME version (Could be mitigated by Add automatic ACME protocol detection mechanism to the acme library #5491 or changes to acme.client.ClientV2)
• If current design of ACMEv2: Add Order support #5374 stays, we have to make CSR before performing challenges in certbot.client.obtain_certificate because the CSR is needed earlier by the client in ACME. (Could be mitigated by Add automatic ACME protocol detection mechanism to the acme library #5491 or changes to ACMEv2: Add Order support #5374)
• certbot.auth_handler.get_authorizations (or closely related function) should take a dict/list of authorization objects instead of domains. We can have another function on the auth_handler take domains and create authzs if we wanted for convenience of supporting ACMEv1 (Could be mitigated by Add automatic ACME protocol detection mechanism to the acme library #5491 by keeping order object internal to acme client)
• We now get back the fullchain from the ACME server rather than separate leaf and chain certs. These need to be split (probably by just looking for the first line containing "END CERTIFICATE"). (Could be mitigated by Add automatic ACME protocol detection mechanism to the acme library #5491)

Apache and Nginx

• Open question: Should Apache/Nginx suggest wildcards found in the config with get_all_names?
• Have Nginx and Apache installers support wildcards (deploy and enhance) in the case when there is one vhost matching the wildcard. Doing this involves making sure we can properly search the config for the vhost serving the wildcard (doing this 100% correctly could be hard/impossible for server names that are regexes) and fixing small things that break if wildcards are used. For example, if ($host = *.example.com) { doesn't make sense for Nginx redirects.
• Possibly have Apache and Nginx installers (offer to) modify all vhosts that match the wildcard.

[bmw]

@jsha, I could use an ACMEv2 sanity check.

Certbot has a flag called --allow-subset-of-names which causes it to issue a cert if it's able to obtain a valid authz for any of the requested domains. Any domains where Certbot failed DV are not added to the CSR. This flag has been controversial and we've considered deprecating/removing it (see #4470), but just to make sure I properly understand our options here, if we wanted to support this flag in ACMEv2, we'd have to create a new order and potentially redo all authorizations correct? (Boulder might automatically reuse authzs, but an arbitrary ACME server might not.)

[jsha]

Yep, you'd create a second order. In practice Boulder would reuse the valid authorization, but true that that is not guaranteed by ACME.

[bmw]

@joohoi and @ohemorange, we should come up with a plan for how we're going to handle wildcards in Apache and Nginx for deploy_cert and enhance. It'd also be great if you guys can own those tasks respectively. We should aim to have support for this landed in less than two weeks from now. (If that means I should take something from you, let me know.)

I believe both of you have expressed a preference for the same plan: If the domain is a wildcard, show the user a list of vhosts to choose from. As far as I'm concerned, this can be all vhosts we'd consider for the request for now. Allow the user to select multiple vhosts and perform the enhancement or deploy the cert to all of them. We should make sure that things like redirects work for wildcard certs.

Any objections or better ideas?

(As a minor note, Certbot could ask Apache/Nginx to perform a challenge for a wildcard domain if a future ACME server supports wildcard issuance for HTTP/TLSSNI. Not sure what the (current) CA baseline requirements say here, but ACME has no objections to this. With this in mind, it might not be a bad idea to add a check for this early in perform that raises an exception.)

[joohoi]

we should come up with a plan for how we're going to handle wildcards in Apache and Nginx for deploy_cert and enhance. It'd also be great if you guys can own those tasks respectively. We should aim to have support for this landed in less than two weeks from now. (If that means I should take something from you, let me know.)

As we discussed on the call, I'm going to take the Apache part here.

I believe both of you have expressed a preference for the same plan: If the domain is a wildcard, show the user a list of vhosts to choose from. As far as I'm concerned, this can be all vhosts we'd consider for the request for now. Allow the user to select multiple vhosts and perform the enhancement or deploy the cert to all of them. We should make sure that things like redirects work for wildcard certs.

Yeah, I'm still convinced that this is the best option.

(As a minor note, Certbot could ask Apache/Nginx to perform a challenge for a wildcard domain if a future ACME server supports wildcard issuance for HTTP/TLSSNI. Not sure what the (current) CA baseline requirements say here, but ACME has no objections to this. With this in mind, it might not be a bad idea to add a check for this early in perform that raises an exception.)

I don't think it could and/or should happen, as I tried to explain here, so I'd be in favor of not spending cycles on this ATM.

[bmw]

This has been resolved!