SSL certificate failure when tearing down SRE

[jemrobinson]

✅ Checklist

• I have searched open and closed issues for duplicates.
• This is a problem observed when deploying a Data Safe Haven.
• I can reproduce this with the latest version.
• I have read through the documentation.
• This isn't an open-ended question (open a discussion if it is).

💻 System information

• Operating System: macOS
• Data Safe Haven version: develop @ 8e76a2f

🚫 Describe the problem

When an SRE is torn down, the SSL certificate fails with a message that it's already being deleted.

🌳 Log messages

Relevant log messages

2024-04-16 10:49:52 [   ERROR]     error: Exception calling application: Failed to delete SSL certificate apple-green-develop-turingsafehaven-ac-uk for apple.green.develop.turingsafehaven.ac.uk.                                                     cli.py:99
2024-04-16 10:49:52 [   ERROR]     Failed to remove certificate 'apple-green-develop-turingsafehaven-ac-uk' from Key Vault 'shmgreensreapplesecrets'.                                                                                                                cli.py:99
2024-04-16 10:49:52 [   ERROR]     (Conflict) Certificate is currently being deleted.                                                                                                                                                                                cli.py:99
2024-04-16 10:49:52 [   ERROR]     Code: Conflict                                                                                                                                                                                                                    cli.py:99
2024-04-16 10:49:52 [   ERROR]     Message: Certificate is currently being deleted.                                                                                                                                                                                  cli.py:99
2024-04-16 10:49:52 [   ERROR]     Inner error: {                                                                                                                                                                                                                    cli.py:99
2024-04-16 10:49:52 [   ERROR]         "code": "ObjectIsBeingDeleted"                                                                                                                                                                                                cli.py:99
2024-04-16 10:49:52 [   ERROR]     }

♻️ To reproduce

Attempt to teardown a Pulumi SRE

[JimMadge]

Possibly because SSL cert dynamic resource provider attempts to remove cert

data-safe-haven/data_safe_haven/infrastructure/components/dynamic/ssl_certificate.py

Lines 150 to 171 in bfbde01

	def delete(self, id_: str, props: dict[str, Any]) -> None:
	"""Delete an SSL certificate."""
	# Use `id` as a no-op to avoid ARG002 while maintaining function signature
	id(id_)
	try:
	# Remove the DNS record
	azure_api = AzureApi(props["subscription_name"], disable_logging=True)
	azure_api.remove_dns_txt_record(
	record_name="_acme_challenge",
	resource_group_name=props["networking_resource_group_name"],
	zone_name=props["domain_name"],
	)
	# Remove the Key Vault certificate
	azure_api.remove_keyvault_certificate(
	certificate_name=props["certificate_secret_name"],
	key_vault_name=props["key_vault_name"],
	)
	except Exception as exc:
	cert_name = f"[green]{props['certificate_secret_name']}[/]"
	domain_name = f"[green]{props['domain_name']}[/]"
	msg = f"Failed to delete SSL certificate {cert_name} for {domain_name}.\n{exc}"
	raise DataSafeHavenSSLError(msg) from exc

However, this will already have scheduled for deletion by tearing down the KeyVault.