[API] Align DNS provider handling

[timuthy]

How to categorize this issue?

/area usability
/kind api-change

What would you like to be added:

Before the introduction of DNSRecords (#4231), shoot.spec.dns.providers could be reused by any shoot DNS extension (e.g. https://github.com/gardener/gardener-extension-shoot-dns-service).
Gardener created DNSProviders in the shoot control-plane, and DNS requests from workload within the shoot was redirected by the extension to the very same provider.
At the same time, the same primary DNS provider was used by Gardener to manage the shoot domain shoot.spec.dns.domain.

Today, handling for shoot domains (shoot.spec.dns.domain) happens via the Gardener "core" exclusive type DNSRecords. It is separated from any shoot DNS service, API objects and feature sets likewise.
DNS extensions need to provide their own provider configuration (example) instead of reusing what was once configured in shoot.spec.dns.providers.
As a consequence, nearly all providers in shoot.spec.dns.providers become obsolete, except the primary one (if available).

To clean up this technical debt and respect backwards compatibility we plan the following steps:

• Deprecate shoot.spec.dns.providers handling #9471:
  Deprecate support for non-primary DNS providers in shoot.spec.dns.providers. Give DNS extensions time to replicate providers to their provider config (already happens for github.com/gardener/gardener-extension-shoot-dns-service, see admission logic)

• Adapt handling of DNS Providers dashboard#1723
  Dashboard needs to adapt its DNS providers handling accordingly. Drop functionality or built exclusively for known github.com/gardener/gardener-extension-shoot-dns-service extension.

• DNS Extensions (esp. shoot-dns-service) to stop provider replication. This is a prerequisite for Gardener to remove non-primary providers.

• Remove all non-primary providers from shoot.spec.dns.providers for existing and new shoots.

• Remove obsolete fields domains and zones

We refrain from deprecating and dropping shoot.spec.dns.providers (in favor of shoot.spec.dns.provider) now because it'll affect too many users who need to adapt to this change, whereas we only gain little benefit out of it.
A future release of the v1.core.gardener.cloud might be a better scope for such adjustments.

/cc @MartinWeindel

[timuthy]

@MartinWeindel I created this issue to link it in the changes of #9145. Feel free to add your points.

[MartinWeindel]

Deactivation of the replication of the providers by the admission logic of the shoot-dns-service and the adaptions of the Gardener Dashboard need to be introduced with the same Gardener version (to be defined). The cleanup of the shoot.spec.dns.providers can also happen at the same time, but not earlier.

[MartinWeindel]

After switching to the new behaviour,I suggest to keep the UI for primary provider and the shoot-dns-service providers strictly separate and without any replication/synchronisation efforts.
The only exception could be if a custom domain is configured on shoot creation. In this case the Dashboard UI could replicate the provider to the shoot.extensions.providerConfig[@type="shoot-dns-service].providers section.

[timuthy]

/assign @MartinWeindel @timuthy