Native support for Konnect integration in DataPlane

[czeslavo]

Problem Statement

Currently, to deploy DataPlanes that will sync with Konnect Control Plane, one has to define a very verbose PodTemplateSpec patch that will define all the required environment variables Kong Gateway needs for that purpose:

apiVersion: gateway-operator.konghq.com/v1beta1
kind: DataPlane
metadata:
  name: dataplane-example
  namespace: kong
spec:
  deployment:
    podTemplateSpec:
      spec:
        containers:
        - name: proxy
          image: kong/kong-gateway:3.6.1.3
          env:
            - name: KONG_ROLE
              value: data_plane
            - name: KONG_DATABASE
              value: "off"
            - name: KONG_CLUSTER_MTLS
              value: pki
            - name: KONG_CLUSTER_CONTROL_PLANE
              value: YOUR_CP_ID.us.cp0.konghq.com:443
            - name: KONG_CLUSTER_SERVER_NAME
              value: YOUR_CP_ID.us.cp0.konghq.com
            - name: KONG_CLUSTER_TELEMETRY_ENDPOINT
              value: YOUR_CP_ID.us.tp0.konghq.com:443
            - name: KONG_CLUSTER_TELEMETRY_SERVER_NAME
              value: YOUR_CP_ID.us.tp0.konghq.com
            - name: KONG_CLUSTER_CERT
              value: /etc/secrets/kong-cluster-cert/tls.crt
            - name: KONG_CLUSTER_CERT_KEY
              value: /etc/secrets/kong-cluster-cert/tls.key
            - name: KONG_LUA_SSL_TRUSTED_CERTIFICATE
              value: system
            - name: KONG_KONNECT_MODE
              value: "on"
            - name: KONG_VITALS
              value: "off"
          volumeMounts:
            - name: cluster-certificate
              mountPath: /var/cluster-certificate
            - name: kong-cluster-cert
              mountPath: /etc/secrets/kong-cluster-cert/
              readOnly: true
        volumes:
          - name: cluster-certificate
          - name: kong-cluster-cert
            secret:
              secretName: kong-cluster-cert
              defaultMode: 420

It's not the best user experience as there's a lot of repetition and it's easy to make a mistake.

We should make DataPlane natively integrated with Konnect by extending the CRD with a Konnect-specific spec section.

Proposed Solution

Extend DataPlaneOptions with Konnect section as below:

// DataPlaneOptions defines the information specifically needed to
// deploy the DataPlane.
type DataPlaneOptions struct {
        // ...

	// +optional
	Konnect *KonnectOptions `json:"konnect,omitempty"`
}

// DataPlaneKonnectKind defines the kind/role of the DataPlane deployment.
type DataPlaneKonnectKind string

const (
	ManagedByCloudControlPlaneDataPlaneKonnectKind DataPlaneKonnectKind = "managed-by-cloud-control-plane"

	// Note: just for future compatibility if we decide to implement Gateway (KIC + DPs) with Konnect integration as well.
	ManagedByIngressControllerPlaneDataPlaneKonnectKind = "managed-by-ingress-controller"
)

type KonnectOptions struct {
	// Kind defines the kind of the DataPlane deployment, which determines how it will be configured.
	Kind DataPlaneKonnectKind `json:"kind"`

	// ControlPlaneID is the identifier of the Konnect Control Plane.
	ControlPlaneID string `json:"controlPlaneID"`

	// ControlPlaneRegion is the region of the Konnect Control Plane.
	// If not set, 'us' is used as the default region.
	// +optional
	ControlPlaneRegion *string `json:"controlPlaneRegion,omitempty"`

	// ClusterCertificate is a name of the Secret containing the Konnect Control Plane's cluster certificate.
	ClusterCertificate string `json:"clusterCertificate"`
}

This will reduce the repetition and allow rigid validation of the input based on the defined schema.

Acceptance Criteria

• As a Konnect-hosted Control Plane user who wants to deploy DataPlanes in Kubernetes using KGO, I'm provided with an API that will allow me to configure those with minimal information provided (CP ID, cluster certificate secret name), so I do not have to know what exact environment variables need to be adjusted in their Deployments

[czeslavo]

@mheap After a discussion with @lahabana I created this issue to have it properly tracked in GH. I added an ad-hoc proposed solution, but I think we will need to iterate over it with a proper review.

BTW Should we create KGO 1.4 or "API summit" milestone and assign the issue to it?