Unable to use controller-runtime for ClusterWorkspaceType initializer

[MatousJobanek]

Consider that I have a ClusterWorkspaceType and an initializer that is supposed to create an APIBinding. That initializer is watching a VirtualWorkspace of the given CWT, however, when it tries to create an APIBinding, then it fails with:

1.65719959317604e+09	ERROR	controller.clusterworkspace	Reconciler error	{"reconciler group": "tenancy.kcp.dev", "reconciler kind": "ClusterWorkspace", "name": "appp", "namespace": "", "error": "no matches for kind \"APIBinding\" in version \"apis.kcp.dev/v1alpha1\""}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/mjobanek/go-workspace/src/github.com/kcp-dev/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/mjobanek/go-workspace/src/github.com/kcp-dev/controller-runtime/pkg/internal/controller/controller.go:227

The reason is that the rest mapper tries to do the discovery call before calling the POST for the APIBinding. But it does it for the VirtualWorkspace endpoint with the wildcard /clusters/* at the end of the URL, not with the name of the workspace that is being initialized /clusters/root:plane:usersignup:foo/:

I0707 13:13:13.510869       1 round_trippers.go:463] GET https://192.168.1.133:6443/services/initializingworkspaces/root:plane:usersignup:Appstudio/clusters/%2A/api/v1
I0707 13:13:13.510875       1 round_trippers.go:469] Request Headers:
I0707 13:13:13.510882       1 round_trippers.go:473]     Authorization: Bearer <masked>
I0707 13:13:13.510888       1 round_trippers.go:473]     Accept: application/json, */*
I0707 13:13:13.521749       1 round_trippers.go:574] Response Status: 404 Not Found in 10 milliseconds
I0707 13:13:13.521845       1 request.go:1372] body was not decodable (unable to check for Status): couldn't get version/kind; json parse error: json: cannot unmarshal string into Go value of type struct { APIVersion string "json:\"apiVersion,omitempty\""; Kind string "json:\"kind,omitempty\"" }
I0707 13:13:13.524075       1 round_trippers.go:574] Response Status: 200 OK in 13 milliseconds
1.6571995935242307e+09	ERROR	controller.clusterworkspace	Reconciler error	{"reconciler group": "tenancy.kcp.dev", "reconciler kind": "ClusterWorkspace", "name": "appp", "namespace": "", "error": "no matches for kind \"APIBinding\" in version \"apis.kcp.dev/v1alpha1\""}

There are two problems in the rest mapper code:

1. The rest mapper doesn't use the cluster-aware client nor the cluster-aware round-tripper. But even if I modify the code so it uses the cluster-aware round-tripper, then I face the second problem:
2. The context that is being passed as part of the discovery call is context.TODO() https://github.com/kubernetes/client-go/blob/release-1.23/discovery/discovery_client.go#L172 and not the one that is provided from the controller. This means that the context doesn't have the cluster name value so the cluster-aware round-tripper cannot do its job.

[MatousJobanek]

I created a reproducer for this issue:
https://github.com/MatousJobanek/controller-runtime-example/tree/cwt-reproducer

1. build & deploy the controller
2. create workspace of the type Widget

[stevekuznetsov]

But it does it for the VirtualWorkspace endpoint, not for the actual workspace that is being initialized:

This is actually fine - the virtual workspace will proxy the request through to the correct cluster. The proximal issue here is that the request is not being made to the correct cluster.

[MatousJobanek]

Yeah, I used the wrong wording - what I meant was that it's using the URL with the wildcard at the end instead of putting the name of the workspace that is being initialized. I fixed it in the description.

[kcp-ci-bot]

Issues go stale after 90d of inactivity.
After a furter 30 days, they will turn rotten.
Mark the issue as fresh with /remove-lifecycle stale.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[kcp-ci-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten