Error "unable to retrieve the complete list of server APIs: metrics.k8s.io" when any K8s resource is being created by operator through ansible K8s module.

[subrnath]

What happened?

We are trying to deploy a CNF in K8s cluster 1.23.1 with ansible operator from quay.io and following error is coming. This is seen when the metrics-server is installed. If metrics-server is deleted, this error is not seen and all the K8s resources are created fine by the Operator.

{"level":"error","ts":1646985248.079765,"logger":"proxy","msg":"Unable to determine if virtual resource","gvk":"/v1, Kind=Service","error":"unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: an error on the server ("Internal Server Error: "/apis/metrics.k8s.io/v1beta1?timeout=32s": the server could not find the requested resource") has prevented the request from succeeding","stacktrace":"github.com/operator-framework/operator-sdk/internal/ansible/proxy.(*cacheResponseHandler).ServeHTTP\n\t/workspace/internal/ansible/proxy/cache_response.go:97\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2879\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1930"}

Note 1: This issue is not seen in the previous version of K8s. Issue is coming after upgrade to K8s 1.23.

Note: In the role used by the operator, metrics.k8s.io is added.

apiGroups:
metrics.k8s.io
resources:
''
verbs:
''

What did you expect to happen?

No error when a k8s resource is created by operator through ansible k8s module.

name: Create Headless services
tags: deploy
k8s:
state: present
definition: "{{ lookup('template', 'app_services.j2') | from_yaml }}"
namespace: "{{ ansible_operator_meta.namespace }}"

How can we reproduce it (as minimally and precisely as possible)?

Deploy K8s 1.23.1
Deploy metrics server.
Run the ansible operator image from https://quay.io/repository/operator-framework/ansible-operator
Run any playbook from https://docs.ansible.com/ansible/2.9/modules/k8s_module.html which uses k8s module to access K8s API server.

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
1.23.1

</details>


### Cloud provider

NA.

K8s is installed with kubeadm.

### OS version

<details>

```console
# On Linux:
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.3 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic


$ uname -a

Linux lm913-Master 5.4.0-99-generic #112~18.04.1-Ubuntu SMP Thu Feb 3 14:09:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

/sig api-machinery
/wg api-expression

[pacoxu]

Helm fixed a similar issue helm/helm#6361 by helm/helm#6908.
/sig api-machinery instrumentation

[leilajal]

/triage accepted

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.