Label filter ignored when using crictl stats

[jacksgt]

Hello,

thanks everyone for your work on CRIO and crictl!

What happened:

I was trying to extract some metrics from specific pods and came across the --label option for crictl stats.

 crictl stats -h
NAME:
   crictl stats - List container(s) resource usage statistics

USAGE:
   crictl stats [command options] [ID]

OPTIONS:
   --all, -a                  Show all containers (default shows just running) (default: false)
   --id value                 Filter by container id
   --label value              Filter by key=value label
   --output value, -o value   Output format, One of: json|yaml|table
   --pod value, -p value      Filter by pod id
   --seconds value, -s value  Sample duration for CPU usage in seconds (default: 1)
   --watch, -w                Watch pod resources (default: false)
   --help, -h                 show help (default: false)

Unfortunately, it seems that the label is completely ignored by the CLI - it simply lists all the pods!

crictl stats --label io.kubernetes.pod.namespace=openshift-image-registry | wc -l
46
crictl stats --label io\.kubernetes\.pod\.namespace=openshift-image-registry | wc -l
46

What you expected to happen:

When I filter according to a label (in this case: the namespace), I except to only get the metrics for those specific pods, like this:

crictl stats --label io.kubernetes.pod.namespace=openshift-image-registry | wc -l
1

How to reproduce it (as minimally and precisely as possible):

see previous sections

Anything else we need to know?:

This issue was previously reported in #448

Environment:

• Container runtime or hardware configuration:

$ crictl version
Version:  0.1.0
RuntimeName:  cri-o
RuntimeVersion:  1.22.1
RuntimeApiVersion:  v1alpha2
$ crio --version
crio version 1.22.1
Version:          1.22.1
GitCommit:        63ca93845d5fe05cdca826367afcb601ece8d7ad
GitTreeState:     clean
BuildDate:        2022-02-08T09:11:33Z
GoVersion:        go1.16.13
Compiler:         gc
Platform:         linux/amd64
Linkmode:         dynamic
BuildTags:        seccomp, selinux
SeccompEnabled:   true
AppArmorEnabled:  false

• OS (e.g: cat /etc/os-release): Fedora CoreOS 34
• Kernel (e.g. uname -a): Linux standard-tzbgh 5.14.14-200.fc34.x86_64 #1 SMP Wed Oct 20 16:15:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[jacksgt]

The issue is still present and the root cause is not understood.

/remove-lifecycle stale

/lifecycle frozen

[kundan2707]

/remove-lifecycle frozen

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[jacksgt]

/remove-lifecycle stale

/lifecycle frozen

[brianpursley]

This appears to be working for me, with containerd.

ubuntu@k8s-worker-2:~$ sudo crictl stats --label io.kubernetes.pod.name=foo | wc -l
3
ubuntu@k8s-worker-2:~$ sudo crictl stats | wc -l
7
ubuntu@k8s-worker-2:~$ sudo crictl version
Version:  0.1.0
RuntimeName:  containerd
RuntimeVersion:  1.6.12-0ubuntu1~20.04.1
RuntimeApiVersion:  v1

---

The problem is in cri-o.

Notice ListContainerStats only calls filterContainerList() (which only filters on container ID or pod sandbox ID), but it also needs to call filterContiner() (which filters on labels):

Here is an example in cri-o where it does filter correctly, by calling both funcs:
https://github.com/cri-o/cri-o/blob/fd21f12597cc205e61f10def9849d70cb1220b3f/server/container_list.go#L75-L88

ListContainerStats needs to be changed to be like that.