Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ListMeta to paginate large lists into more chunks #2574

Open
szuecs opened this issue Sep 4, 2023 · 0 comments
Open

Use ListMeta to paginate large lists into more chunks #2574

szuecs opened this issue Sep 4, 2023 · 0 comments

Comments

@szuecs
Copy link
Member

szuecs commented Sep 4, 2023

Is your feature request related to a problem? Please describe.

To make a better p99 latency in Kubernetes apiserver for LIST resources it makes sense to use pagination in dataclient/kubernetes resource listings.

Describe the solution you would like

Basically along the docs in https://kubernetes.io/docs/reference/using-api/api-concepts/#retrieving-large-results-sets-in-chunks
Some list object like:

type ListMetadata struct {
	*Metadata `json:"metadata"`
	// Continue is non-empty in case there are more items in the list to fetch from the apiserver
	// see also https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/list-meta/#ListMeta and https://kubernetes.io/docs/reference/kubernetes-api/common-parameters/common-parameters/#continue
	Continue string `json:"continue"`
	// Remaining is the number of remaining items to fetch in case there are more items in the list to fetch from the apiserver
	// see also https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/list-meta/#ListMeta
	Remaining int `json:"remainingItemCount"`
}

would be helpful to use pagination in all object listings.
Also getJSON() func needs to support it.

szuecs added a commit that referenced this issue Sep 4, 2023
fix: remove ListMetadata as we do not need it, follow up in #2574

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
szuecs added a commit that referenced this issue Sep 4, 2023
fix: remove ListMetadata as we do not need it, follow up in #2574

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant