Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delegate HPA management to KEDA #14877

Open
skonto opened this issue Feb 12, 2024 · 2 comments
Open

Delegate HPA management to KEDA #14877

skonto opened this issue Feb 12, 2024 · 2 comments
Labels
kind/feature Well-understood/specified features, ready for coding.

Comments

@skonto
Copy link
Contributor

skonto commented Feb 12, 2024

Describe the feature

Knative's autoscaler-hpa manages hpa object on behalf of the user. For custom metrics users need to setup Prometheus and Prometheus adapter to register the metrics with the corresponding K8s api (example).
However Prometheus adapter is under maintenance upstream and KEDA among others supports hpa based autoscaling.
Given the popularity of KEDA for pull model, event based autoscaling delegating the hpa management to KEDA would help users to only deal with KEDA or Knative resources and not K8s low level resources.
That means Knative autoscaler-hpa controller could instead create scaledObjects on behalf of the user that map to a K8s HPA object managed by KEDA. This idea could be implemented as a Knative extension. Here is a poc of how this looks in practice as part of the current autoscaler-hpa component.

cc @dprotaso @ReToCode

@skonto skonto added the kind/feature Well-understood/specified features, ready for coding. label Feb 12, 2024
@yuzisun
Copy link

yuzisun commented Mar 2, 2024

We are interested in this too, as the request driven is not necessarily good for scaling LLM inference services, we need to scale based on token metrics.

@skonto
Copy link
Contributor Author

skonto commented Mar 6, 2024

I moved the code in a separate repo here, now it runs a standalone component. I will iterate on it and probably move it to knative-extensions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Well-understood/specified features, ready for coding.
Projects
None yet
Development

No branches or pull requests

2 participants