Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

koordlet: add mid runtimehooks #1984

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

j4ckstraw
Copy link
Contributor

@j4ckstraw j4ckstraw commented Apr 3, 2024

Ⅰ. Describe what this PR does

Ⅱ. Does this pull request fix one issue?

fix #1442

Ⅲ. Describe how to verify it

Ⅳ. Special notes for reviews

V. Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests
  • All checks passed in make test

@koordinator-bot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign fillzpp, jasonliu747 after the PR has been reviewed.
You can assign the PR to them by writing /assign @fillzpp @jasonliu747 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

codecov bot commented Apr 3, 2024

Codecov Report

Attention: Patch coverage is 92.85714% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 68.82%. Comparing base (f85aefc) to head (6cc39a2).
Report is 20 commits behind head on main.

Files Patch % Lines
pkg/koordlet/runtimehooks/reconciler/reconciler.go 70.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1984      +/-   ##
==========================================
+ Coverage   68.77%   68.82%   +0.04%     
==========================================
  Files         422      422              
  Lines       39058    39072      +14     
==========================================
+ Hits        26864    26891      +27     
+ Misses       9870     9852      -18     
- Partials     2324     2329       +5     
Flag Coverage Δ
unittests 68.82% <92.85%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zwzhang0107 zwzhang0107 changed the title Add mid runtimehooks koordlet: add mid runtimehooks Apr 7, 2024
@j4ckstraw
Copy link
Contributor Author

I have no idea how to deal with the BatchResource featuregate, What do you think? @zwzhang0107

@j4ckstraw j4ckstraw marked this pull request as ready for review April 7, 2024 07:18
@j4ckstraw
Copy link
Contributor Author

/rerun

return apiext.GetQoSClassByAttrs(labels, annotations) == apiext.QoSBE
func isPodQoSMatchedByAttr(labels map[string]string, annotations map[string]string) bool {
podQoS := apiext.GetQoSClassByAttrs(labels, annotations)
return podQoS == apiext.QoSBE || podQoS == apiext.QoSLS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which cgroup will Mid+LS pods use
/besteffort may be not appropriate
more things need to be clarify if using /burstable, such as the fairness of cpushares value between Prodand Mid

Copy link
Contributor Author

@j4ckstraw j4ckstraw Apr 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is just a filter which means it allow LS and BE.
I think mid+LS pods use /besteffort or /burstable should be restricted in validating webhook.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mid+BE in /besteffort seems reasonable, while Mid+LS should better use /burstable(but this needs some hack works since kubelet is not allowed yet)

I think now we can let koordlet supports Mid+BE first.

So here are the things we should fix:

  1. keep podQOSConditions still as apiext.QoSBE
  2. define independent GetContainerMidMilliCPURequest and Memory instead of using GetContainerBatchMilliCPURequest

Moreover, there are several QoS policies for BE, we need to clarify whether they should support Mid+BE and how to support, for example the evict order of pods. it will be better if we have some docs first.

Signed-off-by: j4ckstraw <j4ckstraw@foxmail.com>

add test for util/extend_resource

Signed-off-by: j4ckstraw <j4ckstraw@foxmail.com>
Signed-off-by: j4ckstraw <j4ckstraw@foxmail.com>
Signed-off-by: j4ckstraw <j4ckstraw@foxmail.com>
Signed-off-by: j4ckstraw <j4ckstraw@foxmail.com>
Signed-off-by: j4ckstraw <j4ckstraw@foxmail.com>
Signed-off-by: j4ckstraw <j4ckstraw@foxmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[proposal] Support the resource management for Mid-tier pods.
2 participants