koordlet: tc plugin for netqos #1920

lucming · 2024-02-23T10:58:44Z

Ⅰ. Describe what this PR does

A plugin for netqos, and based on tc.
other discussion: #1764

btw, there may be some duplicate defination with pr #1843 .

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it

Ⅳ. Special notes for reviews

V. Checklist

I have written necessary docs and comments
I have added necessary unit tests and integration tests
All checks passed in make test

koordinator-bot · 2024-02-23T10:58:52Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign hormes after the PR has been reviewed.
You can assign the PR to them by writing /assign @hormes in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

zwzhang0107 · 2024-02-27T06:25:28Z

it will be great if you can add some implement details in doc about tc plugin https://github.com/koordinator-sh/koordinator/blob/main/docs/proposals/koordlet/20231208-support-netqos.md#internal-plugins

lucming · 2024-02-27T12:53:46Z

it will be great if you can add some implement details in doc about tc plugin https://github.com/koordinator-sh/koordinator/blob/main/docs/proposals/koordlet/20231208-support-netqos.md#internal-plugins

write in this pr?

zwzhang0107 · 2024-03-08T08:57:15Z

it will be great if you can add some implement details in doc about tc plugin https://github.com/koordinator-sh/koordinator/blob/main/docs/proposals/koordlet/20231208-support-netqos.md#internal-plugins

write in this pr?

another one with only proposal is better

lucming · 2024-03-15T07:15:11Z

it will be great if you can add some implement details in doc about tc plugin https://github.com/koordinator-sh/koordinator/blob/main/docs/proposals/koordlet/20231208-support-netqos.md#internal-plugins

write in this pr?

another one with only proposal is better

done. and this is proposal pr: #1954

pkg/koordlet/runtimehooks/config.go

pkg/koordlet/runtimehooks/hooks/tc/tc.go

apis/extension/netqos_tc.go

pkg/koordlet/runtimehooks/hooks/tc/iptables.go

pkg/koordlet/util/system/common_linux.go

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go

pkg/koordlet/runtimehooks/hooks/tc/iptables.go

zwzhang0107 · 2024-03-19T11:49:12Z

codecov · 2024-03-25T11:47:54Z

Codecov Report

Attention: Patch coverage is 28.92460% with 575 lines in your changes are missing coverage. Please review.

Project coverage is 67.78%. Comparing base (44d8d1b) to head (769d712).

Files	Patch %	Lines
pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go	25.67%	347 Missing and 9 partials ⚠️
pkg/koordlet/runtimehooks/hooks/tc/rule.go	14.11%	72 Missing and 1 partial ⚠️
pkg/koordlet/runtimehooks/hooks/tc/netqos_tc.go	0.00%	41 Missing ⚠️
pkg/koordlet/runtimehooks/hooks/tc/iptables.go	27.08%	34 Missing and 1 partial ⚠️
pkg/koordlet/runtimehooks/hooks/tc/ipset.go	33.33%	19 Missing and 1 partial ⚠️
pkg/koordlet/runtimehooks/protocol/pod_context.go	0.00%	10 Missing and 1 partial ⚠️
pkg/koordlet/resourceexecutor/reader.go	0.00%	10 Missing ⚠️
pkg/koordlet/runtimehooks/hooks/tc/helper.go	87.09%	7 Missing and 1 partial ⚠️
pkg/koordlet/runtimehooks/reconciler/reconciler.go	0.00%	8 Missing ⚠️
pkg/koordlet/runtimehooks/protocol/protocol.go	0.00%	6 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1920      +/-   ##
==========================================
- Coverage   68.57%   67.78%   -0.80%     
==========================================
  Files         430      436       +6     
  Lines       39383    40192     +809     
==========================================
+ Hits        27007    27244     +237     
- Misses      10037    10594     +557     
- Partials     2339     2354      +15

Flag	Coverage Δ
unittests	`67.78% <28.92%> (-0.80%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go

pkg/koordlet/runtimehooks/hooks/tc/netqos_tc.go

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go

zwzhang0107 · 2024-05-06T03:15:39Z

go mod tidy?

zwzhang0107 · 2024-05-06T03:31:28Z

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go

+		newRule.enable = false
+	}
+
+	p.rule.Update(&newRule)


check whether rule is updated with reflect.DeepEqual

zwzhang0107 · 2024-05-06T03:57:12Z

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go

+	}
+
+	return apierror.NewAggregate([]error{
+		p.ensureFilter(filterKey(strconv.Itoa(SYSTEM_CLASS_MINOR_ID)), genFilterCmd("add", SYSTEM_CLASS_PRIO, SYSTEM_CLASS_MINOR_ID),


could add/change be seperated? so we don't need add if already exist.

zwzhang0107 · 2024-05-06T06:15:17Z

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go

+		}
+		needLimitAtPodLevel := ing != 0 || egress != 0
+
+		if pod.Pod.DeletionTimestamp != nil {


pod may be already deleted before we catch the deleteTimestamp

maybe we should list all rules and delete those danglings.

zwzhang0107 · 2024-05-06T06:24:03Z

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go

+
+	minorHex := ""
+	key := string(uid)
+	for i := 0; i < 5; i++ {


maybe we can use other hash algo to avoid the conflicts.

bysph · 2024-05-11T01:34:34Z

pkg/koordlet/runtimehooks/hooks/tc/ipset.go

+}
+
+func (p *tcPlugin) DestoryIpset() error {
+	klog.V(5).Infof("start to delete ipset rules crated by tc plugin.")


crated -> created

bysph · 2024-05-11T01:44:03Z

pkg/koordlet/runtimehooks/hooks/tc/iptables.go

+}
+
+func (p *tcPlugin) DelIptables() error {
+	klog.V(5).Infof("start to delete iptables rules crated by tc plugin.")


crated -> created

zwzhang0107 · 2024-05-21T01:57:43Z

code needs rebase
make lint?

Signed-off-by: lucming <2876757716@qq.com>

koordinator-bot bot requested review from eahydra, FillZpp, jasonliu747, saintube and yihuifeng February 23, 2024 10:58

koordinator-bot bot added the size/XXL label Feb 23, 2024

lucming force-pushed the netqos-tc branch 9 times, most recently from 7205dd1 to fa87361 Compare February 23, 2024 13:53

lucming force-pushed the netqos-tc branch 3 times, most recently from 278d46a to 4f2060b Compare March 5, 2024 07:26

lucming force-pushed the netqos-tc branch 2 times, most recently from 03257aa to 96ae4e0 Compare March 15, 2024 07:14

zwzhang0107 reviewed Mar 19, 2024

View reviewed changes

lucming force-pushed the netqos-tc branch from 96ae4e0 to e656c39 Compare March 25, 2024 11:41

lucming force-pushed the netqos-tc branch from e656c39 to d190e52 Compare March 28, 2024 08:23

lucming force-pushed the netqos-tc branch 4 times, most recently from 347bd58 to 12deec2 Compare March 29, 2024 03:51

zwzhang0107 reviewed Mar 29, 2024

View reviewed changes

lucming force-pushed the netqos-tc branch from 12deec2 to 8372870 Compare March 29, 2024 12:07

lucming force-pushed the netqos-tc branch 8 times, most recently from 058a017 to 1c8ced4 Compare April 21, 2024 22:53

zwzhang0107 reviewed Apr 22, 2024

View reviewed changes

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go Outdated Show resolved Hide resolved

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go Outdated Show resolved Hide resolved

zwzhang0107 reviewed Apr 23, 2024

View reviewed changes

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go Outdated Show resolved Hide resolved

zwzhang0107 reviewed Apr 23, 2024

View reviewed changes

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go Outdated Show resolved Hide resolved

pkg/koordlet/runtimehooks/hooks/tc/tc_linux.go Show resolved Hide resolved

lucming force-pushed the netqos-tc branch 2 times, most recently from 05a8197 to 85a32a3 Compare May 5, 2024 15:45

zwzhang0107 reviewed May 6, 2024

View reviewed changes

bysph reviewed May 11, 2024

View reviewed changes

lucming force-pushed the netqos-tc branch 2 times, most recently from e08de79 to 72148e2 Compare May 19, 2024 07:33

lucming force-pushed the netqos-tc branch 3 times, most recently from 9a764c5 to eb5f2af Compare May 21, 2024 06:26

koordlet: tc plugin for netqos

769d712

Signed-off-by: lucming <2876757716@qq.com>

lucming force-pushed the netqos-tc branch from eb5f2af to 769d712 Compare May 21, 2024 06:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

koordlet: tc plugin for netqos #1920

koordlet: tc plugin for netqos #1920

lucming commented Feb 23, 2024

koordinator-bot bot commented Feb 23, 2024

zwzhang0107 commented Feb 27, 2024

lucming commented Feb 27, 2024

zwzhang0107 commented Mar 8, 2024

lucming commented Mar 15, 2024 •

edited

zwzhang0107 commented Mar 19, 2024

codecov bot commented Mar 25, 2024 •

edited

zwzhang0107 commented May 6, 2024

zwzhang0107 May 6, 2024

zwzhang0107 May 6, 2024

zwzhang0107 May 7, 2024

zwzhang0107 May 6, 2024

zwzhang0107 May 6, 2024

bysph May 11, 2024

bysph May 11, 2024

zwzhang0107 commented May 21, 2024

koordlet: tc plugin for netqos #1920

Are you sure you want to change the base?

koordlet: tc plugin for netqos #1920

Conversation

lucming commented Feb 23, 2024

Ⅰ. Describe what this PR does

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it

Ⅳ. Special notes for reviews

V. Checklist

koordinator-bot bot commented Feb 23, 2024

zwzhang0107 commented Feb 27, 2024

lucming commented Feb 27, 2024

zwzhang0107 commented Mar 8, 2024

lucming commented Mar 15, 2024 • edited

zwzhang0107 commented Mar 19, 2024

codecov bot commented Mar 25, 2024 • edited

Codecov Report

zwzhang0107 commented May 6, 2024

zwzhang0107 May 6, 2024

Choose a reason for hiding this comment

zwzhang0107 May 6, 2024

Choose a reason for hiding this comment

zwzhang0107 May 7, 2024

Choose a reason for hiding this comment

zwzhang0107 May 6, 2024

Choose a reason for hiding this comment

zwzhang0107 May 6, 2024

Choose a reason for hiding this comment

bysph May 11, 2024

Choose a reason for hiding this comment

bysph May 11, 2024

Choose a reason for hiding this comment

zwzhang0107 commented May 21, 2024

lucming commented Mar 15, 2024 •

edited

codecov bot commented Mar 25, 2024 •

edited