Enforce strict host tag check #9017

vishesh92 · 2024-04-30T16:44:04Z

Description

Documentation PR: apache/cloudstack-documentation#398

Currently, an administrator can break host tag compatibility for a VM administrator by certain operations:

deploy/start VM on a specific host
migrate VM
restore VM
scale VM

This PR allows the user to specify tags which must be checked during these operations.

Global Settings

vm.strict.host.tags - A comma-separated list of tags for strict host check (Default - empty)
vm.strict.resource.limit.host.tag.check - Determines whether the resource limits tags are considered strict or not (Default - true)

During the above operations, we now check and throw an error if host tags compatibility is being broken for tags specified in vm.strict.host.tags. If vm.strict.resource.limit.host.tag.check is set to true, tags set in resource.limit.host.tags are also checked during these operations.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Screenshots (if appropriate):

How Has This Been Tested?

Manually tested

How did you try to break this feature and the system with this change?

codecov-commenter · 2024-04-30T17:04:15Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 4.26%. Comparing base (21af134) to head (5ae9fc3).

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #9017       +/-   ##
============================================
- Coverage     15.28%   4.26%   -11.02%     
============================================
  Files          5425     363     -5062     
  Lines        474138   29565   -444573     
  Branches      58984    5190    -53794     
============================================
- Hits          72486    1262    -71224     
+ Misses       393584   28160   -365424     
+ Partials       8068     143     -7925

Flag	Coverage Δ
uitests	`4.26% <ø> (ø)`
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

vishesh92 · 2024-04-30T20:46:41Z

@blueorangutan package

blueorangutan · 2024-04-30T20:48:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-04-30T22:13:14Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9491

vishesh92 · 2024-05-01T05:52:38Z

@blueorangutan test

blueorangutan · 2024-05-01T05:54:03Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

vishesh92 · 2024-05-01T17:20:41Z

@blueorangutan package

blueorangutan · 2024-05-01T17:22:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-05-01T18:33:23Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9507

vishesh92 · 2024-05-01T18:38:18Z

@blueorangutan test

blueorangutan · 2024-05-01T18:40:03Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-05-01T19:31:46Z

[SF] Trillian test result (tid-10102)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 47445 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9017-t10102-kvm-centos7.zip
Smoke tests completed. 128 look OK, 4 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	426.75	test_events_resource.py
test_01_restore_vm	`Error`	0.25	test_restore_vm.py
test_02_restore_vm_allocated_root	`Error`	0.15	test_restore_vm.py
ContextSuite context=TestRestoreVM>:teardown	`Error`	1.35	test_restore_vm.py
test_02_trigger_shutdown	`Failure`	346.70	test_safe_shutdown.py
test_02_restore_vm_strict_tags_failure	`Failure`	41.64	test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure	`Failure`	44.85	test_vm_strict_host_tags.py
test_06_deploy_vm_on_any_host_with_strict_tags_failure	`Failure`	1.30	test_vm_strict_host_tags.py
ContextSuite context=TestVMDeploymentPlannerStrictTags>:teardown	`Error`	53.96	test_vm_strict_host_tags.py

blueorangutan · 2024-05-02T09:44:50Z

[SF] Trillian test result (tid-10124)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 52549 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9017-t10124-kvm-centos7.zip
Smoke tests completed. 127 look OK, 5 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	428.10	test_events_resource.py
test_01_restore_vm	`Error`	0.24	test_restore_vm.py
test_02_restore_vm_allocated_root	`Error`	0.14	test_restore_vm.py
ContextSuite context=TestRestoreVM>:teardown	`Error`	1.23	test_restore_vm.py
test_02_list_cpvm_vm	`Failure`	0.05	test_ssvm.py
test_04_cpvm_internals	`Failure`	0.04	test_ssvm.py
test_01_secure_vm_migration	`Error`	135.30	test_vm_life_cycle.py
test_01_secure_vm_migration	`Error`	135.31	test_vm_life_cycle.py
test_02_restore_vm_strict_tags_failure	`Failure`	46.75	test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure	`Failure`	51.04	test_vm_strict_host_tags.py
test_06_deploy_vm_on_any_host_with_strict_tags_failure	`Failure`	1.31	test_vm_strict_host_tags.py
ContextSuite context=TestVMDeploymentPlannerStrictTags>:teardown	`Error`	77.52	test_vm_strict_host_tags.py

vishesh92 · 2024-05-02T13:18:58Z

@blueorangutan package

blueorangutan · 2024-05-02T13:20:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-05-02T14:38:36Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9517

vishesh92 · 2024-05-02T18:13:21Z

@blueorangutan test

blueorangutan · 2024-05-02T18:14:03Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-05-03T08:24:58Z

[SF] Trillian test result (tid-10146)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49258 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9017-t10146-kvm-centos7.zip
Smoke tests completed. 129 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	437.35	test_events_resource.py
test_01_restore_vm	`Error`	0.21	test_restore_vm.py
test_02_restore_vm_allocated_root	`Error`	0.14	test_restore_vm.py
ContextSuite context=TestRestoreVM>:teardown	`Error`	1.23	test_restore_vm.py
ContextSuite context=TestVMDeploymentPlannerStrictTags>:teardown	`Error`	59.13	test_vm_strict_host_tags.py

vishesh92 · 2024-05-04T06:37:22Z

@blueorangutan package

blueorangutan · 2024-05-04T06:38:04Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-05-04T07:54:08Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9532

vishesh92 · 2024-05-04T07:57:08Z

@blueorangutan test

blueorangutan · 2024-05-04T07:58:03Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-05-04T21:11:21Z

[SF] Trillian test result (tid-10154)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45981 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9017-t10154-kvm-centos7.zip
Smoke tests completed. 130 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	413.70	test_events_resource.py
test_01_restore_vm	`Error`	0.28	test_restore_vm.py
test_02_restore_vm_allocated_root	`Error`	0.16	test_restore_vm.py
ContextSuite context=TestRestoreVM>:teardown	`Error`	1.25	test_restore_vm.py

shwstppr

Code changes look good. Functionality will add flexibility for operators to control deployment/migration behaviours.
Needs manual testing

vishesh92 · 2024-05-07T07:51:58Z

@blueorangutan package

blueorangutan · 2024-05-07T07:54:07Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-05-07T09:13:59Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9560

vishesh92 · 2024-05-07T11:41:14Z

@blueorangutan test

blueorangutan · 2024-05-07T11:42:04Z

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-05-08T01:33:43Z

[SF] Trillian test result (tid-10182)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 48103 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9017-t10182-kvm-centos7.zip
Smoke tests completed. 131 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	428.02	test_events_resource.py

github-actions · 2024-05-08T15:29:29Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

vishesh92 · 2024-05-21T08:20:06Z

@blueorangutan package

blueorangutan · 2024-05-21T08:22:03Z

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-05-21T09:38:20Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9647

boring-cyborg bot added component:integration-test component:integration component:marvin Python Warning... Python code Ahead! labels Apr 30, 2024

vishesh92 force-pushed the enforce-strict-host-tag-check branch from 5e42531 to 3c53cf4 Compare April 30, 2024 20:46

vishesh92 mentioned this pull request May 1, 2024

Update documentation for strict host tags apache/cloudstack-documentation#398

Draft

vishesh92 force-pushed the enforce-strict-host-tag-check branch from a1091b0 to 8769a93 Compare May 2, 2024 12:54

shwstppr approved these changes May 6, 2024

View reviewed changes

DaanHoogland added the status:needs-testing label May 6, 2024

DaanHoogland closed this May 6, 2024

DaanHoogland reopened this May 6, 2024

apache deleted a comment from blueorangutan May 7, 2024

vishesh92 force-pushed the enforce-strict-host-tag-check branch from d14540b to 8ec0347 Compare May 7, 2024 05:08

vishesh92 added 5 commits May 7, 2024 13:17

Enforce strict host tag checking

e7ff032

Add e2e tests

1892426

Update global settings descrption

958c92a

fixup

5ffe5e2

Fix e2e test teardown

74977c4

vishesh92 force-pushed the enforce-strict-host-tag-check branch from 8ec0347 to 74977c4 Compare May 7, 2024 07:51

github-actions bot added the status:has-conflicts label May 8, 2024

Merge branch 'main' into enforce-strict-host-tag-check

5ae9fc3

github-actions bot removed the status:has-conflicts label May 21, 2024

Enforce strict host tag check #9017

Are you sure you want to change the base?

Enforce strict host tag check #9017

Conversation

vishesh92 commented Apr 30, 2024 • edited

Description

Global Settings

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov-commenter commented Apr 30, 2024 • edited by codecov bot

Codecov Report

vishesh92 commented Apr 30, 2024

blueorangutan commented Apr 30, 2024

blueorangutan commented Apr 30, 2024

vishesh92 commented May 1, 2024

blueorangutan commented May 1, 2024

vishesh92 commented May 1, 2024

blueorangutan commented May 1, 2024

blueorangutan commented May 1, 2024

vishesh92 commented May 1, 2024

blueorangutan commented May 1, 2024

blueorangutan commented May 1, 2024

blueorangutan commented May 2, 2024

vishesh92 commented May 2, 2024

blueorangutan commented May 2, 2024

blueorangutan commented May 2, 2024

vishesh92 commented May 2, 2024

blueorangutan commented May 2, 2024

blueorangutan commented May 3, 2024

vishesh92 commented May 4, 2024

blueorangutan commented May 4, 2024

blueorangutan commented May 4, 2024

vishesh92 commented May 4, 2024

blueorangutan commented May 4, 2024

blueorangutan commented May 4, 2024

shwstppr left a comment

Choose a reason for hiding this comment

vishesh92 commented May 7, 2024

blueorangutan commented May 7, 2024

blueorangutan commented May 7, 2024

vishesh92 commented May 7, 2024

blueorangutan commented May 7, 2024

blueorangutan commented May 8, 2024

github-actions bot commented May 8, 2024

vishesh92 commented May 21, 2024

blueorangutan commented May 21, 2024

blueorangutan commented May 21, 2024

vishesh92 commented Apr 30, 2024 •

edited

codecov-commenter commented Apr 30, 2024 •

edited by codecov bot