Add cert-blocker status based on coverage guide for IoT test plans (BugFix) #1206
Conversation
The "Internet of Things Certified Hardware Coverage for Ubuntu Core 22 / Ubuntu 22.04" document specifies what jobs should be considered as blockers. For IoT devices, the after-suspend jobs are not considered blockers, so they have been left out. Fix CER-2593
pieqq
changed the title
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1206 +/- ##
=======================================
Coverage 43.32% 43.32%
=======================================
Files 356 356
Lines 38658 38658
Branches 6559 6559
=======================================
Hits 16750 16750
Misses 21245 21245
Partials 663 663
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
I've checked the cert-blocker status on the Desktop 22.04 test plan and it looks appropriate, so no changes are required. This PR is ready for review. |
There was a problem hiding this comment.
Hi @pieqq,
A few years earlier, there was a discussion between Cert/CE/OEMQA and the decision made in that meeting was[1]
Any job included in the project provider is a blocker
And the "CE IoT QA - Enablement Cycle QA Process" document has been updated according to that conclusion[2]
If we're now only adding blocker certification status according to the jammy test coverage guide then the certification scope will be changed for commercial enablement projects and ODM program projects as well.
Therefore I'd like to suggest applying blocker certification status to all the test jobs included in the client-cert-iot and client-cert-odm test plans.
[1] https://docs.google.com/document/d/1j0sOvJSODp_qlsTvjQcJA6umHSv-B18h8e506fz1sSE/edit#heading=h.6llmkwta1g0g
[2] https://docs.google.com/document/d/1QIaNbzu0bQXLMNTKPHJ21Tvb0upgHKb3UVvn0KQV6h8/edit#heading=h.pvqm21uto89l
Description
The "Internet of Things Certified Hardware Coverage for Ubuntu Core 22 / Ubuntu 22.04" document specifies what jobs should be considered as blockers.
For IoT devices, the after-suspend jobs are not considered blockers, so they have been left out.
Considerations
The document I used as a "source of truth" specifies a few things that look suspicious. Feedback from other teams would be appreciated:
Resolved issues
CER-2593
Documentation
Tests
Running a
list-bootstrappedof theclient-cert-iot-ubuntucore-22test plan on my laptop yields the following output:Notes:
non-blockerlist-bootstrapped, that's why they don't appear below