Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DAOS-15841 control: Improve coverage and fs mocking in storage packages #14379

Merged
merged 4 commits into from
May 20, 2024
Merged

DAOS-15841 control: Improve coverage and fs mocking in storage packages #14379

merged 4 commits into from
May 20, 2024

Conversation

tanabarr
Copy link
Contributor

@tanabarr tanabarr commented May 15, 2024 •
edited

Some control-plane storage unit tests are inadvertently calling into
test runner host OS filesystem. Fix by consolidating SystemProvider
interface and applying storage subsystem provider stubs by default
in the unit test framework. As a result coverage can be improved by
exercising a greater number of code paths.

Features: control
Required-githooks: true

Before requesting gatekeeper:

  • Two review approvals and any prior change requests have been resolved.
  • Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
  • Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
  • Commit messages follows the guidelines outlined here.
  • Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

  • You are the appropriate gatekeeper to be landing the patch.
  • The PR has 2 reviews by people familiar with the code, including appropriate owners.
  • Githooks were used. If not, request that user install them and check copyright dates.
  • Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
  • All builds have passed. Check non-required builds for any new compiler warnings.
  • Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
  • If applicable, the PR has addressed any potential version compatibility issues.
  • Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
  • Extra checks if forced landing is requested
    • Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
    • No new NLT or valgrind warnings. Check the classic view.
    • Quick-build or Quick-functional is not used.
  • Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

@tanabarr tanabarr self-assigned this May 15, 2024
@tanabarr tanabarr requested review from a team as code owners May 15, 2024 13:40
@github-actions GitHub Actions
Copy link

github-actions bot commented May 15, 2024 •
edited

Ticket title is 'Improve test coverage and filesystem mocking in storage packages'
Status is 'In Review'
Labels: 'Go,unit_test'
https://daosio.atlassian.net/browse/DAOS-15841

@tanabarr tanabarr added control-plane work on the management infrastructure of the DAOS Control Plane go Pull requests that update Go code labels May 15, 2024
@daosbuild1
Copy link
Collaborator

Test stage Build on Leap 15.5 with Intel-C and TARGET_PREFIX completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/1/execution/node/381/log

@daosbuild1
Copy link
Collaborator

Test stage Build on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/1/execution/node/386/log

@tanabarr tanabarr force-pushed the tanabarr/control-readfile-mocking branch from 936d31d to 5aa4b3c Compare May 15, 2024 13:47
@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/1/execution/node/342/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/1/execution/node/308/log

@tanabarr
DAOS-15841 control: Improve coverage and fs mocking in storage packages
1274a61 
Features: control
Required-githooks: true

Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>
@tanabarr tanabarr force-pushed the tanabarr/control-readfile-mocking branch from 5aa4b3c to 1274a61 Compare May 15, 2024 13:49
@tanabarr tanabarr changed the title DAOS-15841: Improve coverage and fs mocking in storage packages DAOS-15841 control: Improve coverage and fs mocking in storage packages May 15, 2024
@daosbuild1
Copy link
Collaborator

Test stage Build on Leap 15.5 with Intel-C and TARGET_PREFIX completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/3/execution/node/387/log

@daosbuild1
Copy link
Collaborator

Test stage Build on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/3/execution/node/359/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/3/execution/node/343/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/3/execution/node/380/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/3/execution/node/339/log

@daosbuild1
Copy link
Collaborator

Test stage Build DEB on Ubuntu 20.04 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14379/3/execution/node/355/log

@tanabarr
Merge remote-tracking branch 'origin/master' into tanabarr/control-re…
1099e1c 
…adfile-mocking

Features: control
Required-githooks: true

Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>
mjmac
mjmac previously approved these changes May 16, 2024
View reviewed changes
@tanabarr
revert monolith SystemProvider interface in favour of smaller per-pac…
de9574b 
…kage SystemProvider interfaces

Features: control
Required-githooks: true

Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>
@tanabarr
Merge remote-tracking branch 'origin/master' into tanabarr/control-re…
ab20315 
…adfile-mocking

Features: control
Signed-off-by: Tom Nabarro <tom.nabarro@intel.com>
@tanabarr tanabarr requested a review from mjmac May 16, 2024 19:25
kjacque
kjacque approved these changes May 16, 2024
View reviewed changes
Copy link
Contributor

@kjacque kjacque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't block on my comment, but it is something to think about. Everything else looks okay.

src/control/provider/system/mocks.go
StatErrors map[string]error
RealStat bool
ReadFileResults map[string][]byte
ReadFileErrors map[string]error
RealReadFile bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see RealStat already exists up there, but I don't feel great about adding this kind of complexity to a single mock struct, to toggle the real version of the function on and off. IMO, if there are certain tests that want some real and some mocked functions, those tests should create their own mock struct to customize the behavior.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding, it seems to have the advantage to factorize this mocking code and the complexity seems to still be acceptable.
However, as I am not a GO expert, I am probably missing something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@knard-intel those were my thoughts when adding the flag. After chatting with @kjacque I can see benefit in moving to smaller package-local (as @mjmac also suggested elsewhere) SystemProvider mocks which would be consistent with having package-level SystemProvider interfaces. I will make changes if I have to repush or the next time I visit this area.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense, thanks for the explanation :-)

knard-intel
knard-intel approved these changes May 17, 2024
View reviewed changes
Copy link
Contributor

@knard-intel knard-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tanabarr tanabarr requested a review from a team May 18, 2024 10:42
@tanabarr tanabarr requested a review from a team May 20, 2024 15:05
@phender phender merged commit ed21e30 into master May 20, 2024
57 checks passed
@phender phender deleted the tanabarr/control-readfile-mocking branch May 20, 2024 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@knard-intel knard-intel knard-intel approved these changes

@mjmac mjmac mjmac approved these changes

@kjacque kjacque kjacque approved these changes

Assignees

@tanabarr tanabarr

Labels
control-plane work on the management infrastructure of the DAOS Control Plane go Pull requests that update Go code
Milestone
No milestone
6 participants
@tanabarr @daosbuild1 @mjmac @kjacque @knard-intel @phender