Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement filtering by packages through the config #944

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

josieang
Copy link
Collaborator

@josieang josieang commented Apr 26, 2024

I'd like feedback on the config yaml schema, the filter message and it's behaviour if the version is empty (it filters any version of that package).

This is in response to #814

pkg/config/config.go Outdated Show resolved Hide resolved
pkg/config/config.go Outdated Show resolved Hide resolved
pkg/osvscanner/osvscanner.go Show resolved Hide resolved
@josieang
Copy link
Collaborator Author

I think I need a better way to filter based on semver parts. I know think deps.dev/util/semver does this, I will come back to it once I'm back from leave.

Copy link
Collaborator

@another-rex another-rex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Main comment is that I think we can consolidate the two overrides into one, so they can share the package matching logic.

LoadPath string `toml:"LoadPath"`
GoVersionOverride string `toml:"GoVersionOverride"`
IgnoredVulns []IgnoreEntry `toml:"IgnoredVulns"`
IgnoredPackageVersions []IgnorePackageVersionEntry `toml:"IgnoredPackageVersions"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like ignore package versions and override package versions both could have the same "package matching" logic.

Can we consolidate them into the same field, and just have a bool ignore which if true ignores the package entirely, otherwise an override field where someone can override the license.

E.g.

[[Package]]
name = "pkg-name"
exactVersion = "1.0.0"
ecosystem = "Go"
ignore = false # if true ignores the package entirely
licenseOverride = ["MIT", "0BSD"]
# In the future, if someone requests we can also add a versionOverride field here, or other overrides
reason = "abc"
# major ...
# minor ...
# ...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1.

version (Which I think we should exactVersion to) is optional right?

Additionally, would it make sense to make license a structured field of some sort instead?

I.e.

[[Package]]
[[Package.license]]
override = ["MIT"]

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me! done.

pkg/config/config.go Outdated Show resolved Hide resolved
pkg/osvscanner/osvscanner.go Show resolved Hide resolved
Allowlist: []models.License{models.License("MIT"), models.License("0BSD")},
},
},
Results: []models.PackageSource{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For maintainability, can we just use the snapshot library here to store/match the results?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be done but check my code just in case I'm not understanding the snapshot library correctly

pkg/config/config.go Outdated Show resolved Hide resolved
pkg/config/config.go Outdated Show resolved Hide resolved
LoadPath string `toml:"LoadPath"`
GoVersionOverride string `toml:"GoVersionOverride"`
IgnoredVulns []IgnoreEntry `toml:"IgnoredVulns"`
IgnoredPackageVersions []IgnorePackageVersionEntry `toml:"IgnoredPackageVersions"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1.

version (Which I think we should exactVersion to) is optional right?

Additionally, would it make sense to make license a structured field of some sort instead?

I.e.

[[Package]]
[[Package.license]]
override = ["MIT"]

pkg/config/config.go Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented May 7, 2024

Codecov Report

Attention: Patch coverage is 81.96721% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 63.75%. Comparing base (cc7261e) to head (a11bd38).
Report is 1 commits behind head on main.

Files Patch % Lines
pkg/osvscanner/osvscanner.go 8.33% 10 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #944      +/-   ##
==========================================
+ Coverage   63.71%   63.75%   +0.03%     
==========================================
  Files         146      146              
  Lines       11958    12000      +42     
==========================================
+ Hits         7619     7650      +31     
- Misses       3875     3885      +10     
- Partials      464      465       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@josieang
Copy link
Collaborator Author

josieang commented May 7, 2024

Was I supposed to provide a token?

https://github.com/google/osv-scanner/actions/runs/8978083643/job/24657957118

Error: Codecov token not found. Please provide Codecov token with -t flag.
Error: Codecov: Failed to properly create commit: The process '/Users/runner/work/_actions/codecov/codecov-action/5ecb98a3c6b[74](https://github.com/google/osv-scanner/actions/runs/8978083643/job/24657957118#step:5:77)7ed38dc09f787459979aebb39be/dist/codecov' failed with exit code 1

@another-rex
Copy link
Collaborator

We seem to be getting these errors every now and then, current workaround is just rerun the tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants