Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse javascript dependency trees #2142

Open
wants to merge 22 commits into
base: main
Choose a base branch
from

Conversation

noqcks
Copy link
Contributor

@noqcks noqcks commented Sep 16, 2023

implements the same functionality desired by #2020

This PR implements a new cataloger called javascript-cataloger that collects full dependency trees and packages with for javascript ecosystem pkg managers -- [pnpm, yarn, npm]

This allows one to see the relationships between packages such as with the dependencies field in CycloneDX https://cyclonedx.org/docs/1.5/json/#dependencies.

Note: all parsed package.json or lock files will now have a root package name and version. if name and version exist in the parsed file, then we use that, if no name or version exists, we use the dir name. This allows us to create a dep tree with a top level node.

EDIT: Please hold off on reviewing this. I have some major updates to push. Ready for review.

Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
@noqcks noqcks changed the title Benji/javascript deps Parse javascript deps tree Sep 16, 2023
@noqcks noqcks changed the title Parse javascript deps tree Parse javascript dependency trees Sep 16, 2023
@noqcks noqcks marked this pull request as ready for review September 16, 2023 04:01
@@ -13,6 +13,8 @@ func TestNpmPackageLockDirectory(t *testing.T) {
sbom, _ := catalogDirectory(t, "test-fixtures/npm-lock")

foundPackages := internal.NewStringSet()
// root pkg
foundPackages.Add("npm-lock")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: new top level root pkg being added

@@ -144,6 +144,8 @@ var dirOnlyTestCases = []testCase{
pkgType: pkg.NpmPkg,
pkgLanguage: pkg.JavaScript,
pkgInfo: map[string]string{
"yarn": "0.0.0",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: new top level root pkg being added

Copy link

@nikpivkin nikpivkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@noqcks Do you mind if I leave a couple comments?

syft/pkg/cataloger/javascript/filter/filter.go Outdated Show resolved Hide resolved
syft/pkg/cataloger/javascript/parser/yarn/parse.go Outdated Show resolved Hide resolved
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
@wagoodman wagoodman self-assigned this Oct 11, 2023
@noqcks
Copy link
Contributor Author

noqcks commented Oct 11, 2023

@wagoodman I actually had some things to cleanup in this PR. I saw you just self-assigned. Mind if I push some commits over the next hour or so and then let you know when it's ready?

@wagoodman
Copy link
Contributor

have at it! (and just to be transparent we assign internally to track who's taking point on reviewing on the community github project)

@noqcks
Copy link
Contributor Author

noqcks commented Oct 11, 2023

@wagoodman ready now

Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Signed-off-by: Benji Visser <benji@093b.org>
Copy link
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've got a rebased version of this branch locally that I'm reviewing -- I can push up my changes (might not be today though)

// NewJavaScriptCataloger returns a new JavaScript cataloger object based on detection
// of npm based packages and lock files to provide a complete dependency graph of the
// packages.
func NewJavaScriptCataloger() *generic.GroupedCataloger {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would need to be reconciled with the NewLockCataloger() above. At surface glance this cataloger would replace the lock cataloger.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, that's true, was looking for guidance here on how to deprecate an existing cataloger on favor of a new cataloger and what the process would be for that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wagoodman any thoughts here? thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just picked this back up for re-review today. I'm going to try and help get the PR rebased and working again, so I'll be pushing some commits. I'll get back to this question while I'm trying to square it with the current state on main.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Review
Development

Successfully merging this pull request may close these issues.

None yet

3 participants