Support yaml as an output format #1395

craigbox · 2023-09-19T01:33:06Z

craigbox@mac:~/Documents/Projects/kubescape.io$ kubescape scan image nginx -f yaml
✅  Successfully scanned image: nginx
❌  format "yaml"is not supported for image scanning

expected: output manifest in YAML format, same as VulnerabilityManifest in the cluster
actual: error message

craigbox@mac:~/Documents/Projects/kubescape.io$ kubescape scan framework nsa -f yaml
 ❗ Invalid format "yaml", default format "pretty-printer" is applied
✅  Initialized scanner
✅  Loaded policies
✅  Loaded exceptions
✅  Loaded account configurations
✅  Accessed Kubernetes objects
✅  Collected RBAC resources
Control: C-0070 100% |███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| (24/24, 81 it/s)        
✅  Done scanning. Cluster: docker-desktop
✅  Done aggregating results

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Controls: 24 (Failed: 12, Passed: 10, Action Required: 2)
Failed Resources by Severity: Critical — 0, High — 4, Medium — 15, Low — 3

┌──────────┬─────────────────────────────────────────────┬──────────────────┬───────────────┬────────────────────┐
│ SEVERITY │                CONTROL NAME                 │ FAILED RESOURCES │ ALL RESOURCES │ % COMPLIANCE-SCORE │
├──────────┼─────────────────────────────────────────────┼──────────────────┼───────────────┼────────────────────┤
│ Critical │ Disable anonymous access to Kubelet service │        0         │       0       │ Action Required *  │
│ Critical │ Enforce Kubelet client TLS authentication   │        0         │       0       │ Action Required *  │
│ High     │ Resource limits                             │        3         │      20       │        85%         │
│ High     │ Host PID/IPC privileges                     │        1         │      20       │        95%         │
│ Medium   │ Non-root containers                         │        3         │      20       │        85%         │
│ Medium   │ Allow privilege escalation                  │        2         │      20       │        90%         │
│ Medium   │ Ingress and Egress blocked                  │        3         │      20       │        85%         │
│ Medium   │ Automatic mapping of service account        │        2         │      67       │        97%         │
│ Medium   │ Cluster internal networking                 │        1         │       5       │        80%         │
│ Medium   │ Linux hardening                             │        2         │      20       │        90%         │
│ Medium   │ Secret/ETCD encryption enabled              │        1         │       1       │         0%         │
│ Medium   │ Audit logs enabled                          │        1         │       1       │         0%         │
│ Low      │ Immutable container filesystem              │        2         │      20       │        90%         │
│ Low      │ PSP enabled                                 │        1         │       1       │         0%         │
├──────────┼─────────────────────────────────────────────┼──────────────────┼───────────────┼────────────────────┤
│          │              RESOURCE SUMMARY               │        9         │      175      │       74.88%       │
└──────────┴─────────────────────────────────────────────┴──────────────────┴───────────────┴────────────────────┘
FRAMEWORK NSA

* This control requires the host-scanner capability. To activate the host scanner capability, proceed with the installation of the kubescape operator chart found here: https://github.com/kubescape/helm-charts/tree/main/charts/kubescape-cloud-operator

✅  Scan results saved. filename: report.txt

expected: YAML output
actual: pretty-printed output

The text was updated successfully, but these errors were encountered:

dwertent · 2023-09-19T05:15:14Z

I'm not sure why you want an output like the CRDs we have in the cluster.
Also, this will not work with configuration scanning, so there is no consistency here anyway.

craigbox · 2023-09-19T07:58:42Z

Also, this will not work with configuration scanning, so there is no consistency here anyway.

Why not? Why couldn't we get output in the same format as the ConfigurationScanSummary object?

prady0t · 2023-09-27T04:24:59Z

Can I take up this issue?

craigbox · 2023-09-27T08:29:49Z

With all good first issue labels, our suggestion is that you look at the code, and propose a design/implementation in a Google doc - no more than one page - for the maintainers to review.

VaibhavMalik4187 · 2023-09-29T20:16:12Z

I wonder if using the JSON printer to generate the report and then converting it to the YAML format would do the trick. I managed to make it work locally. Attaching the YAML report for reference.

YAML presenter is not present at https://github.com/anchore/grype/tree/main/grype/presenter. Hence, I thought that converting JSON to YAML could be a potential solution.

This commit adds the support for using yaml as an output format to store the reports. For Example: kubescape scan image nginx -f yaml The report will be stored in the "report.yaml" file. Fixes: kubescape#1395 Signed-off-by: VaibhavMalik4187 <vaibhavmalik2018@gmail.com>

dwertent · 2023-10-01T09:25:10Z

The YAML output should be aligned with the CRDs created by the kubpescape-operator.
Here is an example of image scanning:

% kubectl -n kubescape get vulnerabilitymanifests quay.io-kubescape-gateway-v0.1.15-aa5f58 -o yaml
apiVersion: spdx.softwarecomposition.kubescape.io/v1beta1
kind: VulnerabilityManifest
metadata:
  annotations:
    kubescape.io/image-id: quay.io/kubescape/gateway@sha256:21160c0767f8ab425184d7d63bcde7b10982fb9c4481efdc3c81bf4417aa5f58
    kubescape.io/status: ""
  creationTimestamp: "2023-09-30T12:39:24Z"
  labels:
    kubescape.io/context: non-filtered
    kubescape.io/image-id: quay-io-kubescape-gateway-sha256-21160c0767f8ab425184d7d63bcde7
    kubescape.io/image-name: quay-io-kubescape-gateway
  name: quay.io-kubescape-gateway-v0.1.15-aa5f58
  namespace: kubescape
  resourceVersion: "1"
  uid: 7cd79549-b6c7-47ef-a59a-eb43948f80f2
spec:
  metadata:
    report:
      createdAt: null
    tool:
      databaseVersion: sha256:dbd1660b90a8ac16cd629c17ffd139e1cdfa9a29ed082de94c64779ed3c7f625
      name: ""
      version: v0.61.0
    withRelevancy: false
  payload:
    descriptor:
      configuration: null
      db:
        built: "2023-09-29T12:45:46Z"
        checksum: sha256:dbd1660b90a8ac16cd629c17ffd139e1cdfa9a29ed082de94c64779ed3c7f625
        error: null
        location: /home/nonroot/.cache/grype/db/5
        schemaVersion: 5
      name: grype
      version: '[not provided]'
    distro:
      idLike:
      - debian
      name: debian
      version: "11"
    matches: null
    source:
      target:
        architecture: ""
        config: null
        imageID: ""
        imageSize: 0
        layers: null
        manifest: null
        manifestDigest: ""
        mediaType: ""
        os: ""
        repoDigests: []
        tags: []
        userInput: ""
      type: image
status: {}

We are still working on the configuration scanning output.

VaibhavMalik4187 · 2023-10-01T10:53:45Z

The YAML output should be aligned with the CRDs created by the kubpescape-operator. Here is an example of image scanning:

% kubectl -n kubescape get vulnerabilitymanifests quay.io-kubescape-gateway-v0.1.15-aa5f58 -o yaml
apiVersion: spdx.softwarecomposition.kubescape.io/v1beta1
kind: VulnerabilityManifest
metadata:
  annotations:
    kubescape.io/image-id: quay.io/kubescape/gateway@sha256:21160c0767f8ab425184d7d63bcde7b10982fb9c4481efdc3c81bf4417aa5f58
    kubescape.io/status: ""
  creationTimestamp: "2023-09-30T12:39:24Z"
  labels:
    kubescape.io/context: non-filtered
    kubescape.io/image-id: quay-io-kubescape-gateway-sha256-21160c0767f8ab425184d7d63bcde7
    kubescape.io/image-name: quay-io-kubescape-gateway
  name: quay.io-kubescape-gateway-v0.1.15-aa5f58
  namespace: kubescape
  resourceVersion: "1"
  uid: 7cd79549-b6c7-47ef-a59a-eb43948f80f2
spec:
  metadata:
    report:
      createdAt: null
    tool:
      databaseVersion: sha256:dbd1660b90a8ac16cd629c17ffd139e1cdfa9a29ed082de94c64779ed3c7f625
      name: ""
      version: v0.61.0
    withRelevancy: false
  payload:
    descriptor:
      configuration: null
      db:
        built: "2023-09-29T12:45:46Z"
        checksum: sha256:dbd1660b90a8ac16cd629c17ffd139e1cdfa9a29ed082de94c64779ed3c7f625
        error: null
        location: /home/nonroot/.cache/grype/db/5
        schemaVersion: 5
      name: grype
      version: '[not provided]'
    distro:
      idLike:
      - debian
      name: debian
      version: "11"
    matches: null
    source:
      target:
        architecture: ""
        config: null
        imageID: ""
        imageSize: 0
        layers: null
        manifest: null
        manifestDigest: ""
        mediaType: ""
        os: ""
        repoDigests: []
        tags: []
        userInput: ""
      type: image
status: {}

We are still working on the configuration scanning output.

Thanks for the additional information @dwertent. I noticed that the command you shared fetches the vulnerability manifests of a single resource out of many resources in the cluster.

The report.yaml file contains the information of all the resources under the resources key. So, to align with the kubescape-operator, do I need to eliminate all the other information except the resources?

Attaching a screenshot for better clarification.

I'm running a local kind cluster. I don't have similar CRDs, any information on how to set up the Kubscape namespace and similar resources for testing would be very helpful. Thanks.

This commit adds the support for using yaml as an output format to store the reports. For Example: kubescape scan image nginx -f yaml The report will be stored in the "report.yaml" file. Fixes: kubescape#1395 Signed-off-by: VaibhavMalik4187 <vaibhavmalik2018@gmail.com>

vladklokun · 2023-10-03T14:19:39Z

@craigbox
Why do we need this? What would be the use case?

My concern is that we would be adding more code for us to support when our JSON/YAML is not supposed to be human-readable, we don’t use the YAMLs for interchange between components and I don’t think we intend to. So what value would we give if this feature were implemented?

craigbox · 2023-10-03T23:12:42Z

I think it's worth being consistent. If the intention is all interchange happen in JSON, then we should remove support for YAML in scan output results.

VaibhavMalik4187 · 2023-11-21T07:03:17Z

@matthyx any updates on this issue?

matthyx · 2023-11-21T07:05:54Z

@matthyx any updates on this issue?

not yet, don't waste time on that for the moment

VaibhavMalik4187 · 2023-11-21T07:49:13Z

@matthyx any updates on this issue?

not yet, don't waste time on that for the moment

Okay, understood.

craigbox added feature New feature or request good first issue Good for newcomers labels Sep 19, 2023

VaibhavMalik4187 mentioned this issue Sep 30, 2023

Added support for YAML as an output format #1401

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support yaml as an output format #1395

Support yaml as an output format #1395

craigbox commented Sep 19, 2023

dwertent commented Sep 19, 2023

craigbox commented Sep 19, 2023

prady0t commented Sep 27, 2023

craigbox commented Sep 27, 2023

VaibhavMalik4187 commented Sep 29, 2023 •

edited

dwertent commented Oct 1, 2023

VaibhavMalik4187 commented Oct 1, 2023 •

edited

vladklokun commented Oct 3, 2023

craigbox commented Oct 3, 2023

VaibhavMalik4187 commented Nov 21, 2023

matthyx commented Nov 21, 2023

VaibhavMalik4187 commented Nov 21, 2023

Support yaml as an output format #1395

Support yaml as an output format #1395

Comments

craigbox commented Sep 19, 2023

dwertent commented Sep 19, 2023

craigbox commented Sep 19, 2023

prady0t commented Sep 27, 2023

craigbox commented Sep 27, 2023

VaibhavMalik4187 commented Sep 29, 2023 • edited

dwertent commented Oct 1, 2023

VaibhavMalik4187 commented Oct 1, 2023 • edited

vladklokun commented Oct 3, 2023

craigbox commented Oct 3, 2023

VaibhavMalik4187 commented Nov 21, 2023

matthyx commented Nov 21, 2023

VaibhavMalik4187 commented Nov 21, 2023

VaibhavMalik4187 commented Sep 29, 2023 •

edited

VaibhavMalik4187 commented Oct 1, 2023 •

edited