Measuring Security Risks - Scorecard

[avelino]

looking at the scorecard-trusted process history it takes an average of ⚠️ 180m, it doesn't make sense to have an action running on every pull request (this comment), ideally run as a periodic task.

With so much software today relying on open-source projects, consumers need an easy way to judge whether their dependencies are safe. Scorecards helps reduce the toil and manual effort required to continually evaluate changing packages when maintaining a project’s supply chain. Consumers can automatically assess the risks that dependencies introduce and use this data to make informed decisions about accepting these risks, evaluating alternative solutions, or working with the maintainers to make improvements.
continue reading

ref: https://github.com/ossf/scorecard/blob/main/.github/workflows/integration.yml

example:

# Copyright 2021 Security Scorecard Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Run secret-dependent integration tests
on:
  pull_request:
  repository_dispatch:

name: scorecard tests
env:
  IMAGE_NAME: scorecard
  OUTPUT_PATH: output
  REPO: https://github.com/prest/prest
jobs:
  scorecard-trusted:
    runs-on: ubuntu-latest
    if: (github.event_name == 'repository_dispatch') || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)
    steps:
      - id: debug-outputs
        run: |
          echo ${{ github.event_name }}
          echo ${{ github.event.client_payload }}

      - id: verify-sha-input
        if: github.event_name == 'repository_dispatch'
        run: |
          echo \"${{ github.event.client_payload.pull_request.head.sha }}\"
          echo \"${{ github.event.client_payload.slash_command.args.named.sha }}\"
          SHAINPUT=$(echo ${{github.event.client_payload.slash_command.args.named.sha}} | cut -c1-7)
          if [ ${#SHAINPUT} -le 6 ]; then echo "error::input sha not at least 7 characters long" ; exit 1
          else echo "done"
          fi
          SHAHEAD=$(echo ${{github.event.client_payload.pull_request.head.sha}} | cut -c1-7)
          echo ${#SHAINPUT}
          echo ${#SHAHEAD}
          if [ "${SHAHEAD}" != "${SHAINPUT}" ]; then echo "sha input from slash command does not equal the head sha" ; exit 1
          else echo "shas are equal"
          fi

      - name: pull_request actions/checkout
        if: github.event_name == 'pull_request'
        uses: actions/checkout@5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f # v2.3.4

      - name: pull_request actions/checkout
        if: github.event_name == 'repository_dispatch'
        uses: actions/checkout@5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f # v2.3.4
        with:
          ref: 'refs/pull/${{ github.event.client_payload.pull_request.number }}/merge'

      - name: setup-go
        uses: actions/setup-go@37335c7bb261b353407cff977110895fa0b4f7d8 # v2.1.3
        with:
          go-version: '1.16'

      - name: Prepare test env
        run: |
            go mod download
            go get github.com/ossf/scorecard
            go install github.com/ossf/scorecard
            mkdir $OUTPUT_PATH

      - name: Test Blob cache
        env:
          GITHUB_AUTH_TOKEN: ${{ secrets.GH_AUTH_TOKEN }}
          USE_BLOB_CACHE: true
          BLOB_URL: ${{secrets.BUCKET_URL}}
        run: |
            scorecard --repo=$REPO --show-details --metadata=openssf  --format json > ./$OUTPUT_PATH/results.json

      - name: find comment
        if: ${{ always() }}
        uses: peter-evans/find-comment@309ce798ba1c3627e3809dd68c694a29ef048fe2 # v1.2.0
        id: fc
        with:
          issue-number: ${{ github.event.pull_request.number || github.event.client_payload.pull_request.number }}
          comment-author: 'github-actions[bot]'
          body-includes: Integration tests ran for

      - name: create or update comment
        if: (${{ always() }})
        uses: peter-evans/create-or-update-comment@a35cf36e5301d70b76f316e867e7788a55a31dae # v1.4.5
        with:
          issue-number: ${{ github.event.pull_request.number || github.event.client_payload.pull_request.number }}
          comment-id: ${{ steps.fc.outputs.comment-id }}
          body: |
            Integration tests ${{ job.status }} for [${{ github.event.client_payload.slash_command.args.named.sha || github.event.pull_request.head.sha }}](/prest/prest/actions/runs/${{ github.run_id }})

      - name: set fork job status
        uses: actions/github-script@a3e7071a34d7e1f219a8a4de9a5e0a34d1ee1293 # v4.0.2
        if: ${{ always() }}
        id: update-check-run
        env:
          number: ${{ github.event.client_payload.pull_request.number }}
          job: ${{ github.job }}
          # Conveniently, job.status maps to https://developer.github.com/v3/checks/runs/#update-a-check-run
          conclusion: ${{ job.status }}
          sha: ${{ github.event.client_payload.slash_command.args.named.sha }}
          event_name: ${{ github.event_name }}
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            if (process.env.event_name !== 'repository_dispatch') {
              console.log("Not repository_dispatch... nothing to do!");
              return process.env.event_name;
            }

            const ref = process.env.sha;

            const { data: checks } = await github.checks.listForRef({
              ...context.repo,
              ref
            });

            const check = checks.check_runs.filter(c => c.name === process.env.job);
            console.log(check);

            const { data: result } = await github.checks.update({
              ...context.repo,
              check_run_id: check[0].id,
              status: 'completed',
              conclusion: process.env.conclusion
            });

            return result;