Discussion: Code and security position scanning tools for Adoptium repos

[tellison]

Interested to hear people’s opinions about the code scanning and policy checking tools we should be using on our repos. These cover static analysis security testing (SAST) and supply chain management tools.

There are lots of tool options to choose from!

We have a few, but not all, of our repos already set-up with CodeQL (GitHub’s SAST scanner) (e.g. adoptium/adoptium.net). More of our repos, but by no means all, are set-up with dependabot (e.g. adoptium/blog.adoptium.net) to keep our dependencies up to date and secure our supply chain.

I've also been looking at the OSSF Scorecard tooling that provides a number of security-related recommendations (e.g. adoptium/adoptium.net). The OSSF scorecard covers a variety of OSSF's best practices for code/dependencies/policies.

Alongside these, Eclipse are encouraging the use of the Step-Security Secure Workflows tool, though I haven't yet seen how to integrate that into actions. I haven't searched very hard though. I'm running that on our repos as part of #187, but see that more of a remediation tool than a constant scanner at the moment.

I also see broad use of Renovate, which is another dependency updater, akin to dependabot.

So... I propose

• rather than follow an ad hoc policy of utilising the available tools and reports on a per repository basis, we agree on a set that we shall use across the project and configure our repository and tools accordingly,
• we set-up common configurations where possible and reuse them in our repos, like we do for a number of our other workflows to simplify management of the tools and configs.

Discuss.

[karianna]

I think this landscape is going to continue to evolve. Today I would like us to use the minimum of CodeQL and Dependabot as they are free as in beer and have a large install and user base and are well understood.

MSFT is a signed up member of OpenSSF and although it is a new body it does seem to have the multi-vendor buy-in which means it has a good chance of becoming that defacto standard going forward, so I think trying out the scorecard is prudent.

I'm less sold on Step-Security Secure Workflows tool. However, it looks like it can integrate with the OpenSSF Scorecard which helps a little to dedup tooling.

Have not used Renovate so can't comment on it.

[tellison]

Thanks for the comment @karianna. I agree with that.

The Secure Workflow tool is handy to figure out/check the action hashes and permissions to commit (though now I've done a few of them, it's quite easy to do manually if required), and don't plan to use the workflow hardening action in our workflows, just because I have no insight into the daemon that would be installed on our runners.

In addition to the scorecard, OSSF also have the allstar project that allows enforcement of org-level policies. It seems to overlap/complement the scorecard - and would require we create configuration rules to ensure we don't get spammed about the repos we have that are mirrors/binary releases/etc as they will likely cause many warnings with default settings.