feat: Enable granular regression tests

[laurentsimon]

Per discussion in #707, we'd like to be able to verify certain things end-to-end and need a way to ignore signature verification.

@trishankatdatadog @ianlewis

[trishankatdatadog]

need a way to ignore signature verification.

Or at least use test keys :)

[trishankatdatadog]

The more I think about it, I'm increasingly of the opinion that we should never forgo signature verification, even while testing. Better to err on the side of caution and make sure we are always verifying signatures correctly.

[laurentsimon]

Indeed, that's one reason I have not built this in the first place. Some things we do is set special env variables and check the identity of the platform it runs on to turn on things like this. Like "CI": "true" and "repository"="slsa-framework/slsa-verifier" and "SLSA_VERIFIER_TEST"="1". Eg see.

slsa-verifier/verifiers/internal/gha/builder.go

Lines 175 to 181 in f6ae402

	if (id.SourceRepository == trustedBuilderRepository ||
	id.SourceRepository == e2eTestRepository) &&
	options.TestingEnabled() {
	// Allow verification on the main branch to support e2e tests.
	if ref == "refs/heads/main" {
	return nil
	}

for our end-to-end tests

[ianlewis]

Maybe I'm misunderstanding but for regression tests we just need to make sure the thing that used to fail doesn't fail anymore and run the test in an ongoing way to make sure we don't regress. So it should just be a matter of modifying the digest in the publish attestation JSON and check to make sure the signature validation fails no?

For the regression test the actual digest check shouldn't really come into play (as it's hidden behind the signature check that will fail), and we shouldn't necessarily need to touch the signature I don't think. We can capture tests for behavior here in unit-tests of more granular functions directly with mocks etc.

[trishankatdatadog]

So it should just be a matter of modifying the digest in the publish attestation JSON and check to make sure the signature validation fails no?

Sorry, maybe I didn't sufficiently clarify what I meant: suppose a publish attestation was correctly signed, but for some reason (e.g., a bug or, worse, a key compromise) the subject digest in it did not match the provenance attestation. We should test whether we can catch this, no?