Clarifications for using tracks sparingly

[arewm]

In https://slsa.dev/spec/v1.1/principles, for the guidelines to use tracks sparingly, we state

Use tracks sparingly. Additional tracks add extra complexity to SLSA, so a new track should be seen as a last resort. Each track should have a clear, distinct purpose with a crisply defined objective, such as trustworthy provenance for the Build track. As a rule of thumb, a new track may be warranted if it addresses threats unrelated to another track. Try to avoid tracks that sound confusingly similar in either name or objective.

I have two lines of questioning

Tracks and attestations

We state

Each track should have a clear, distinct purpose with a crisply defined objective, such as trustworthy provenance for the Build track

This example with provenance and a build track creates a mapping in my mind that a track should have an associated attestation. This has been discussed generically in the specification calls in the past, but we have never come out and stated this as a pattern/anti-pattern. Should each track be associated with some specific type of attestation? If so, would the suitability of a specific attestation for informing the facts of some supply chain guarantees be a guideline as to whether requirements should be combined into an existing track instead of creating an additional track?

Threat mitigation redundancy

I read

a new track may be warranted if it addresses threats unrelated to another track

to imply that if a proposed new track addresses threats related to another track that it is likely not warranted. As supply chain security requires a defense-in-depth approach, I feel like this would be an anti-pattern. If we can identify multiple ways to protect against the same threat (and each of these ways can be implemented/tracked in their own rite), then we would have a stronger footing to stand on.

[MarkLodato]

(This is a really good question. It took me a long time to think through the answer, and I'm still not 100% confident.)

Should each track be associated with some specific type of attestation?

I think the answer is "not necessarily," but it probably depends on how you interpret "be associated with" and "some specific type of attestation". 😁

As a reminder, the higher level principle is that each level/track focuses on a concrete security outcome.

The Build track is a bit unique because having trustworthy build metadata is the desired outcome. Here an "attestation" is needed because (1) this build metadata is not usually persisted in any other way and (2) we want to pass the attestation around so that downstream systems can make fast, reliable, and secure decisions based on that metadata.

I don't think an attestation is a defining characteristic of a track, but it might be the common way it's implemented in practice.

Let's consider the future Source track as an example. I can see three ways of implementing that have varying degrees of "attestations":

• Suppose you have a single monorepo in your company, and it is configured to enforce all of the Source track requirements: attribution, retention, two-party review, etc. Knowing the URI is sufficient to determine the Source level. In this case, you don't need a Source attestation at all. In essence, because there's only one monorepo, you can hardcode the rule into the verifier.

• Suppose you have a git host that enables Source track requirements on a per-repo basis, and it has a query API to determine what SLSA Source level a particular repository is at. You could have your Build platform query this API and stick the enum in the Provenance as an annotation. This isn't really a separate "attestation".

• Same as above, but instead the API returns a true attestation signed by the Source platform. Now you do have a separate attestation for the repo.

Does that help?

I read

a new track may be warranted if it addresses threats unrelated to another track

to imply that if a proposed new track addresses threats unrelated to another track that it is likely not warranted.

Is there a typo there? That doesn't make sense.

[arewm]

Thanks for the examples relating to the future Source track. Tying to an in-toto attestation is too narrowly scoped. There is no requirement to have a specific data format for each track nor is there even a requirement to be able to inform the level achieved within a track by just referencing the "supporting data" (i.e. the provenance has no indication as to whether the dependencies are complete outside of what is inherent by the builder.id).

Instead, tracks and levels within the tracks should be scoped such that sufficient evidence of conformance can be associated with a given subject. This feels very "hand wavy" but the key part is that there are measurable requirements which need to be met in order to achieve a specific level. Trust will be placed upon some entity, whether it is the entity producing the evidence or some entity which has the ability to verify the evidence meets some bar (my mind draws a connection to #849 here).

In the examples above, verification of evidence might be performed by:

• The company managing the monorepo, enforcing the requirement before the monorepo is updated. Unless there are different requirements imposed upon the context where content is updated, there would likely just be a single level achieved by all content
• The git host upon API request, assessing the commit against the track's measurable requirements in order to return the enum
• Depending on the content of the attestation, this might just contain the data required for an external verifier to make a level determination, it could have a VSA verifying the level, or it might have a combination of these

In approaches 1 or 2, a VSA can be created based on the knowledge of the systems/data returned from the API to convert this output to the source and/or the artifacts produced from the source.

---

I read

a new track may be warranted if it addresses threats unrelated to another track

to imply that if a proposed new track addresses threats unrelated to another track that it is likely not warranted.

Is there a typo there? That doesn't make sense.

I was trying to tie into part of a comment thread by @kpk47 in the reproducible builds gDoc, copied here in case the comment gets blown away:

How do these levels correspond to SLSA's threat model? AFAICT, the Levels 1 and 2 address Threat D (Use compromised dependency), Level 3 addresses Threat E (Compromise build process), and Level 4 addresses Threat F (Upload modified package). I don't like the idea of having a track that addresses threats already covered by other tracks.

If we phrase the need for a new track just based on the fact that we are addressing new threats, this makes me as a reader interpret the statement to say "if you are trying to protect against a threat covered in another track, just add to that track." This wording almost implies that addressing a single threat across multiple tracks is an anti-pattern. I think that we should be clear that this isn't an anti-pattern as a redundant/defense-in-depth approach would add extra protection against specific threats.

Additionally, it is possible for modifications to be made to a current track to address additional threats than have been considered currently. I am not sure how to update the new-track heuristic might look to be more helpful (i.e. combining new threats, new methods, additional parties) or whether it should just be removed.

[MarkLodato]

Tracks and attestations

Hmm, followed your last comment but couldn't figure out what the ultimate conclusion was. 😄 Could you clarify what you are suggesting we change?

Note that we also have Prefer attestations over inferences, which perhaps could be updated?

This wording almost implies that addressing a single threat across multiple tracks is an anti-pattern. I think that we should be clear that this isn't an anti-pattern as a redundant/defense-in-depth approach would add extra protection against specific threats.

In my mind, a track is framed around a particular threat (not a particular solution), and the level describes the degree to which that threat is addressed. If a threat is addressed by multiple defenses, you could represent that in the level. (That said, it might be the case that each track has a "main" threat, and other threats are covered incidentally. In that case, it might make sense to overlap.)

Perhaps we should talk about a specific example to make this more concrete? I feel like that would help us come to a better result. Do you have an example in mind?

[arewm]

The main point that I was trying to drive is:

Instead, tracks and levels within the tracks should be scoped such that sufficient evidence of conformance can be associated with a given subject. This feels very "hand wavy" but the key part is that there are measurable requirements which need to be met in order to achieve a specific level.

If a requirement is measurable, then it should be possible to easily convert that result into an in-toto attestation like a VSA if there is not a specific attestation associated with the level. An update to Prefer attestations over inferences might be reasonable enough as the current wording may be too narrowly scoped to the build track. Levels need to be identified such that the specific improvement can be easily measured.

While some tracks may have associated attestations (i.e. build/provenance), others should be created to prefer measurements so that attestations (i.e. VSAs) can be generated if needed to represent attainment of the different levels.

---

Rereading one of your original questions...

I read

a new track may be warranted if it addresses threats unrelated to another track

to imply that if a proposed new track addresses threats unrelated to another track that it is likely not warranted.

Is there a typo there? That doesn't make sense.

Yes, there was a typo that I noticed upon rereading (now corrected in the top post):

to imply that if a proposed new track addresses threats unrelated to another track that it is likely not warranted.

One example that comes to mind here is using compromised dependencies (Dependency threat D).

Many (most?) package managers provide a way to specify checksums when defining dependencies. This behavior therefore enables a threat mitigation by pinning dependencies in the source code. This requirement could be included in the source track as it is a check that can be enforced during the code review process (either manually or using automation). So while it isn't directly related to the integrity of the source, it can inform a consumer's trust when leveraging the source.

The accuracy of provenance is also not necessarily affected by whether the dependencies are pinned in source (the provenance can still be accurate based on the dependencies as resolved), including the dependency pinning requirement in the build track doesn't follow logically. As described in the Threats & mitigations, however, this dependency threat can be mitigated by the build track upon verification of the SLSA provenance.

While it is generally accepted that there is a best practice to pin dependencies due to the verification in package managers, where does it make most sense to include this requirement? Should it be included in one or more of the source and build tracks or should it be included in a separate track since it does not cleanly fit into the clear and crisp purpose of other tracks?