proposal: cmd/go: add flag `go test -no-cache`

#24573 is closed, but it has all the information. -count=1 to most people does not mean -no-cache. It's just not intuitive.

This is a request to implement -no-cache flag on go test so that this kind of behavior isn't hidden from users.

Alternatively (backwards incompatible), since caching of test results is a bit surprising, instead allow uses to decide to cache or not explicitly with -cache flag, and set the default to not cache.

[ianlancetaylor]

CC @bcmills @jayconrod

[ghostsquad]

for clarity, count=1 has the side effect of disabling caching. But the documentation specifies:

    -count n
        Run each test and benchmark n times (default 1).
        If -cpu is set, run n times for each GOMAXPROCS value.
        Examples are always run once.

so this is not a request to have -no-cache be equivalent to -count=1, but rather can be combined with -count flag if desired.

Not sure if the title should be updated or not.

[ianlancetaylor]

A minor point: we don't currently have any -no-XXX flags, so perhaps -cache=false would be a better fit with existing options.

[bcmills]

I do not think we should switch test caching from opt-out to opt-in. The vast majority of tests should be hermetic and cacheable.

It might make sense to have a -cache flag, but in that case -cache=true should force caching on even if other flags would ordinary render the result uncacheable.

Even so, I'm not sure that the added clarity of an explicit flag would be worth the added cost of having yet another flag for users to consider.

CC @jayconrod @matloob

[myitcv]

Just noting the documentation on this particular point, from go help testflag:

When 'go test' runs in package list mode, 'go test' caches successful
package test results to avoid unnecessary repeated running of tests. To
disable test caching, use any test flag or argument other than the
cacheable flags. The idiomatic way to disable test caching explicitly
is to use -count=1.

[ghostsquad]

If it's decided that caching by default is actually not desirable (what I prefer), that would be great. The community can go about their business and be confident that every time they run go test with any sort of flags, it is actually testing things.

If it's decided that caching by default is a good idea, then it would additionally be nice if how go decided to cache was fixed up a bit more

as an example:

go test ./dir/... uses a cache but go test ./dir/... - bypasses the cache, but does not invalidate the cache. So running (assuming nothing has been previously cached)

# cached (if available)
go test ./dir/...
# not cached
go test ./dir/... -
# cached
go test ./dir/...

[ghostsquad]

Linking this issue to other things I've found in the community regarding cleaning cache and dealing with it

https://stackoverflow.com/questions/48882691/force-retesting-or-disable-test-caching/48882892