Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make unit tests faster #1202

Merged
merged 1 commit into from May 11, 2024
Merged

Make unit tests faster #1202

merged 1 commit into from May 11, 2024

Conversation

jonjohnsonjr
Copy link
Contributor

I was flummoxed to find that go test ./... was both very slow and did not work. The very slowness comes from a couple sources. One is that we use a 1 QPS rate limiter in a lot of places, even when we are using an in-process http server to serve up testdata. I've removed those instances. In one particular python test, we are fetching a 10MB targz from the internet twice. I've gated that behind an e2e build tag, so to run that test you need to execute:

go test -tags e2e ./...

That's cumbersome, so I've added it to the make test target.

I've also added the e2e build tag to the go fips generated APK test because it requires you to first generate a binary before the test will pass. This is unacceptable to me, so the make test target will still test that, but go test ./... will not.

Before:

go test -count=1 ./...
?   	chainguard.dev/melange	[no test files]
?   	chainguard.dev/melange/docs	[no test files]
?   	chainguard.dev/melange/internal/contextreader	[no test files]
?   	chainguard.dev/melange/internal/gen-jsonschema	[no test files]
?   	chainguard.dev/melange/internal/logwriter	[no test files]
?   	chainguard.dev/melange/pkg/cli	[no test files]
?   	chainguard.dev/melange/pkg/container	[no test files]
?   	chainguard.dev/melange/pkg/container/dagger	[no test files]
?   	chainguard.dev/melange/pkg/container/docker	[no test files]
ok  	chainguard.dev/melange/pkg/build	0.352s
ok  	chainguard.dev/melange/pkg/cond	0.424s
?   	chainguard.dev/melange/pkg/convert/github	[no test files]
?   	chainguard.dev/melange/pkg/convert/relmon	[no test files]
?   	chainguard.dev/melange/pkg/http	[no test files]
ok  	chainguard.dev/melange/pkg/config	0.553s
?   	chainguard.dev/melange/pkg/linter/defaults	[no test files]
?   	chainguard.dev/melange/pkg/manifest	[no test files]
?   	chainguard.dev/melange/pkg/renovate	[no test files]
?   	chainguard.dev/melange/pkg/renovate/cache	[no test files]
ok  	chainguard.dev/melange/pkg/container/k8s	0.738s
ok  	chainguard.dev/melange/pkg/convert/apkbuild	0.224s
ok  	chainguard.dev/melange/pkg/convert/gem	5.139s
ok  	chainguard.dev/melange/pkg/convert/python	37.305s
ok  	chainguard.dev/melange/pkg/convert/wolfios	1.257s
ok  	chainguard.dev/melange/pkg/index	0.891s
ok  	chainguard.dev/melange/pkg/linter	1.309s
ok  	chainguard.dev/melange/pkg/renovate/bump	1.267s
ok  	chainguard.dev/melange/pkg/sbom	1.267s
ok  	chainguard.dev/melange/pkg/sca	1.414s
ok  	chainguard.dev/melange/pkg/sign	1.328s
ok  	chainguard.dev/melange/pkg/util	1.397s

After:

go test -count=1 ./...
?   	chainguard.dev/melange	[no test files]
?   	chainguard.dev/melange/docs	[no test files]
?   	chainguard.dev/melange/internal/contextreader	[no test files]
?   	chainguard.dev/melange/internal/gen-jsonschema	[no test files]
?   	chainguard.dev/melange/internal/logwriter	[no test files]
?   	chainguard.dev/melange/pkg/cli	[no test files]
?   	chainguard.dev/melange/pkg/container	[no test files]
?   	chainguard.dev/melange/pkg/container/dagger	[no test files]
?   	chainguard.dev/melange/pkg/container/docker	[no test files]
ok  	chainguard.dev/melange/pkg/build	0.352s
ok  	chainguard.dev/melange/pkg/cond	0.473s
ok  	chainguard.dev/melange/pkg/config	0.602s
?   	chainguard.dev/melange/pkg/convert/github	[no test files]
?   	chainguard.dev/melange/pkg/convert/relmon	[no test files]
?   	chainguard.dev/melange/pkg/http	[no test files]
?   	chainguard.dev/melange/pkg/linter/defaults	[no test files]
?   	chainguard.dev/melange/pkg/manifest	[no test files]
?   	chainguard.dev/melange/pkg/renovate	[no test files]
?   	chainguard.dev/melange/pkg/renovate/cache	[no test files]
ok  	chainguard.dev/melange/pkg/container/k8s	0.834s
ok  	chainguard.dev/melange/pkg/convert/apkbuild	0.183s
ok  	chainguard.dev/melange/pkg/convert/gem	0.336s
ok  	chainguard.dev/melange/pkg/convert/python	1.593s
ok  	chainguard.dev/melange/pkg/convert/wolfios	1.083s
ok  	chainguard.dev/melange/pkg/index	0.989s
ok  	chainguard.dev/melange/pkg/linter	1.379s
ok  	chainguard.dev/melange/pkg/renovate/bump	1.197s
ok  	chainguard.dev/melange/pkg/sbom	1.292s
ok  	chainguard.dev/melange/pkg/sca	1.404s
ok  	chainguard.dev/melange/pkg/sign	1.369s
ok  	chainguard.dev/melange/pkg/util	1.440s

I was flummoxed to find that `go test ./...` was both very slow and did
not work. The very slowness comes from a couple sources. One is that we
use a 1 QPS rate limiter in a lot of places, even when we are using an
in-process http server to serve up testdata. I've removed those
instances. In one particular python test, we are fetching a 10MB targz
from the internet twice. I've gated that behind an `e2e` build tag, so
to run that test you need to execute:

  go test -tags e2e ./...

That's cumbersome, so I've added it to the `make test` target.

I've also added the `e2e` build tag to the go fips generated APK test
because it requires you to first generate a binary before the test will
pass. This is unacceptable to me, so the `make test` target will still
test that, but `go test ./...` will not.

Signed-off-by: Jon Johnson <jon.johnson@chainguard.dev>
@jonjohnsonjr jonjohnsonjr merged commit 2fbc085 into chainguard-dev:main May 11, 2024
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants