Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the tests, like, a lot. #264

Open
Datseris opened this issue Jul 25, 2022 · 0 comments
Open

Improve the tests, like, a lot. #264

Datseris opened this issue Jul 25, 2022 · 0 comments
Labels
code-clarity Will increase code clarity of the source. good first issue help wanted testing

Comments

@Datseris
Copy link
Member

Datseris commented Jul 25, 2022

One of the things the Good Scientific Code Workshop teaches is writing good unit tests. I have to admit, ChaosTools.jl suffers a lot from "bad tests", partly because it is the oldest thing in the organization, partly because it is a package really hard to test due to the vast amount of content and the fact that all content is atomic. Nevertheless, these aren't real excuses, and little by little we should try to improve the test suite. I am pasting here the slide with "good advice on writing tests":

  • Actually unit: test atomic, self-contained functions. Each test must test only one thing, the unit. When a test fails, it should pinpoint the location of the problem. Testing entire processing pipelines (a.k.a. integration tests) should be done only after units are covered, and only if resources/time allow for it!
  • Known output / Deterministic: tests defined through minimal examples that their result is known analytically are the best tests you can have! If random number generation is necessary, either test valid output range, or use seed for RNG
  • Robust: Test that the expected outcome is met, not the implementation details. Test that the target functionality is met without utilizing knowledge about the internals. Also, never use internal functions in the test suite.
  • High coverage: the more functionality of the code is tested, the better
  • Clean slate: each test file should be runnable by itself, and not rely on previous test files
  • Fast: use the minimal amount of computations to test what is necessary
  • Regression: Whenever a bug is fixed, a test is added for this case
  • Input variety: attempt to cover a wide gambit of input types

One doesn't have to worry about re-writing all tests. In fact, a PR "correcting" a single test file is already very much welcomed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code-clarity Will increase code clarity of the source. good first issue help wanted testing
Projects
None yet
Development

No branches or pull requests

1 participant