Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect test counting in typed/rackunit (test count doubled) #132

Open
jessealama opened this issue Jan 31, 2021 · 0 comments
Open

Incorrect test counting in typed/rackunit (test count doubled) #132

jessealama opened this issue Jan 31, 2021 · 0 comments
Labels

Comments

@jessealama
Copy link
Sponsor

Working with typed/rackunit on 7.9, CS variant, macOS. Here's a rela

#lang typed/racket/base

(module+ test
  (require typed/rackunit))

(module+ test
  (test-case "one" (check-not-false 1)))

and then do raco test, you'll get

2 tests passed

If you add a test, so that we have two test-cases, like so:

#lang typed/racket/base

(module+ test
  (require typed/rackunit))

(module+ test
  (test-case "one" (check-not-false 1))
  (test-case "two" (check-not-false 2)))

you'll now get

4 tests passed

Looks like the count is always twice the number of checks. But there's more: if you make a failing test, like this:

#lang typed/racket/base

(module+ test
  (require typed/rackunit))

(module+ test
  (test-case "one" (check-not-false 1))
  (test-case "two" (check-not-false #f))) ; fails

Then you'll get:

--------------------
two
FAILURE
name:       check-not-false
location:
  /Applications/Racket v7.9/share/pkgs/rackunit-typed/rackunit/main.rkt:41:2
params:     '(#f)
--------------------
2/4 test failures

It looks like we're double counting the number of tests and double counting the number of failed tests.

The described issue doesn't arise in untyped rackunit.

jessealama pushed a commit to jessealama/rackunit that referenced this issue Jan 31, 2021
It's not necessary to wrap a test case in
`current-test-case-around`; that will be done later. Doing
so increments the running test count (and, possibly, the
running test failure count) twice.

closes racket#132
@jackfirth jackfirth added the bug label Feb 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants