Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Double-count issue when using two reporters #23

Open
AndrewGnagy opened this issue Sep 18, 2017 · 5 comments
Open

Double-count issue when using two reporters #23

AndrewGnagy opened this issue Sep 18, 2017 · 5 comments

Comments

@AndrewGnagy
Copy link

I'm attempting to use a custom reporter to report results to both junit and the pretty-printer.

What I've found is that this will double-count the number of assertions and failures. For example:

    Ran 5 tests in 6.059 seconds
    10 assertions, 2 failures, 2 errors.

(Should be 1 assertion per test for 5 total. 1 failure and 1 error.)

A work-around suggested by @miikka is binding clojure.test/*report-counters* to nil for one of the reports:

    (defn report [m]
      (pretty/report m)
      (binding [clojure.test/*report-counters* nil]
        (xml-report m)))

Which works.

Is there a more proper way of handling this?

@weavejester
Copy link
Owner

Because of the way clojure.test works, that might be the best solution. We could add in something that combines multiple reporters automatically, or potentially we could try and create a new reporter abstraction we could layer on top of clojure.test/report, but adding new abstractions is something I tend to be cautious about.

@Deraen
Copy link
Contributor

Deraen commented Oct 19, 2017

I'm using following function in boot-alt-test to combine collection of reporters, should we add this to eftest.report?

(defn combined-reporter
  "Combines the reporters by running first one directly,
  and others with clojure.test/*report-counters* bound to nil."
  [report & rst]
  (fn [m]
    (report m)
    (doseq [report rst]
      (binding [clojure.test/*report-counters* nil]
        (report m)))))

@aviflax
Copy link
Contributor

aviflax commented Jun 25, 2018

FWIW, I just wrote my own version of such a function, and I’m not encountering the double counting.

Update: Ah, never mind, I didn’t read the TP closely enough. I am indeed seeing the same issue with my function.

My function
(defn- multi-report
  "Accepts n reporting functions, returns a reporting function that will call
  them all for their side effects and return nil. I tried to just use juxt but
  it didn’t work. Maybe because some of the reporting functions provided by
  eftest are multimethods, I don’t know."
  [& fs]
  (fn [event]
    (doseq [f fs]
      (f event))))

@aviflax
Copy link
Contributor

aviflax commented Jun 26, 2018

Ah, never mind, I didn’t read the TP closely enough. I am indeed seeing the same issue with my function.

@buzzdan
Copy link

buzzdan commented Mar 4, 2019

i'm upvoting the need for a duel reporter --> i wanna be able to run lein eftest in my CI that produces a junit report while outputing to console in development.
(or just use both)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants