Skip to content

Commit

Permalink
Introduce flags for line and form coverage thresholds
Browse files Browse the repository at this point in the history
  • Loading branch information
Kitti Kristof committed Feb 16, 2024
1 parent 87fd10f commit 698e9ad
Show file tree
Hide file tree
Showing 5 changed files with 98 additions and 70 deletions.
11 changes: 6 additions & 5 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,19 @@ provide a short summary description of the particular changes of the patch.
Include the issue number (#xxx) which will link back to the originating issue
in github. Commentary on the change should appear as a nested, unordered list. -->

## 1.2.5 (WIP)
## 1.2.5

Nothing yet!
- Features
- Option to set thresholds separately for line and form coverage

## 1.2.4

- Make validation errors in the project settings fail Cloverage runs
- Fixes [#331](https://github.com/cloverage/cloverage/issues/331)
- Upgrade various dependencies
- Fixes [218](https://github.com/cloverage/cloverage/issues/218)
- Fixes [319](https://github.com/cloverage/cloverage/issues/319)
- Fixes [334](https://github.com/cloverage/cloverage/issues/334)
- Fixes [218](https://github.com/cloverage/cloverage/issues/218)
- Fixes [319](https://github.com/cloverage/cloverage/issues/319)
- Fixes [334](https://github.com/cloverage/cloverage/issues/334)

Also some more development niceties, like improvements to CI (including a
proper CircleCI test matrix).
Expand Down
59 changes: 30 additions & 29 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,35 +52,36 @@ project value is used.

Available options and command-line arguments:
```
Project Switches Default Desc
------- -------- ------- ----
:output -o, --output target/coverage Output directory.
:text? --no-text, --text false Produce a text report.
:html? --no-html, --html true Produce an HTML report.
:emma-xml? --no-emma-xml, --emma-xml false Produce an EMMA XML report. [emma.sourceforge.net]
:lcov? --no-lcov, --lcov false Produce a lcov/gcov report.
:codecov? --no-codecov, --codecov false Generate a JSON report for Codecov.io
:coveralls? --no-coveralls, --coveralls false Send a JSON report to Coveralls if on a CI server
:junit? --no-junit, --junit false Output test results as junit xml file. Supported in :clojure.test runner
:raw? --no-raw, --raw false Output raw coverage data (for debugging).
:summary? --no-summary, --summary true Prints a summary
:fail-threshold --fail-threshold 0 Sets the percentage threshold at which cloverage will abort the build. Default: 0%
:low-watermark --low-watermark 50 Sets the low watermark percentage (valid values 0..100). Default: 50%
:high-watermark --high-watermark 80 Sets the high watermark percentage (valid values 0..100). Default: 80%
:debug? -d, --no-debug, --debug false Output debugging information to stdout.
:runner -r, --runner :clojure.test Specify which test runner to use. Built-in runners are `clojure.test`, `midje` and `eftest`.
:runner-opts (not allowed as a cli arg) {} Configure specified runner with any options map.
:nop? --no-nop, --nop false Instrument with noops.
:ns-regex -n, --ns-regex [] Regex for instrumented namespaces (can be repeated).
:ns-exclude-regex -e, --ns-exclude-regex [] Regex for namespaces not to be instrumented (can be repeated).
:exclude-call --exclude-call [] Name of fn/macro whose call sites are not to be instrumented (can be repeated).
:test-ns-regex -t, --test-ns-regex [] Regex for test namespaces (can be repeated).
:src-ns-path -p, --src-ns-path [] Path (string) to directory containing source code namespaces (can be repeated).
:test-ns-path -s, --test-ns-path [] Path (string) to directory containing test namespaces (can be repeated).
:extra-test-ns -x, --extra-test-ns [] Additional test namespace (string) to add (can be repeated).
:custom-report -c, --custom-report Load and run a custom report writer. Should be a namespaced symbol. The function is passed
project-options args-map output-directory forms
:help? -h, --no-help, --help false Show help.
Project Switches Default Desc
------- -------- ------- ----
:output -o, --output target/coverage Output directory.
:text? --no-text, --text false Produce a text report.
:html? --no-html, --html true Produce an HTML report.
:emma-xml? --no-emma-xml, --emma-xml false Produce an EMMA XML report. [emma.sourceforge.net]
:lcov? --no-lcov, --lcov false Produce a lcov/gcov report.
:codecov? --no-codecov, --codecov false Generate a JSON report for Codecov.io
:coveralls? --no-coveralls, --coveralls false Send a JSON report to Coveralls if on a CI server
:junit? --no-junit, --junit false Output test results as junit xml file. Supported in :clojure.test runner
:raw? --no-raw, --raw false Output raw coverage data (for debugging).
:summary? --no-summary, --summary true Prints a summary
:fail-threshold --fail-threshold 0 Sets the percentage threshold for both line and form coverage at which cloverage will abort the build. Default: 0%
:line-fail-threshold --line-fail-threshold 0 Sets the percentage threshold for line coverage at which cloverage will abort the build. Ignored if --fail-threshold is non-zero. Default: 0%
:form-fail-threshold --form-fail-threshold 0 Sets the percentage threshold for form coverage at which cloverage will abort the build. Ignored if --fail-threshold is non-zero. Default: 0%
:low-watermark --low-watermark 50 Sets the low watermark percentage (valid values 0..100). Default: 50%
:high-watermark --high-watermark 80 Sets the high watermark percentage (valid values 0..100). Default: 80%
:debug? -d, --no-debug, --debug false Output debugging information to stdout.
:runner -r, --runner :clojure.test Specify which test runner to use. Built-in runners are `clojure.test`, `midje` and `eftest`.
:runner-opts (not allowed as a cli arg) {} Configure specified runner with any options map.
:nop? --no-nop, --nop false Instrument with noops.
:ns-regex -n, --ns-regex [] Regex for instrumented namespaces (can be repeated).
:ns-exclude-regex -e, --ns-exclude-regex [] Regex for namespaces not to be instrumented (can be repeated).
:exclude-call --exclude-call [] Name of fn/macro whose call sites are not to be instrumented (can be repeated).
:test-ns-regex -t, --test-ns-regex [] Regex for test namespaces (can be repeated).
:src-ns-path -p, --src-ns-path [] Path (string) to directory containing source code namespaces (can be repeated).
:test-ns-path -s, --test-ns-path [] Path (string) to directory containing test namespaces (can be repeated).
:extra-test-ns -x, --extra-test-ns [] Additional test namespace (string) to add (can be repeated).
:custom-report -c, --custom-report Load and run a custom report writer. Should be a namespaced symbol. The function is passed project-options args-map output-directory forms
:help? -h, --no-help, --help false Show help.
```

### mvn
Expand Down
66 changes: 39 additions & 27 deletions cloverage/src/cloverage/args.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,32 +19,34 @@
(every? symbol? coll))

(def valid
{:text? boolean?
:html? boolean?
:raw? boolean?
:emma-xml? boolean?
:junit? boolean?
:lcov? boolean?
:codecov? boolean?
:coveralls? boolean?
:summary? boolean?
:colorize? boolean?
:fail-threshold integer?
:low-watermark integer?
:high-watermark integer?
:debug? boolean?
:nop? boolean?
:extra-test-ns regexes-or-strings?
:help? boolean?
:ns-regex regexes-or-strings?
:test-ns-regex regexes-or-strings?
:ns-exclude-regex regexes-or-strings?
:exclude-call symbols?
:src-ns-path regexes-or-strings?
:runner keyword?
:runner-opts map?
:test-ns-path regexes-or-strings?
:custom-report symbol?})
{:text? boolean?
:html? boolean?
:raw? boolean?
:emma-xml? boolean?
:junit? boolean?
:lcov? boolean?
:codecov? boolean?
:coveralls? boolean?
:summary? boolean?
:colorize? boolean?
:fail-threshold integer?
:line-fail-threshold integer?
:form-fail-threshold integer?
:low-watermark integer?
:high-watermark integer?
:debug? boolean?
:nop? boolean?
:extra-test-ns regexes-or-strings?
:help? boolean?
:ns-regex regexes-or-strings?
:test-ns-regex regexes-or-strings?
:ns-exclude-regex regexes-or-strings?
:exclude-call symbols?
:src-ns-path regexes-or-strings?
:runner keyword?
:runner-opts map?
:test-ns-path regexes-or-strings?
:custom-report symbol?})

(defn- fn-sym [^Object f]
(let [[_ f-ns f-n] (re-matches #"(.*)\$(.*?)(__[0-9]+)?" (.. f getClass getName))]
Expand Down Expand Up @@ -140,7 +142,17 @@
["--[no-]colorize"
"Adds ANSI color to the summary" :default true]
["--fail-threshold"
"Sets the percentage threshold at which cloverage will abort the build. Default: 0%"
"Sets the percentage threshold for both line and form coverage at which cloverage will abort the build. Default: 0%"
:default 0
:parse-fn #(Integer/parseInt %)]
["--line-fail-threshold"
"Sets the percentage threshold for line coverage at which cloverage will abort the build.
Ignored if --fail-threshold is non-zero. Default: 0%"
:default 0
:parse-fn #(Integer/parseInt %)]
["--form-fail-threshold"
"Sets the percentage threshold for form coverage at which cloverage will abort the build.
Ignored if --fail-threshold is non-zero. Default: 0%"
:default 0
:parse-fn #(Integer/parseInt %)]
["--low-watermark"
Expand Down
30 changes: 21 additions & 9 deletions cloverage/src/cloverage/coverage.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -288,16 +288,25 @@
:output output
:forms forms})))

(defn- coverage-under? [forms failure-threshold]
(when (pos? failure-threshold)
(let [pct-covered (apply min (vals (rep/total-stats forms)))
failed? (< pct-covered failure-threshold)]
(when failed?
(println "Failing build as coverage is below threshold of" failure-threshold "%"))
failed?)))
(defn- coverage-under? [forms failure-threshold line-failure-threshold form-failure-threshold]
(let [{:keys [percent-lines-covered percent-forms-covered]} (rep/total-stats forms)]
(if (pos? failure-threshold)
(let [pct-covered (min percent-lines-covered percent-forms-covered)
failed? (< pct-covered failure-threshold)]
(when failed?
(println "Failing build as coverage is below threshold of" failure-threshold "%"))
failed?)
(when (or (pos? line-failure-threshold) (pos? form-failure-threshold))
(let [line-failed? (< percent-lines-covered line-failure-threshold)
form-failed? (< percent-forms-covered form-failure-threshold)]
(when line-failed?
(println "Failing build as line coverage is below threshold of" line-failure-threshold "%"))
(when form-failed?
(println "Failing build as form coverage is below threshold of" form-failure-threshold "%"))
(or line-failed? form-failed?))))))

(defn run-main
[[{:keys [debug? fail-threshold help?], :as opts} add-nses help] project-opts]
[[{:keys [debug? fail-threshold line-fail-threshold form-fail-threshold help?], :as opts} add-nses help] project-opts]
(binding [*ns* (find-ns 'cloverage.coverage)
debug/*debug* debug?]
(if help?
Expand All @@ -308,7 +317,10 @@
exit-code (cond
(not test-result) -1
(> num-errors 128) -2
(coverage-under? forms fail-threshold) -3
(coverage-under? forms
fail-threshold
line-fail-threshold
form-fail-threshold) -3
:else num-errors)]
(report-results opts project-opts forms)
(if *exit-after-test*
Expand Down
2 changes: 2 additions & 0 deletions cloverage/tests.edn
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
:lcov? false,
:high-watermark 80,
:fail-threshold 0,
:line-fail-threshold 0,
:form-fail-threshold 0,
:output "target/coverage",
:low-watermark 50,
:ns-regex [],
Expand Down

0 comments on commit 698e9ad

Please sign in to comment.