Test matrix

what tests are there

For each module in the code there is a ...\src\test\java directory that holds the tests, you list all files using the file search and look for "Test.java"

test coverage

Test coverage is measured using both CodeCov, you can see some pretty pictures there, as well as JoCoCo (on the -private- Jenkins server).

current test matrix

database	Test OS	database OS	CI server	unit tests	Viewer integration tests	Viewer-admin integration tests	integration test container	scan for vulnerable dependencies (OWASP scanner)	java runtime
Oracle 12.2	Ubuntu	Oracle linux	Jenkins	☐	☑	☑	Tomcat 7	☑	Oracle JDK 8
PostgreSQL 9.6	Ubuntu	Ubuntu	Travis-CI	☐	☑	☑	Tomcat 7	☐	Open JDK 8
PostgreSQL 9.6	Ubuntu	Ubuntu	Travis-CI	☐	☑	☑	Tomcat 7	☐	Oracle JDK 8
MS SQL Server 2017	Windows 2016	Windows 2016	AppVeyor	☐	☑	☑	Tomcat 7	☐	Oracle JDK 8
HSQLDB	Ubuntu & Windows		all	☑	☐	☐		☐	Oracle & Open JDK 8

This page may go out-of-date; the actual matrix can be distilled using the following files in the source:

• Jenkinsfile which runs on Jenkins (note that this is a private instance, not available on the internet)
• .travis.yml which runs on Travis-CI
• appveyor.yml which runs on AppVeyor

future plans

Some things we need to add/expand on:

• expand the set of tested databases and versions
• run integration tests on Tomcat 9 (as well as Tomcat 7), see: #1365
• start testing on Java 11, see: #1366
• execute unit tests against deployment databases (not just HSQLDB), this requires changes in TestUtil#setUp()
• integration testing of the various ActionBean (rest) interfaces, eg. "edit a feature"
• integration or unit testing of the GUI using eg. HtmlUnit or Selenium
• ...