Testing guide

It is very important to create automated tests for the code we write. This helps us to recognize when a change breaks some existing functionality. It also helps to update the third-party dependencies (frontend/node_modules, backend/vendor, etc.) automatically. And it lets us deploy often with confidence that the application will continue to work as expected.

Running tests

The tests are run automatically on every pull request and on every branch that we work on. You can also run them locally on your computer, once you have the application up and running:

• API: docker compose exec php composer test
• Frontend: docker compose exec frontend npm run test:unit
• Print service: docker compose exec print npm run test
• Client-side PDF service: docker compose exec pdf npm run test:unit
• End-to-end tests: docker run -v $PWD/e2e:/e2e -w /e2e --network host cypress/included:12.14.0

Running just a single test or single test file

• API: docker compose exec php composer test -- tests/Api/Categories/CreateCategoryTest.php runs a single test file
• Frontend: docker compose exec frontend npm run test:unit auth runs all test files containing auth in the filename
• Print service: docker compose exec print npm run test package runs all test files containing package in the filename
• Client-side PDF service: docker compose exec pdf npm run test:unit renderer runs all test files containing renderer in the filename
• End-to-end tests: docker run -v $PWD/e2e:/e2e -w /e2e --network host cypress/included:12.14.0 --spec specs/login.cy.js runs a single test file

Writing tests

Generally, we should write a test every time we fix a bug, and every time we implement or change code which could conceivably break in the future. So, for a trivial getter and setter, we might not need a test. But as soon as the getter does more than just return a variable, it might be worth it to write a unit test for it.

Since the different services of eCamp v3 use different technologies and programming languages, each service has its own infrastructure for testing. That's why we have separate guides for writing API tests (TODO), frontend tests and end-to-end tests (TODO).

There are also different types of tests.

For one, there are unit tests, which usually test no more than a single method or function. Unit tests are the fastest to execute and the cheapest to maintain, so most of our automated tests should be unit tests.

Then there are higher abstraction levels of tests, such as snapshot tests, integration tests and HTTP request tests. These usually test multiple components working together, and are a little harder to write in a way that doesn't break on every code change to one of the involved components.

Finally, there are end-to-end tests, which simulate a user clicking through the UI and check for displayed texts and similar. End-to-end tests (or E2E tests in short) are the most expensive to execute, because they need the full application to be running, and because it is a lot of work to prepare the state of the application (i.e. the database contents, the page the browser is currently on, ...) before each test in a consistent, reproducible way. E2E tests are also very hard to write, such that they don't break all the time when new features are implemented. For these reasons, we only write E2E tests for a few essential use cases, such as a user logging in.