New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recreates OmniFixture dump used for UI tests #22170
Conversation
f062af8
to
475a087
Compare
Just out of curiosity, do we have some documentation on how to recreate the dump / omni fixture from a newer version? |
@michalkleiner Yes. That's documented here: https://developer.matomo.org/guides/tests-system#omnifixture |
1d0937f
to
ac46233
Compare
0f6cad7
to
10619a5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to give anyone a heads-up before merging this so they are aware that their test results could change?
@mneudert Any UI test that does not specify an own fixture by default uses this one. So it might affect certain plugins... |
@matomo-org/plugin-reviewers We are going to update the dataset of UITestFixture with this PR. Any plugin UI test that relies on this fixture, might afterwards display different data, as there had been a lot changes since the fixture had been last updated. So if there are any tests not using a custom fixture, they might start failing afterwards. |
Description:
Many of our UI tests are using a prepared database dump to set up their fixture. This dump was last recreated somewhen before release of Matomo 4. So every test actually first performes an update from Matomo 3.16 to Matomo 5.
In addition that means that some data might not even be available for tests, like values for dimensions that had been added after the dump had been created.
This PR recreates the database dump and updates all screenshots accordingly.
There are a lot of changes in the screenshots. This is caused by the way our OmniFixture is created. It basically sets up all test fixtures that are defined in core tests at once. Due to many changes in such fixtures since Matomo 3.16, a lot of report data has changed now. I've looked through all the changes and too everything looked like a reasonable change in displayed report data only.
Some resulting screenshots in Treemap plugin also will need to be updated before merging this one.
Note: This will be required to run UI tests on TiDb
Review