Cache request responses in UI tests to avoid some external requests

[sgiehl]

Description:

Many of our UI tests might currently request the Marketplace or the Matomo API to look for updates and similar. Depending on the response time this can slow down the tests.

This PR implements a simple caching for requests sent to any Matomo domain during a UI test run. After the run, the cache will be cleared.

Across our 4 UI test suites this currently saves around 200 - 300 requests. Haven't counted that in detail though.
Hard to say how much time that saves. Comparing the run times of this PR and 5.x-dev it seems to be around 1-5 minutes for each suite.

Note: This is not meant to be a final solution and only to improve the current situation a bit. A proper solution would imho be to implement proper response mocks, where we manually place the (expected) responses instead, so the tests don't perform any requests to marketplace/api anymore.

@matomo-org/core-reviewers happy to your opinions on this (temporary) approach.

Review

• Functional review done
• Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
• Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
• Security review done
• Wording review done
• Code review done
• Tests were added if useful/possible
• Reviewed for breaking changes
• Developer changelog updated if needed
• Documentation added if needed
• Existing documentation updated if needed

[michalkleiner]

Interesting approach. So we're basically saying the back-end might be slow so we store and reuse the processed responses. In principle I don't have a problem with this but it might complicate debugging later on when someone tests things locally and changes some data and expects that to be visible but the cached response will be returned and it may take time to realise that the cache is there.
Also when we briefly touched on this, for some reason I thought it would be cached at the headless browser end running the tests, but that could be even trickier to implement.
Ultimately this is a temporary measure and mock responses should be used as the final solution, so I'd be happy with this for a while 👍.

[sgiehl]

Interesting approach. So we're basically saying the back-end might be slow so we store and reuse the processed responses. In principle I don't have a problem with this but it might complicate debugging later on when someone tests things locally and changes some data and expects that to be visible but the cached response will be returned and it may take time to realise that the cache is there.

The requests are only cached during UI tests. And each run of ./console tests:run-ui by default will clear the cached files in the beginning. So there shouldn't be any issues like that.

Also when we briefly touched on this, for some reason I thought it would be cached at the headless browser end running the tests, but that could be even trickier to implement.

How should that be possible. The requests are sent from the backend/php and not from the browser, so there is no way how the browser could cache them.

[github-actions]

If you don't want this PR to be closed automatically in 28 days then you need to assign the label 'Do not close'.

[github-actions]

This PR was last updated more than one month ago, maybe it's time to close it. Please check if there is anything we still can do or close this PR. ping @matomo-org/core-reviewers