Dynamic Wikipedia Contract Tests

[data-sync-user]

Add contract tests for the Wikipedia provider.

High Level System Overview w Contract Test Scope

!image-20221230-181021.png|width=997,height=245!

Test Plan (Scenarios)

Scenario	Behaviour
A query string that exactly matches a full Wikipedia article title is sent to Merino	A Wikipedia suggestion for the exact match article is returned (popularity score ignored)
A query string that matches multiple Wikipedia article titles is sent to Merino	A Wikipedia suggestion for the article with the highest popularity score is returned
A query string that doesn’t match any Wikipedia article titles is sent to Merino	No suggestions are returned by Merino

|Wikipedia takes longer than the query_timeout_sec to provide data

**Contingent on a mechanism to delay a response from ElasticSearch.|No suggestions are returned by Merino|

|----|----|
|A query string that results in suggestions from multiple providers is sent to merino|The Dynamic Wikipedia suggestion is ranked accordingly (Score=0.23)|

Acceptance Criteria

• A dockerized Elasticsearch fake is added to the contract test solution
  • https://hub.docker.com/_/elasticsearch/
  • Alternative could be to use a 'Stage' Elasticsearch instance.
    • This could block the CI\CD with false negatives caused by events like network outages or Elasticsearch maintenance activities
    • Developers would be unable to execute or develop tests without internet access
• The merino contract test client is updated to support verification of Wikipedia suggestions
• All cases defined in the test plan (see task description) are implemented
• Contract test README documentation is updated accordingly

┆Issue is synchronized with this Jira Task
┆Attachments: image-20221230-181021.png
┆Epic: Productionize Dynamic Wikipedia

[data-sync-user]

➤ Taddes Korris commented:

We have a bit of dependency on getting the local dev figured out for this in order for testing to work, so perhaps we’ll break this up into iterative tasks Nan Jiang . Maybe we can first test against the stage cluster and have some option to skip if unavailable so things do not break, while we also sort out the local dev requirements. Talked with Michelle about it a bit yesterday and we’ll have to do a little brainstorming.

[data-sync-user]

➤ Nan Jiang commented:

I recall that we landed on using the staging ES cluster for contract tests. But yup, let’s realign on this to finalize all the details.

[data-sync-user]

➤ Taddes Korris commented:

Thx Nan Jiang , we can discuss further just to make sure I have the right idea on this.

[data-sync-user]

➤ Michelle Tran commented:

We can probably use the staging cluster for contract tests. This means that this wouldn’t be blocked on having local dev environment.