Adding a new Dataset: Arabic Reviews of SHEIN

[Ruqyai]

Checklist for adding MMTEB dataset

Reason for dataset addition:

• I have tested that the dataset runs with the mteb package.
• I have run the following models on the task (adding the results to the pr). These can be run using the mteb run -m {model_name} -t {task_name} command.
  • sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
  • intfloat/multilingual-e5-small
• I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
• If the dataset is too big (e.g. >2048 examples), considering using self.stratified_subsampling() under dataset_transform()
• I have filled out the metadata object in the dataset file (find documentation on it here).
• Run tests locally to make sure nothing is broken using make test.
• Run the formatter to format the code using make lint.
• I have added points for my submission to the points folder using the PR number as the filename (e.g. 438.jsonl).

[imenelydiaker]

LGTM! Thanks for this addition!
I left some comments, also missing intfloat/multilingual-e5-small results.

[Ruqyai]

LGTM! Thanks for this addition! I left some comments, also missing intfloat/multilingual-e5-small results.

I did

[imenelydiaker]

LGTM, I'll let you update the points with the correct reviewers name and we'll be able to merge

[Ruqyai]

Done .. is it ok? @imenelydiaker

[Ruqyai]

Question?
As I know, today is the last day to contribute by adding a new dataset:
https://github.com/embeddings-benchmark/mteb/tree/main/docs/mmteb
Can my contribution be counted as almost done, just waiting for the merge?

@imenelydiaker
@KennethEnevoldsen
@KranthiGV

[imenelydiaker]

Question? As I know, today is the last day to contribute by adding a new dataset: https://github.com/embeddings-benchmark/mteb/tree/main/docs/mmteb Can my contribution be counted as almost done, just waiting for the merge?

@imenelydiaker @KennethEnevoldsen @KranthiGV

No worries about this, LGTM, let's merge.

[imenelydiaker]

@Ruqyai can you please run make pr locally and push changes?

[Ruqyai]

@Ruqyai can you please run make pr locally and push changes?

Done ..

======================================================== short test summary info ========================================================
FAILED tests/test_TaskMetadata.py::test_all_metadata_is_filled - ValueError: The metadata of the following datasets is not filled: ['OnlineStoreReviewSentimentClassification']
FAILED tests/test_all_abstasks.py::test_dataset_availability - aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host huggingface.co:443 ssl:True [SSLCertVerificationEr...
================================== 2 failed, 527 passed, 149 skipped, 5 warnings in 181.77s (0:03:01) ===================================
make[1]: *** [test] Error 1
make: *** [pr] Error 2

I fixed them