fix: Convert Reddit cluster s2s and p2p to fast

[isaac-chung]

Checklist for adding MMTEB dataset

Resolve #728

• I have tested that the dataset runs with the mteb package.
• I have run the following models on the task (adding the results to the pr). These can be run using the mteb run -m {model_name} -t {task_name} command.
  • sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
  • intfloat/multilingual-e5-small
• I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
• If the dataset is too big (e.g. >2048 examples), considering using self.stratified_subsampling() under dataset_transform()
• I have filled out the metadata object in the dataset file (find documentation on it here).
• Run tests locally to make sure nothing is broken using make test.
• Run the formatter to format the code using make lint.
• I have added points for my submission to the points folder using the PR number as the filename (e.g. 438.jsonl).

[isaac-chung]

When subsampling for the p2p dataset, there are a few labels that only has 1 count. So we get the following error:

ValueError: The least populated class in labels column has only 1 member, which is too few. The minimum number of groups for any class cannot be less than 2.

Should we exclude them before subsampling? Or force include them somehow?

[KennethEnevoldsen]

When subsampling for the p2p dataset, there are a few labels that only has 1 count.

Clusters with only one label?

[isaac-chung]

Clusters with only one label?

Yes. You should be able to see this error when you check out the PR and run

gh pr checkout 729
mteb -t RedditFastClusteringP2P -m sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

[KennethEnevoldsen]

Hmm, that seems weird. This should be there before conversion as well I assume? It might be that the cluster sets does not have the same labels across the sets.

[isaac-chung]

Yeah. But before conversion, we do not use subsampling.

The following labels only have 1 sample in the clusters after combining all rows:

array(['Bariloche', 'CRISPR', 'RomanianWolves', 'TherosDMs',
       'ValorantBrasil', 'auntienetwork', 'candlemagick', 'graylog',
       'imaginarypenpals', 'visualization'], dtype='<U21')

[KennethEnevoldsen]

We should probably filter out all labels with less than N. I assume N=1-5 is reasonable?

[isaac-chung]

Went with N=1.