Skip to content

Latest commit

 

History

History
306 lines (303 loc) · 96.4 KB

highest.md

File metadata and controls

306 lines (303 loc) · 96.4 KB

Tatoeba Challenge Data - v2023-09-26

This is the "highest" sub-set of the Tatoeba data. Download the data files from the link in the table below. There is a total of

  • 296 language pairs in this sub-set
lang-pair test dev train
Afrikaans - English afr-eng 1374 1054 43952952
Arabic - German ara-deu 1209 1057 36446485
Arabic - Modern Greek (1453-) ara-ell 428 27300022
Arabic - English ara-eng 13290 19529 119177436
Arabic - French ara-fra 1569 125 75524167
Arabic - Hebrew ara-heb 1208 107 25176723
Arabic - Italian ara-ita 238 11 36183892
Arabic - Japanese ara-jpn 1336 11 12047170
Arabic - Polish ara-pol 208 35782511
Arabic - Russian ara-rus 3715 1304 54895295
Arabic - Spanish ara-spa 1511 1060 88211149
Arabic - Turkish ara-tur 1263 127 35974396
Azerbaijani - English aze-eng 2659 1035 76463206
Belarusian - English bel-eng 2500 4439 45986211
Belarusian - Russian bel-rus 2500 2832 22998266
Bengali - English ben-eng 2500 3063 78764429
Bulgarian - German bul-deu 317 47383235
Bulgarian - English bul-eng 10000 7835 113270896
Bulgarian - French bul-fra 451 43704625
Bulgarian - Italian bul-ita 2500 4006 43149114
Bulgarian - Russian bul-rus 1247 1001 44461448
Bulgarian - Spanish bul-spa 286 50263683
Bulgarian - Turkish bul-tur 860 1 32510816
Bulgarian - Ukrainian bul-ukr 1021 12562842
Bulgarian - Chinese bul-zho 416 15635549
Catalan - English cat-eng 1631 257 29231872
Catalan - French cat-fra 701 2 25468272
Catalan - Italian cat-ita 298 12737704
Catalan - Portuguese cat-por 747 3 14710350
Catalan - Spanish cat-spa 1534 1320 122927301
Cebuano - English ceb-eng 424 12929308
Czech - German ces-deu 3490 1260 80012943
Czech - English ces-eng 18434 16189 208877037
Czech - French ces-fra 450 5 80295146
Czech - Hungarian ces-hun 1911 192 60141550
Czech - Italian ces-ita 1100 1 54375409
Czech - Polish ces-pol 570 3 82081646
Czech - Russian ces-rus 3535 5466 46847691
Czech - Slovenian ces-slv 512 17 35171034
Czech - Spanish ces-spa 1807 219 90047439
Welsh - English cym-eng 1007 19 19178710
Danish - German dan-deu 9998 7750 85418280
Danish - English dan-eng 12678 20089 164950651
Danish - Finnish dan-fin 2665 1854 49411725
Danish - French dan-fra 1731 89 77220719
Danish - Hungarian dan-hun 882 6 39096324
Danish - Italian dan-ita 286 4 63225160
Danish - Dutch dan-nld 1643 67 71005999
Danish - Norwegian dan-nor 1311 1050 39503372
Danish - Portuguese dan-por 876 4 73045085
Danish - Russian dan-rus 1713 1065 19444845
Danish - Spanish dan-spa 5000 5165 81687841
Danish - Swedish dan-swe 2348 1440 67424560
Danish - Turkish dan-tur 770 4 19482641
Danish - Chinese dan-zho 250 12949875
German - Modern Greek (1453-) deu-ell 2500 2869 64421019
German - English deu-eng 32565 289779 775736768
German - Estonian deu-est 265 1 41253664
German - Persian deu-fas 3185 1051 14137912
German - Finnish deu-fin 3320 7142 75002315
German - French deu-fra 17388 98158 269772981
German - Serbo-Croatian deu-hbs 1959 708 58967617
German - Hebrew deu-heb 3090 1213 22663182
German - Hungarian deu-hun 23848 54083 66068675
German - Italian deu-ita 10392 12198 165213375
German - Japanese deu-jpn 12649 29943 23246645
German - Korean deu-kor 1127 9 15717548
German - Latvian deu-lav 200 33584993
German - Lithuanian deu-lit 1115 541 37435405
German - Malay (macrolanguage) deu-msa 564 2 33449005
German - Dutch deu-nld 11046 26387 156204242
German - Norwegian deu-nor 3651 1026 51017401
German - Polish deu-pol 5000 6188 96443936
German - Portuguese deu-por 10000 7821 148264788
German - Romanian deu-ron 1141 1112 60607361
German - Russian deu-rus 18028 100323 73410989
German - Slovenian deu-slv 659 7 41570897
German - Spanish deu-spa 11376 75061 245781268
German - Swedish deu-swe 3410 1378 107221536
German - Turkish deu-tur 5000 7481 43675726
German - Ukrainian deu-ukr 10682 11721 15254884
German - Vietnamese deu-vie 403 19344274
German - Chinese deu-zho 4985 7590 30458932
Modern Greek (1453-) - English ell-eng 13055 12920 158601859
Modern Greek (1453-) - French ell-fra 1506 36 91986748
Modern Greek (1453-) - Italian ell-ita 433 56385254
Modern Greek (1453-) - Dutch ell-nld 928 67 56236142
Modern Greek (1453-) - Portuguese ell-por 1161 23 94216999
Modern Greek (1453-) - Russian ell-rus 2500 2549 31922794
Modern Greek (1453-) - Spanish ell-spa 1829 1023 101015001
Modern Greek (1453-) - Swedish ell-swe 253 45936499
Modern Greek (1453-) - Turkish ell-tur 1469 200 40043669
English - English eng-eng 12663 96608 25770313
English - Esperanto eng-epo 29548 241675 70341434
English - Estonian eng-est 1359 1186 67539583
English - Basque eng-eus 1060 1005 10163682
English - Persian eng-fas 3762 1121 49093047
English - Finnish eng-fin 11786 69896 140250177
English - French eng-fra 23139 251749 837088250
English - Irish eng-gle 1913 79 24793933
English - Galician eng-glg 1019 3 14469542
English - Serbo-Croatian eng-hbs 10498 14206 214850199
English - Hebrew eng-heb 10915 153571 57840412
English - Hindi eng-hin 5000 6072 71917217
English - Hungarian eng-hun 19625 97144 139799007
English - Armenian eng-hye 1121 1036 16609621
English - Icelandic eng-isl 2658 6938 21979561
English - Italian eng-ita 29212 472974 378109944
English - Javanese eng-jav 269 2 18149854
English - Japanese eng-jpn 20796 194793 91296937
English - Georgian eng-kat 1088 18 13032106
English - Kazakh eng-kaz 434 3 35231799
English - Korean eng-kor 2479 1129 37221550
English - Latvian eng-lav 1631 189 57173732
English - Lithuanian eng-lit 2585 5643 65063233
English - Luxembourgish eng-ltz 387 4 17568120
English - Malayalam eng-mal 802 59661051
English - Marathi eng-mar 11026 43058 42041768
English - Macedonian eng-mkd 10017 70319 17437638
English - Maltese eng-mlt 251 1 22760694
English - Mongolian eng-mon 417 23 17176614
English - Malay (macrolanguage) eng-msa 5000 7487 168311638
English - Burmese eng-mya 220 10751538
English - Dutch eng-nld 17138 62387 325047981
English - Norwegian eng-nor 5000 6264 135146726
English - Polish eng-pol 11713 44189 210320051
English - Portuguese eng-por 23215 204462 429437198
English - Romanian eng-ron 6632 9662 148959401
English - Russian eng-rus 36385 504175 214645928
English - Slovenian eng-slv 2495 1796 69320844
English - Spanish eng-spa 24514 197532 1088295952
English - Albanian eng-sqi 1112 2 94122569
English - Swahili (macrolanguage) eng-swa 434 3 31213354
English - Swedish eng-swe 11452 15558 207819305
English - Tamil eng-tam 356 58105668
English - Telugu eng-tel 262 53464610
English - Tagalog eng-tgl 3978 4796 67551342
English - Thai eng-tha 1178 1 16132390
English - Turkish eng-tur 18588 658320 103684255
English - Ukrainian eng-ukr 15984 159487 48341922
English - Urdu eng-urd 1663 3 34085204
English - Uzbek eng-uzb 480 1 31142431
English - Vietnamese eng-vie 2500 4350 66439474
English - Chinese eng-zho 11344 43078 142020979
Esperanto - French epo-fra 23465 287548 20745752
Estonian - Finnish est-fin 228 30783369
Estonian - Russian est-rus 698 19 18259725
Basque - Spanish eus-spa 1850 1003 13835530
Persian - French fas-fra 390 5 15546771
Finnish - French fin-fra 1920 1085 85896276
Finnish - Hebrew fin-heb 214 4 22667395
Finnish - Hungarian fin-hun 1297 16 47667374
Finnish - Italian fin-ita 1039 1003 56059373
Finnish - Dutch fin-nld 319 4 61102693
Finnish - Norwegian fin-nor 2488 1159 25834628
Finnish - Polish fin-pol 611 50807198
Finnish - Portuguese fin-por 480 2 77097930
Finnish - Russian fin-rus 3643 1019 26486195
Finnish - Spanish fin-spa 2540 7399 91430674
Finnish - Swedish fin-swe 2959 7284 66049439
Finnish - Turkish fin-tur 1796 6 28117234
Finnish - Chinese fin-zho 386 2 14304103
French - Serbo-Croatian fra-hbs 482 75445058
French - Hebrew fra-heb 3281 1120 43205062
French - Hungarian fra-hun 3052 4848 85972093
French - Italian fra-ita 10547 68868 241647663
French - Japanese fra-jpn 10440 28652 24005427
French - Korean fra-kor 357 2 16151092
French - Malay (macrolanguage) fra-msa 1056 3 38132397
French - Dutch fra-nld 12276 16735 126187318
French - Norwegian fra-nor 478 44322734
French - Polish fra-pol 3087 1029 99603185
French - Portuguese fra-por 11338 17062 237762189
French - Romanian fra-ron 1927 80 103361533
French - Russian fra-rus 28800 183050 104428918
French - Slovenian fra-slv 476 42006869
French - Spanish fra-spa 11338 40595 448880106
French - Swedish fra-swe 1407 1158 94211699
French - Turkish fra-tur 2784 6718 54477278
French - Ukrainian fra-ukr 10236 18252 14293798
French - Vietnamese fra-vie 1299 8 23220049
French - Chinese fra-zho 9994 7685 55340114
Galician - Portuguese glg-por 434 3 77641252
Galician - Spanish glg-spa 2121 1013 41706134
Serbo-Croatian - Italian hbs-ita 539 53317434
Serbo-Croatian - Japanese hbs-jpn 400 10469418
Serbo-Croatian - Norwegian hbs-nor 5000 6150 24710098
Serbo-Croatian - Polish hbs-pol 417 73430012
Serbo-Croatian - Russian hbs-rus 2500 4267 50669025
Serbo-Croatian - Spanish hbs-spa 620 2 79678925
Serbo-Croatian - Turkish hbs-tur 203 56072748
Serbo-Croatian - Ukrainian hbs-ukr 945 7 11329784
Serbo-Croatian - Chinese hbs-zho 238 26220456
Hebrew - Hungarian heb-hun 426 11 28172496
Hebrew - Italian heb-ita 1706 48 18797552
Hebrew - Japanese heb-jpn 243 4 10978685
Hebrew - Dutch heb-nld 2500 4372 30156880
Hebrew - Polish heb-pol 5001 7765 31960813
Hebrew - Portuguese heb-por 737 24 51780514
Hebrew - Russian heb-rus 2500 3602 23759381
Hebrew - Spanish heb-spa 1849 1117 46166848
Hebrew - Turkish heb-tur 1377 44 30677034
Hebrew - Chinese heb-zho 709 5 14485399
Hungarian - Italian hun-ita 5000 7329 54016461
Hungarian - Dutch hun-nld 1629 416 53597028
Hungarian - Norwegian hun-nor 311 20865802
Hungarian - Polish hun-pol 1934 56 58729635
Hungarian - Portuguese hun-por 2500 3533 80763178
Hungarian - Russian hun-rus 4233 6098 33905931
Hungarian - Spanish hun-spa 2500 4608 93000023
Hungarian - Swedish hun-swe 2501 1128 43606965
Hungarian - Turkish hun-tur 1059 10 37074809
Hungarian - Chinese hun-zho 443 1 16376905
Italian - Japanese ita-jpn 2665 1099 17358575
Italian - Lithuanian ita-lit 225 27339552
Italian - Malay (macrolanguage) ita-msa 430 23636730
Italian - Dutch ita-nld 2734 6940 91734309
Italian - Norwegian ita-nor 939 11 32565288
Italian - Polish ita-pol 1294 1006 64374775
Italian - Portuguese ita-por 3599 6371 157099014
Italian - Romanian ita-ron 1013 8 66939184
Italian - Russian ita-rus 11874 66133 46309983
Italian - Spanish ita-spa 5000 9634 247394065
Italian - Swedish ita-swe 719 16 66775172
Italian - Turkish ita-tur 10000 5727 39157018
Italian - Vietnamese ita-vie 250 15286754
Italian - Chinese ita-zho 2943 1019 21301073
Japanese - Korean jpn-kor 653 3 12910538
Japanese - Malay (macrolanguage) jpn-msa 2644 1192 15060655
Japanese - Dutch jpn-nld 3443 1125 18847524
Japanese - Polish jpn-pol 10018 14812 18316761
Japanese - Portuguese jpn-por 1943 1245 19286865
Japanese - Russian jpn-rus 10448 15719 19411793
Japanese - Spanish jpn-spa 11712 23471 24517283
Japanese - Swedish jpn-swe 910 5 14277684
Japanese - Turkish jpn-tur 829 4 18669222
Japanese - Chinese jpn-zho 2500 3039 14908595
Korean - Russian kor-rus 221 12246920
Korean - Spanish kor-spa 942 1 17097642
Latvian - Russian lav-rus 280 13123612
Lithuanian - Polish lit-pol 1787 6 29138566
Lithuanian - Russian lit-rus 4998 9272 18047224
Lithuanian - Spanish lit-spa 456 1 45183984
Lithuanian - Turkish lit-tur 1472 6 10996374
Malay (macrolanguage) - Malay (macrolanguage) msa-msa 1000 171 20477305
Malay (macrolanguage) - Spanish msa-spa 254 55129975
Malay (macrolanguage) - Chinese msa-zho 420 8 15380741
Dutch - Norwegian nld-nor 205 1 36766305
Dutch - Polish nld-pol 1193 1022 80448427
Dutch - Portuguese nld-por 2668 4882 112550031
Dutch - Romanian nld-ron 2487 1350 53825367
Dutch - Russian nld-rus 2723 6526 46328595
Dutch - Spanish nld-spa 10554 18068 137230519
Dutch - Turkish nld-tur 2500 3898 48164735
Dutch - Chinese nld-zho 1654 463 20490670
Norwegian - Polish nor-pol 305 23233255
Norwegian - Portuguese nor-por 482 3 39454838
Norwegian - Russian nor-rus 1279 5 13035964
Norwegian - Spanish nor-spa 964 6 52588719
Norwegian - Swedish nor-swe 596 4 44746497
Polish - Portuguese pol-por 740 112679497
Polish - Romanian pol-ron 201 57043671
Polish - Russian pol-rus 3543 1066 58582697
Polish - Spanish pol-spa 2754 5075 108422276
Polish - Swedish pol-swe 1392 61 64907489
Polish - Turkish pol-tur 913 51040052
Polish - Ukrainian pol-ukr 2706 6898 11773783
Polish - Chinese pol-zho 1004 2 20004851
Portuguese - Portuguese por-por 2500 4783 44495182
Portuguese - Romanian por-ron 757 31 90988564
Portuguese - Russian por-rus 10797 10050 69138805
Portuguese - Spanish por-spa 11875 56853 338247062
Portuguese - Swedish por-swe 323 13 90547760
Portuguese - Turkish por-tur 1794 76 79208095
Portuguese - Ukrainian por-ukr 3372 1069 11436463
Portuguese - Chinese por-zho 1052 3 36244863
Romanian - Russian ron-rus 802 7 21542856
Romanian - Spanish ron-spa 1959 115 112769744
Romanian - Turkish ron-tur 2495 1023 38595534
Russian - Slovenian rus-slv 707 45 23459454
Russian - Spanish rus-spa 11867 86903 111138411
Russian - Swedish rus-swe 1282 68 31408868
Russian - Turkish rus-tur 5000 6439 35352052
Russian - Ukrainian rus-ukr 10000 7667 34212183
Russian - Vietnamese rus-vie 313 1 14986395
Russian - Chinese rus-zho 2500 4867 41647108
Slovenian - Chinese slv-zho 826 10956062
Spanish - Swedish spa-swe 1351 254 95280237
Spanish - Turkish spa-tur 11301 18213 62050292
Spanish - Ukrainian spa-ukr 10233 12969 16821154
Spanish - Vietnamese spa-vie 594 8 40638094
Spanish - Chinese spa-zho 2797 6923 58565593
Swedish - Turkish swe-tur 233 6 34050629
Swedish - Chinese swe-zho 1066 2 15809901
Turkish - Chinese tur-zho 986 19 19781955
Vietnamese - Chinese vie-zho 442 4 10781289