language-identifier

Here are 11 public repositories matching this topic...

cisnlp / GlotLID

GlotLID: Language Identification with Support for More Than 2000 Labels (EMNLP 2023).

language-detection multlingual language-detector language-recognition glot lid language-identification language-classification language-identification-toolkit low-resource-languages language-detection-library language-identifier language-detection-lib langid low-resource-nlp

Updated May 12, 2024
Python

currentslab / fastlangid

Star

fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-hant)

identification traditional-chinese simplified-chinese fasttext language-identification cantonese-language language-identifier

Updated Dec 6, 2022
Python

loretoparisi / fastLangID

Star

Stand-alone Language Identification for Node.js JavaScript based on FastText

language-detection neural-networks fasttext language-identifier

Updated Mar 23, 2019
JavaScript

labrijisaad / Language-Identifier-SVM

Star

Language identification script that can detect the language of a given text. Currently supports Swahili, Wolof, French, English, Arabic, and Dyula. Customizable language support.

python svm support-vector-machines language-identifier dyula french-identification swahili-identification arabic-identification wolof-identification english-identification

Updated Dec 19, 2022
Jupyter Notebook

martin-majlis / YALI

Star

Yet Another Language Identifier

nlp perl language-identifier

Updated Feb 2, 2019
Perl

caiselvass / language-identification

Star

An NLP project leveraging character trigrams and smoothing techniques (Lidstone, Linear Discounting, Absolute Discounting) for language identification. Trained on for Spanish, Italian, English, French, Dutch, and German, achieving 99.8932% accuracy. Includes datasets, model parameters, and comprehensive documentation.

nlp natural-language-processing language-identification trigram-model language-identifier smoothing-methods