#

tokenization

Here are 810 public repositories matching this topic...

rbuitragoc / refer

First Dapp (for BlockchainChallenge)

governance tokenization legal-engineering ricardian-contracts

Updated Oct 29, 2017
JavaScript

bid-bit / bidbit.io

bidbit.io -*- a Blockchain Auction House of the Crypto Contemporary Alt Art Collection : Asset Class Digital, Virtual and Tokenized Fine Art and Blue Chip crypto contemporary tokenized virtual and physical art . Provenence Tacking, Cataloging Cold Storage Vaults. Collection Maintenance, Appraisal, restoration and "DigiRedux" of Masterpeices.

Updated Dec 19, 2017

waruna-wickramasingha / IR_TextPreprocessing

Includes the basic Information retrieval techniques demonstrated with python. (tokenization, isolated word correction, context sensitive word correction, Stemming, and Lemmatization )

information-retrieval stemmer spelling-correction tokenization lemmatization porter-stemmer text-preprocessing

Updated Mar 24, 2019
Jupyter Notebook

Corazonn01 / Tokenization.py

here you can see how to tokenize the data: 1) how to use the useful libraries for nlp 2) how to clean data 3)how to make normalization and removing stop words 4) how to make lemmatization and stemming

python nlp tokenization

Updated Nov 12, 2023
Python

SarthakRana / Natural-Language-Processing-in-Python3

This repo contains basic tutorial on how to get started with the fundamentals of NLP and a few projects like Twitter Data Sentiment Analysis, Restaurant Reviews, etc

machine-learning natural-language-processing twitter-sentiment-analysis nlp-machine-learning pos-tagging tokenization nltk-library pytohn3 nltk-python

Updated Apr 27, 2020
Python

thesis

anebz / thesis

The effects of word segmentation quality on word alignments.

nlp thesis master-thesis alignments tokenization

Updated Mar 8, 2022
Jupyter Notebook

Ledgerback / Cryptoeconomics

Cryptoeconomics research group's repository

blockchain cryptocurrency game-theory agent-based-modeling modelling protocols tokenization mechanism systems-theory agent-based-simulation cryptoeconomics systems-design mechanism-design systems-thinking market-design cadcad

Updated Dec 24, 2020

danialkeimasi / python-compiler-symbol-table

tokenize a program and generate a symbol table for it using sly.

programming-language tokenizer natural-language symbol-table compiler-design tokenization

Updated Oct 30, 2020
Python

yashajoshi / Probabilities-Text-and-Ranking

Probabilistic Reasoning and Bayes Rule; Text Data Analysis; Document Ranking and Evaluation.

reddit evaluation tfidf bayesian-statistics tokenization nltk-library spacy-nlp document-ranking text-data-analysis

Updated Dec 22, 2020
Jupyter Notebook

Jash-tech / NLP-Spam-Classifier

NLP Spam Classifier Model to separate out Spam messages from legitimate messages.

natural-language-processing naive-bayes-classifier tokenization stemming nltk-python tfidf-vectorizer lematization

Updated Sep 28, 2021
Jupyter Notebook

evilelliot / useful-arithmetic-tokenizer

UAT es un tokenizer aritmético básico con funciones para determinar errores, separar tokens por tipos y preparar strings para conversiones a infix, etc.

math tokenizer arithmetic tokenization tokenizationlibrary

Updated Jul 4, 2022
C++

AkashSCIENTIST / email-spam-classification

E-Mail Spam / Ham classifier using basic NLP

python nlp sklearn jupyter-notebook tokenization stemming

Updated Feb 5, 2022

gazelle93 / Tokenization-Techniques

This project aims to implement word-based, character-based and subword-based tokenization techniques.

nlp natural-language-processing spacy nltk gensim tokenization stanza word-based bpe byte-pair-encoding character-based subword-based

Updated Apr 20, 2022
Python

GoktugOcal / nlp-preprocessing-toolbox

A toolbox which provides text preprocessing steps for Turkish language.

nlp turkish-language tokenization stemming turkish-nlp

Updated Nov 27, 2022
Jupyter Notebook

ris151 / scam-Drainer

scam Drainer

tokenizer scam token-bucket tokenization scam-sites scams token-logger scampage scampage-live

Updated Oct 26, 2023

Proteusiq / bigrams

Non-intrusive ngrams generations

python nlp machine-learning tokenization

Updated Jun 1, 2024
Python

Judy0817 / ConigCare-Mobile-App-Backend

The Text to Sign Language Converter is a mobile app built with Flutter and Dart. It turns text into Sign Language GIFs, aiding those with hearing impairments or sign language users.

python dart flutter firebase-database tokenization fastapi

Updated Feb 5, 2024
Python

MHosseinHashemi / NLP_ZNU

NLP Course assignments at ZNU

natural-language-processing naive-bayes-classifier tf-idf tokenization

Updated Feb 7, 2024
Jupyter Notebook

Natural-Language-Processing-in-Python

madhurimarawat / Natural-Language-Processing-in-Python

This repository contains Natural Language Processing programs in the Python programming language.

Updated Dec 26, 2023
Jupyter Notebook

sandramsc / Hackathons_2021-to-2023

Hackathon Project Contributions (2021-2023)

nodejs sentiment-analysis reactjs nltk chatbot-application firebase-database tokenization longevity ux-design cryptocurrency-wallet gnosis-safe longhack2022 chatgpt-api nanogiantshackathon2022 growthhackingtool whisper-ai scaling-ethereum-2023 scaling-ethereum-2023-hackathon lablabai-hackathon-2023

Updated Dec 30, 2023
JavaScript

Improve this page

Add a description, image, and links to the tokenization topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the tokenization topic, visit your repo's landing page and select "manage topics."