Skip to content

ardamavi/Vocalize-Sign-Language

Repository files navigation

Vocalize Sign Language

By Arda Mavi

Vocalization sign language with deep learning.

In this project we use our own Sign Language Dataset.

Vocalization sign language iOS App: Vocalization-Sign-Language-iOS

Demo
Watch Demo Videos 🔊

Contents:

For Users

For Developers

! Important Notes

For Users:

Important Notes For Users:

  • This project works best in the white background and good light.

Additional Info:

In this project, I added deep learning to my old lip reading project SesimVar(Turkish).

Running program:

Note: If you are failed, look up For Development title in bellow.

Using Live Vocalize Command:

python3 live.py Note: If you want, you can change the delay time.

Using Predict Command:

python3 predict.py <ImageFileName>

For Developers:

Getting Dataset:

cd Data && chmod +x download_dataset.sh && ./download_dataset.sh

New dataset is available on Kaggle: 27 Class Sign Language Dataset - Kaggle

(Updated) Model coming soon!

Demo
Watch Demo Videos 🔊
DataSet Examples Model Accuracy

Dataset:

In this project we use our own Sign Language Digits Dataset.

Dataset Preview:

Artificial Intelligence Model Accuracy:

At the end of 25 epochs, 97% accuracy was achieved in the test without data augmentation:


At the end of 25 epochs, 99% accuracy was achieved in the test with data augmentation:

Model Architecture:

  • Input Data Shape: 64x64x1

  • Convolutional Layer 32 filter Filter shape: 3x3 Strides: 1x1 Padding: Same

  • Activation Function: ReLu

  • Convolutional Layer 64 filter Filter shape: 3x3 Strides: 1x1 Padding: Same

  • Activation Function: ReLu

  • Max Pooling Pool shape: 2x2 Strides: 2x2

  • Convolutional Layer 64 filter Filter shape: 3x3 Strides: 1x1 Padding: Same

  • Activation Function: ReLu

  • Max Pooling Pool shape: 2x2 Strides: 2x2

  • Convolutional Layer 128 filter Filter shape: 3x3 Strides: 1x1 Padding: Same

  • Activation Function: ReLu

  • Max Pooling Pool shape: 2x2 Strides: 2x2

  • Flatten

  • Dense Size: 526

  • Activation Function: ReLu

  • Dropout Rate: 0.5

  • Dense Size: 128

  • Activation Function: ReLu

  • Dropout Rate: 0.5

  • Dense Size: Class size in dataset

  • Activation Function: Softmax

Optimizer: Adadelta
Loss: Categorical Crossentropy

Total params: 4,507,864
Trainable params: 4,507,864
Non-trainable params: 0

Model Training:

python3 train.py

Not forget to download dataset before training!

Using TensorBoard:

tensorboard --logdir=Data/Checkpoints/logs

Creating Dataset:

For getting dataset look up Getting Dataset title in this file.

For your own dataset:

  • Create 'Data/Train_Data' folder.
  • Create folder in 'Data/Train_Data' folder and rename what you want to add char or string.
  • In your created char or string named folder add much photos about created char or string named folder. Note: We work on 64x64 image also if you use bigger, program will automatically return to 64x64.

Important Notes:

  • Used Python Version: 3.6.0
  • Install necessary modules with sudo pip3 install -r requirements.txt command.
  • Install OpenCV (We use version: 3.2.0-dev)

Cite as:

Mavi, A., (2020), “A New Dataset and Proposed Convolutional Neural Network Architecture for Classification of American Sign Language Digits”, arXiv:2011.08927 [cs.CV]

About

Vocalization sign language with deep learning.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published