Skip to content

Latest commit

 

History

History
192 lines (136 loc) · 4.55 KB

README.md

File metadata and controls

192 lines (136 loc) · 4.55 KB
Raw

Vocalize Sign Language

By Arda Mavi

Vocalization sign language with deep learning.

In this project we use our own Sign Language Dataset.

Vocalization sign language iOS App: Vocalization-Sign-Language-iOS

Demo
Watch Demo Videos 🔊

Contents:

For Users

For Developers

! Important Notes

For Users:

Important Notes For Users:

  • This project works best in the white background and good light.

Additional Info:

In this project, I added deep learning to my old lip reading project SesimVar(Turkish).

Running program:

Note: If you are failed, look up For Development title in bellow.

Using Live Vocalize Command:

python3 live.py Note: If you want, you can change the delay time.

Using Predict Command:

python3 predict.py <ImageFileName>

For Developers:

Getting Dataset:

cd Data && chmod +x download_dataset.sh && ./download_dataset.sh

New dataset is available on Kaggle: 27 Class Sign Language Dataset - Kaggle

(Updated) Model coming soon!

Demo
Watch Demo Videos 🔊
DataSet Examples Model Accuracy

Dataset:

In this project we use our own Sign Language Digits Dataset.

Dataset Preview:

Artificial Intelligence Model Accuracy:

At the end of 25 epochs, 97% accuracy was achieved in the test without data augmentation:


At the end of 25 epochs, 99% accuracy was achieved in the test with data augmentation:

Model Architecture:

  • Input Data Shape: 64x64x1

  • Convolutional Layer 32 filter Filter shape: 3x3 Strides: 1x1 Padding: Same

  • Activation Function: ReLu

  • Convolutional Layer 64 filter Filter shape: 3x3 Strides: 1x1 Padding: Same

  • Activation Function: ReLu

  • Max Pooling Pool shape: 2x2 Strides: 2x2

  • Convolutional Layer 64 filter Filter shape: 3x3 Strides: 1x1 Padding: Same

  • Activation Function: ReLu

  • Max Pooling Pool shape: 2x2 Strides: 2x2

  • Convolutional Layer 128 filter Filter shape: 3x3 Strides: 1x1 Padding: Same

  • Activation Function: ReLu

  • Max Pooling Pool shape: 2x2 Strides: 2x2

  • Flatten

  • Dense Size: 526

  • Activation Function: ReLu

  • Dropout Rate: 0.5

  • Dense Size: 128

  • Activation Function: ReLu

  • Dropout Rate: 0.5

  • Dense Size: Class size in dataset

  • Activation Function: Softmax

Optimizer: Adadelta
Loss: Categorical Crossentropy

Total params: 4,507,864
Trainable params: 4,507,864
Non-trainable params: 0

Model Training:

python3 train.py

Not forget to download dataset before training!

Using TensorBoard:

tensorboard --logdir=Data/Checkpoints/logs

Creating Dataset:

For getting dataset look up Getting Dataset title in this file.

For your own dataset:

  • Create 'Data/Train_Data' folder.
  • Create folder in 'Data/Train_Data' folder and rename what you want to add char or string.
  • In your created char or string named folder add much photos about created char or string named folder. Note: We work on 64x64 image also if you use bigger, program will automatically return to 64x64.

Important Notes:

  • Used Python Version: 3.6.0
  • Install necessary modules with sudo pip3 install -r requirements.txt command.
  • Install OpenCV (We use version: 3.2.0-dev)

Cite as:

Mavi, A., (2020), “A New Dataset and Proposed Convolutional Neural Network Architecture for Classification of American Sign Language Digits”, arXiv:2011.08927 [cs.CV]