Skip to content

jayshah19949596/DeepSign-A-Deep-Learning-Architecture-for-Sign-Language-Recognition

Repository files navigation

DeepSign: A Deep Learning Architecture for Sign-Language-Recognition

DeepSign is a new deep-learning architecture which achieves comparable results with limited training data for Sign Language Recognition.

Paper Link

https://rc.library.uta.edu/uta-ir/bitstream/handle/10106/27803/SHAH-THESIS-2018.pdf?sequence=1&isAllowed=y

Medium Post

https://medium.com/@jayshah_84248/deepsign-a-deep-learning-pipeline-for-sign-language-recognition-a51a8f116dfc

core:

  • This is the folder where the training code is and the testing code is.
  • It has following files:
    1. train_auto_encoder_1.py

      • This file consist the code which trains the auto-encoder.
      • This auto encoder is Model-1.
    2. train_bi_lstm.py

      • This file consist the code which trains the bi-directional LSTM.
      • This bi-directio2nal LSTM is Model-2.
      • This file also loads the Model-1 and only takes output from enocder of Model-1 to bi-directional LSTM.
    3. train_lstm.py

      • This file consist the code which trains the uni-directional LSTM.
      • This uni-directio2nal LSTM is Model-2.
      • This file also loads the Model-1 and only takes output from enocder of Model-1 to uni-directional LSTM.
    4. train_vae.py

      • This file consist the code which trains the variational auto-encoder.
      • This auto encoder is Model-1.
    5. test_bi_lstm.py

      • This file consists of the code which does inference of bi-directional LSTM.
      • This file loads the freezed model and does predictions on test data.
    6. test_lstm.py

      • This file consists of the code which does inference of uni-directional LSTM.
      • This file loads the freezed model and does predictions on test data.

models:

  1. auto_encoder_1.py

    • This file consist the architecture of auto-encoder.
    • This auto-encoder is Model-1.
    • It is 10 Layered encoder and 15 layered decoder.
    • The file also defines the cost function and the optimizer.
    • This file is used by train_auto_encoder_1.py of core module.
  2. bi_lstm.py

    • This file consist the architecture of bi-directional lstm.
    • This bi-directional lstm is Model-2.
    • The file also defines the cost function and the optimizer.
    • This file is used by train_bi_lstm.py of core module.
  3. lstm.py

    • This file consist the architecture of uni-directional lstm.
    • This bi-directional lstm is Model-2.
    • The file also defines the cost function and the optimizer.
    • This file is used by train_lstm.py of core module.
  4. vae.py

    • This file consist the architecture of auto-encoder.
    • This variational auto-encoder is Model-1.
    • The file also defines the cost function and the optimizer.
    • This file is used by train_auto_encoder_1.py of core module.

utils:

  1. constants.py

    • This file consists of constant.py
    • It has defined path to data folder and models
    • Only this file needs to be changed if you want to use a custom path with in the project
  2. cv_utils.py

    • contains all the OPENCV functions that are commonly used by files in the core module.
    • functions like reading frames, converting image to black and white, resizinng video frame.
  3. os_utils.py

    • contains all the os module functions that are commonly used by files in the core module.
    • functions like iteratng a directory, creating a folder, joining paths.
  4. utility.py

    • contains all the functions that are commonly used by files in the core module.
    • functions like freeze_model, prepare_batch_frames_from_bg_data, load_a_frozen_model.