Skip to content

Kestrel is a TensorFlow-powered American Sign Language fingerspelling translator Android app that serves as a convenient way to learn and understand fingerspelling signs

License

Notifications You must be signed in to change notification settings

WenzelArifiandi/kestrel

Repository files navigation

Kestrel

A Bangkit 2021 Capstone Project

Get it on Google Play

Kestrel is a TensorFlow powered American Sign Language fingerspelling translator Android app that serves as a convenient way to learn and understand fingerspelling signs. The Kestrel model builds upon the state of the art MobileNetV2 model that is optimized for speed and latency on mobile devices to accurately recognize and interpret sign language from the phone’s camera with a 96.8% testing accuracy (99.60% training accuracy, 98.66% validation accuracy) and display its translation through a beautiful, convenient and easily accessible Android app.

Thesis Research [v2.0]

Kestrel is now a part of my thesis research. The app is updated with:

  1. A brand new simplified user interface that instantly shows the viewfinder's live view and the top 5 fingerspelling results.
  2. A new MobileNetV2 model trained on a new dataset (Barczak et al., 2011), with 50% more detection categories (36 labels containing 26 alphabets and ten numbers).

Open In Colab

Now available on Google Play

Get it on Google Play

Thesis Documents

English - Machine Translated Copy
Bahasa Indonesia - Original Copy

Updated User Interface

Updated UI Screenshots

Bangkit [v1.0-alpha]

The Kestrel model (95.2% testing accuracy, 98.16% training accuracy and 95.3% validation accuracy) is trained on 65.574 color images (comprising 24 static alphabet signs) from the American Sign Language FingerSpelling Dataset published by Nicolas Pugeault and Richard Bowden on the 2011 IEEE International Conference on Computer Vision Workshops.

User Interface

Screenshots

Accuracy and Loss Graph

Accuracy and Loss Graph

Colab

Open In Colab

Google Play and the Google Play logo are trademarks of Google LLC.

About

Kestrel is a TensorFlow-powered American Sign Language fingerspelling translator Android app that serves as a convenient way to learn and understand fingerspelling signs

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages