My Visual Piano User Guide

Landing Page

Abstract: According to the ALS Association, an average of 15 people are diagnosed with ALS every day. ALS is just one disabling condition that can impair someone’s creative abilities. My Visual Piano is a real-time, live instrument that provides a way for people in this community to express their creativity exclusively using their eyes. It is accessible to musicians hoping to continue their passion or novices wishing to learn piano in a different way. People who have limited mobility or have completely lost the ability to use their arms, hands, or fingers can play piano live – hands and hassle-free.

Getting Started

Prerequisites

• Device with Windows 10 April 2018 Update
• Supported eye trackers: view here

Development

• Built upon the Gaze Interaction Library to enable eye-tracking on UWP apps
• Universal Windows Application developed in Microsoft Visual Studio

Installation

• Download this app from the Microsoft store: download
• Install locally on machine by cloning repo on machine and running app with Microsoft Visual Studio

App Walkthrough

1. Set up Tobii eye-tracker
2. Start up My Visual Piano
3. Start playing music

Detailed My Visual Piano Guide

Note: All buttons have gaze and touch input enabled. Users with eye trackers select buttons by looking and dwelling on them. This app is not limited to those with eye trackers only and buttons can also be activated by touch.

• Select Play Music button until next screen shows up
• Next, Select a Key by choosing a key the music you want to play is in
• Then select Major or Minor to complete key selection
• Once in the piano page there are multiple buttons: note and mode
  • Note (grey border): Play piano notes
  • Mode (blue border):
    • O- / O+: Shifts octaves down or up from octaves 1 - 7
    • Major Chord / Minor Chord: plays major or minor chords of the note button looked at (After first activating button once it is highlighted in blue)
    • Menu (global navigation button): Opens up menu with options to Toggle Gaze Dot (removes red eye gaze dot), get Help, Record, Change Keys, Resume, go Home, or Exit App.

License

This project is licensed under the GPL License - see the LICENSE file for details