EEGxpecto-Patronum

EEGxpecto-Patronum = EEG + Expecto Patronum

Use Electroencephalography (EEG) to simulate Patronus Charm.

2017 Biomedical Engineering Final Project with S. T. Wu and Z. R. Wu.

Description

As a pure, protective magical concentration of happiness and hope (the recollection of a single talisman memory is essential in its creation). ---J.K. Rowling

According to Harry Potter, the Patronus is highly related to mental strength. Thus, we combined EEG signals with the spell and here we are.

EEGxpecto-Patronum.

Hotword detection is used to trigger the magic process when the spell is spoken, and also to determine who is spelling. Then, it takes EEG signals into account in order to measure the mental strength (attention). Finally, a speller-specific Patronus is shown variously depending on the mental strength.

Requirement

EEG signals

Retrieved from EEG headset, for instance, Mindwave Mobile. Checkout mindwave.ino for collecting EEG signals from Mindwave Mobile thru bluetooth using arduino and HC-05 bluetooth module.

Dependencies

• snowboy: DNN based hotword and wake word detection toolkit.
• opencv3: Open Source Computer Vision Library.
• pyserial: Python serial port access library.
• scipy
• numpy

Usage

capture.js

1. Capture patronus videos from pottermore.
2. Convert video from format .webm to .mp4. For instance,

ffmpeg -i input.webm output.mp4

3. Place the .mp4 video files under directory video/.

train_hotword.py

1. Train your personal hotword, or Expecto Patronum, detection model. For detailed instructions, refer to snowboy document.
2. Place the .pmdl model file under directory model/.

main.py

1. Set up EEG signals connection. Here we use pyserial to read EEG data from arduino. Feel free to replace it with other signal sources.
2. Shout out the spell EXPECTO PATRONUM!
3. Enjoy your magic trip~