Signal Processing + AI project: Bionics

Project/research about processig signals, used for bionic prosthetics development.

Project consists of two parts, presented in the EMG_processing.ipynb:

• I Signal Processing: EMG data processing. concentrated on processing of the raw EMG data.
• II AI: Movement classification. concentrated on classification of EMG signals.

Problem description:

The main goal of the project was to learn how to properly process myoelectric signals and use them in the problem of motion classification, which is one of the main tasks in the development of bionic prosthetics.

The task of EMG processing and motion classification itself is a subtask of a wider field - the development of prosthetics for upper and lower limbs. When using biological signals to control prosthetics, companies and independent developers often choose myoelectric signals: these signals are sent by motor neurons and control muscle contraction. After a limb is amputated, neurons continue to send these signals, and, it is possible to record these signals using EMG sensors and use them to control the prosthesis, which replaced the lost limb.

Therefore, my goal was to process ready myoelectric data and classify it. For classification itself I tried different algorithms to be able to compare them in the end.

Data:

I took the data from the BioPatRec repository. Here is a link to descriptions of their datasets: https://github.com/biopatrec/biopatrec/wiki/Data_Repository.md. Here is a link to the exact dataset that I used in the project: https://github.com/biopatrec/biopatrec/tree/Data_Repository/10mov4chForearmUntargeted. This dataset includes EMG recordings of 20 people. For each person, there are 10 movement recordings available. In total in this data set there are representations of ten basic movements: closing and opening of a hand, turns of a hand, firm grip and side grip, thumbs up, etc. Each movement was repeated by each person three times (for a three-second duration) and with breaks for rest between repetitions (also three-second duration ones). Four bipolar electrodes (each with a diameter of 1 cm and a distance between different electrodes of 2 cm) were connected to the person, so the data is available from four channels. The raw data itself has the following dimensions: Number of samples (36,000 in our case) X Number of channels (4 in our case) X Number of types of movements (10 in our case). Signal sampling frequency: 2 kHz, recording time for each type of movement - 18 seconds. Amplifiers were also used in the measurement, signals were preprocessed using the following filters: high-pass fourth-order filter (at 20 Hz), low-pass second-order filter (at 400 Hz), and also Notch filter (50 Hz). For each person, labels are provided in the same order.

Here is comparison of two processed channels of different signals:

Pipeline:

Here is a brief pipeline of both parts of the project:

1. Preparations:
   1. Downloading an EMG dataset
   2. Reading data from the data set
2. Processing each signal from the dataset:
   1. Removing mean EMG (making average 0 volts)
   2. Rectification (using absolute values of the signal)
   3. Butterworth low-pass filter for envelope
3. Data analysis and visualizations:
   1. Aplitude spectrum
   2. Spectrogram
   3. Signal processing with visualization
   4. Visual comparison of signals for different movements
4. Feature extraction for classification:
   1. Variance per channel
   2. Number of ZCs (Zero crossing) per channel
   3. Maximum value per channel
   4. Mean value per channel
5. Classification:
   1. Creating training and testing datasets (k-folds cross-validation)
   2. Decision Tree training
   3. SVM training
   4. KNN training
   5. Naive Bayes training
   6. Neural Network training
   7. Testing models (accuracy, confusion matrix)

Implementations are more comments and information on each part of the pipeline can be found in EMG_processing.ipynb.

Results:

Table with final results for accuracy:

Here is more detailed (with confusion matrix) comparison of final metrics for different results. With concatenation:

With handcrafted features:

With separate channels as signals:

Conclusion:

The best results were achived by SVM for concatentaion and Decision Tree for feature extraction, but even they could not achieve good enough metrics. The problem might be in the little amount of data used, incorrect features extracted or flaws in my classification architecture in general.

Brief notes for Signal Processing part:

Project consists of two parts: processing EMG signals for Signal Processing course and classifying EMG signals by according hand movements for AI course.

EMG is electromyography, procedure used to obtain myoelectric data. This are recording of signals used by motor neurons to cause muscles to contratc (more information here).

This signals are frequently used in bionic prosthetics development. More information on myoelectric pattern recognition for bionic upper limb prosthetics here.

Using EMG we can predict which movement was meant to be performed by the limb, myoelectric data for which was recorded. This is why EMG is used in prosthetics development. In case of my project, I used dataset 10mov4chForearmUntargeted, which contained data only for the upper limb and had representations of 10 basic movements.

Here are examples of spectrograms of two signals representing different movements (only one channel for each):

I used a small signal processing pipeline in order to get the data ready for further work (for example, classification). Here is a little preview of raw data and the final processed signal:

More information about data, processing pipeline, classification and more illustrations are available in the according jupyter notebook.

Sources/literature:

• EMG data

  1. Myoelectric Pattern Recognition for Bionic Arms & Hands | Bionics For Everyone
  2. Electromyography (EMG)
  3. Electromyogram (EMG)
  4. Examples of Electromyograms
  5. Dynamic Brain Platform - Official Site
  6. EMG Physical Action Data Set
  7. Data_Repository.md · biopatrec/biopatrec Wiki
  8. Ninapro info/data

• Processing EMG with python

  1. Python: Analysing EMG signals

• Regarding EMG feature extraction

  1. Interpreting Deep Learning Features for Myoelectric Control: A Comparison With Handcrafted Features
  2. Cardinality as a highly descriptive feature in myoelectric pattern recognition for decoding motor volition
  3. Myoelectric control of prosthetic hands: state-of-the-art review

• General info about bionic prosthetics

  1. A Complete Guide to Bionic Arms & Hands | Bionics For Everyone
  2. A Complete Guide to Bionic Legs & Feet | Bionics For Everyone
  3. Bionic Leg & Foot Control Systems | Bionics For Everyone
  4. Bionic Hands: Finding the Right Myoelectric Control System | Bionics For Everyone
  5. Bionic Arm & Hand Control Systems | Bionics For Everyone
  6. Bionic Arm & Hand Control Systems (myoelectric-dc) | Bionics For Everyone
  7. how does a bionic arm work - Open Bionics

• Papers

  1. https://www.ijedr.org/papers/IJEDR1403083.pdf
  2. (work of Open Bionics, mentions EMG usage for their prosthetics) https://openbionics.org/EMBC2014_Liarokapis_AffordableProstheticFingers.pdf
  3. (work of Open Bionics, describes how EMG sensors can be used) https://www.minasliarokapis.com/2019_RAL_AdaptiveTendonDrivenWearableExogloves.pdf
  4. (work of Open Bionics, describes EMG-based learning scheme) https://ieeexplore.ieee.org/document/6507235
  5. (also discusses EMG pattern-recognition for bionic hands (even homemade ones)) https://www.sciencedirect.com/science/article/abs/pii/S0208521617300323
  6. (this one also discusses MMG) https://www.researchgate.net/publication/301633829_Analyzing_EMG_and_MMG_signals_for_MMG_driven_bionic_arm
  7. https://www.scirp.org/(S(351jmbntvnsjt1aadkposzje))/reference/referencespapers.aspx?referenceid=3008477
  8. https://pubmed.ncbi.nlm.nih.gov/33321915/
  9. https://www.nature.com/articles/s41598-017-09770-5
  10. EMG Signal Processing for Hand Motion Pattern Recognition Using Machine Learning Algorithms

• Theses

  1. https://www.theseus.fi/bitstream/handle/10024/147196/thesis_susan.pdf?sequence=1&isAllowed=y
  2. https://static1.squarespace.com/static/5fdf30e82dcd53187f20b7f4/t/5fe09c7ef5f64226567c5b9e/1608555676841/Low+Cost+Prosthetic+Arm+Thesis.pdf
  3. https://www.youtube.com/watch?v=CIqzeBxkRws
  4. https://digitalcommons.csbsju.edu/cgi/viewcontent.cgi?article=1052&context=honors_theses

• Other

  1. InMoov - open-source 3D printed life-size robot

Previouslu considered sources:

Possible sources of data (EEG):

• EEG Motor Movement/Imagery Dataset
• article EEG datasets for motor imagery brain computer interface
• Materials from "Decoding finger movement in humans using synergy of EEG cortical current signals" paper. Correspondence and requests for materials should be addressed to N.Y. (email: yoshimura@pi.titech.ac.jp)
• (? possibly they dont have any linkts to data in this paper, currently I dont see one) Materials from "Investigating Neural Representation of Finger-Movement Directions Using Electroencephalography Independent Components" paper.
• Evaluation of EEG Features in Decoding Individual Finger Movements from One Hand
• Brain–computer interface robotics for hand rehabilitation after stroke: a systematic review
• EEG Motor Movement/Imagery Dataset 1.0.0

Credits:

• Daria Omelkina

Special Thanks go to BioPatRec team for making EMG data sets available for free access on the web.