MORE | Spring 2019

Video Motion Magnification to Enhance Speech of Patients with Deteriorated Speaking Ability

Health icon, disabled. A red heart with a cardiac rhythm running through it.

The project aims at using degraded speech as well as visual information to improve speech generation. ALS and Parkinson’s are one of the more dreadful diseases where people lose the ability to move their muscles leading to poor articulation and respiration, as well as slurred and slowed speech which may be hard to understand. As a future work, Deep Neural network models will be trained to recover speech from vibrations of objects like plastic bag with High Frame-rate video as input and the Speech signal as the label. The final goal will be to train a model which takes the video of facial vibrations and synthesize speech from it.

Student researcher

Mazher Ali Khan

Electrical engineering

Hometown: Konanur, Karnataka, India

Graduation date: Fall 2019