MORE | Spring 2019
Video Motion Magnification to Enhance Speech of Patients with Deteriorated Speaking Ability
The project aims at using degraded speech as well as visual information to improve speech generation. ALS and Parkinson’s are one of the more dreadful diseases where people lose the ability to move their muscles leading to poor articulation and respiration, as well as slurred and slowed speech which may be hard to understand. As a future work, Deep Neural network models will be trained to recover speech from vibrations of objects like plastic bag with High Frame-rate video as input and the Speech signal as the label. The final goal will be to train a model which takes the video of facial vibrations and synthesize speech from it.