“Silent Speech and Emotion Recognition from Vocal Tract Shape Dynamics in Real-Time MRI” by Pandey and Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

  • ©Laxmi Pandey and Ahmed Sabbir Arif

Conference:


Type(s):


Entry Number: 43

Title:

    Silent Speech and Emotion Recognition from Vocal Tract Shape Dynamics in Real-Time MRI

Presenter(s)/Author(s):



Abstract:


    We propose a novel deep neural network-based learning framework that understands acoustic information in the variable-length sequence of vocal tract shaping during speech production, captured by real-time magnetic resonance imaging (rtMRI), and translate it into text. In an experiment, it achieved a 40.6% PER at sentence-level, much better compared to the existing models. We also performed an analysis of variations in the geometry of articulation in each sub-regions of the vocal tract with respect to different emotions and genders. Results suggest that each sub-regions distortion is affected by both emotion and gender.

Keyword(s):



PDF:



ACM Digital Library Publication:



Overview Page: