Harshavardhana T. Gowda

Harshavardhana T. Gowda, Lee M. Miller

$\textit{Objective.}$ In this article, we present data and methods for decoding hand gestures using surface electromyogram (EMG) signals. EMG-based upper limb interfaces are valuable for amputee rehabilitation, artificial supernumerary limb augmentation, gestural control of computers, and virtual and augmented reality applications. $\textit{Approach.}$ To achieve this, we collect EMG signals from the upper limb using surface electrodes placed at key muscle sites involved in hand movements. Additionally, we design and evaluate efficient models for decoding EMG signals. $\textit{Main results.}$ Our findings reveal that the manifold of symmetric positive definite (SPD) matrices serves as an effective embedding space for EMG signals. Moreover, for the first time, we quantify the distribution shift of these signals across individuals. $\textit{Significance.}$ Overall, our approach demonstrates significant potential for developing efficient and interpretable methods for decoding EMG signals. This is particularly important as we move toward the broader adoption of EMG-based wrist interfaces.

Harshavardhana T. Gowda, Zachary D. McNaughton, Lee M. Miller

Objective. In this article, we present data and methods for decoding speech articulations using surface electromyogram (EMG) signals. EMG-based speech neuroprostheses offer a promising approach for restoring audible speech in individuals who have lost the ability to speak intelligibly due to laryngectomy, neuromuscular diseases, stroke, or trauma-induced damage (e.g., from radiotherapy) to the speech articulators. Approach. To achieve this, we collect EMG signals from the face, jaw, and neck as subjects articulate speech, and we perform EMG-to-speech translation. Main results. Our findings reveal that the manifold of symmetric positive definite (SPD) matrices serves as a natural embedding space for EMG signals. Specifically, we provide an algebraic interpretation of the manifold-valued EMG data using linear transformations, and we analyze and quantify distribution shifts in EMG signals across individuals. Significance. Overall, our approach demonstrates significant potential for developing neural networks that are both data- and parameter-efficient, an important consideration for EMG-based systems, which face challenges in large-scale data collection and operate under limited computational resources on embedded devices.

Harshavardhana T. Gowda, Lee M. Miller

We present a neuromuscular speech interface that translates electromyographic (EMG) signals collected from orofacial muscles during speech articulation directly into audio. We show that self-supervised speech (SS) representations exhibit a strong linear relationship with the electrical power of muscle action potentials: SS features can be linearly mapped to EMG power with a correlation of $r = 0.85$. Moreover, EMG power vectors corresponding to different articulatory gestures form structured and separable clusters in feature space. This relationship: $\text{SS features}$ $\xrightarrow{\texttt{linear mapping}}$ $\text{EMG power}$ $\xrightarrow{\texttt{gesture-specific clustering}}$ $\text{articulatory movements}$, highlights that SS models implicitly encode articulatory mechanisms. Leveraging this property, we directly map EMG signals to SS feature space and synthesize speech, enabling end-to-end EMG-to-speech generation without explicit articulatory models and vocoder training.