Reconstructing Speech Stimuli From Human Auditory Cortex Activity Using a WaveNet Approach
This work addresses speech decoding for neuroscience and brain-computer interfaces, but it is incremental as it builds on prior knowledge of phonetic representations in the STG.
The study tackled the problem of reconstructing speech stimuli from human auditory cortex activity, achieving this using a WaveNet model on limited intracranial recordings from the superior temporal gyrus, and discovered phoneme-level tuning properties in the electrodes.
The superior temporal gyrus (STG) region of cortex critically contributes to speech recognition. In this work, we show that a proposed WaveNet, with limited available data, is able to reconstruct speech stimuli from STG intracranial recordings. We further investigate the impulse response of the fitted model for each recording electrode and observe phoneme level temporospectral tuning properties for the recorded area of cortex. This discovery is consistent with previous studies implicating the posterior STG (pSTG) in a phonetic representation of speech and provides detailed acoustic features that certain electrode sites possibly extract during speech recognition.