Condition-Invariant fMRI Decoding of Speech Intelligibility with Deep State Space Model
This work addresses the challenge of understanding abstract linguistic representations in the brain for computational neuroscience and digital speech processing, though it is incremental as it extends decoding to new conditions.
The researchers tackled the problem of decoding speech intelligibility from fMRI signals across diverse listening conditions, showing their deep state space model significantly outperforms classical approaches and revealing condition-invariant neural codes in auditory, frontal, and parietal regions.
Clarifying the neural basis of speech intelligibility is critical for computational neuroscience and digital speech processing. Recent neuroimaging studies have shown that intelligibility modulates cortical activity beyond simple acoustics, primarily in the superior temporal and inferior frontal gyri. However, previous studies have been largely confined to clean speech, leaving it unclear whether the brain employs condition-invariant neural codes across diverse listening environments. To address this gap, we propose a novel architecture built upon a deep state space model for decoding intelligibility from fMRI signals, specifically tailored to their high-dimensional temporal structure. We present the first attempt to decode intelligibility across acoustically distinct conditions, showing our method significantly outperforms classical approaches. Furthermore, region-wise analysis highlights contributions from auditory, frontal, and parietal regions, and cross-condition transfer indicates the presence of condition-invariant neural codes, thereby advancing understanding of abstract linguistic representations in the brain.