A Convolutional Framework for Mapping Imagined Auditory MEG into Listened Brain Responses
This provides a foundation for brain-computer interfaces involving imagined speech and music, though it is incremental as it builds on existing mapping techniques.
The study tackled decoding imagined speech from MEG data by mapping imagined brain responses to listened responses, showing that a convolutional neural network with subject-specific calibration achieved significantly higher correlations than a null model for nearly all held-out subjects.
Decoding imagined speech engages complex neural processes that are difficult to interpret due to uncertainty in timing and the limited availability of imagined-response datasets. In this study, we present a Magnetoencephalography (MEG) dataset collected from trained musicians as they imagined and listened to musical and poetic stimuli. We show that both imagined and perceived brain responses contain consistent, condition-specific information. Using a sliding-window ridge regression model, we first mapped imagined responses to listened responses at the single-subject level, but found limited generalization across subjects. At the group level, we developed an encoder-decoder convolutional neural network with a subject-specific calibration layer that produced stable and generalizable mappings. The CNN consistently outperformed the null model, yielding significantly higher correlations between predicted and true listened responses for nearly all held-out subjects. Our findings demonstrate that imagined neural activity can be transformed into perception-like responses, providing a foundation for future brain-computer interface applications involving imagined speech and music.