Cross-Subject Semantic Decoding with Shared-Space Alignment for Generalized Neural Representation Learning
For researchers working on brain-computer interfaces and neural decoding, this work provides a method to generalize across subjects without retraining, addressing a key bottleneck in invasive neural recordings.
The paper proposes a cross-subject semantic decoding framework that aligns neural responses from multiple subjects into a shared latent space using the shared response model, then decodes contextual semantic embeddings. The method consistently outperforms baselines and reduces performance drop from source to held-out subjects, demonstrating improved cross-subject generalization.
Generalizing across subjects remains challenging in invasive neural recordings because electrode configurations, anatomical structures, and neural signal patterns vary substantially across individuals. To investigate such inter-subject variability, we propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space and learns a mapping from the aligned neural representations to contextual embeddings. More specifically, using electrocorticography data collected during natural language comprehension, we estimate the shared space using the shared response model and train a decoder to predict contextual semantic embeddings from projected neural responses. For a held-out subject, we estimate a subject-specific projection into the predefined shared space, and directly apply the pretrained decoder without any retraining. Experimental results demonstrate that the proposed framework consistently outperforms baseline methods across evaluation settings and exhibits a reduced performance drop from source subject to held-out subject testing, indicating improved cross-subject generalization. These results suggest that aligning neural activity into a shared latent space, while decoding in a semantic embedding space, provides an effective strategy for improving cross-subject generalization by reducing subject-specific differences in neural responses while effectively capturing shared stimulus-related representations.