Decoding Predictive Inference in Visual Language Processing via Spatiotemporal Neural Coherence
This work addresses how the brain processes visual language, offering insights for neuroscience and AI, but it is incremental as it builds on existing predictive inference models.
The study tackled the problem of decoding neural responses to visual language in Deaf signers by using coherence between EEG signals and motion features, revealing that left-hemispheric and frontal low-frequency coherence are key for language comprehension and correlate with age.
Human language processing relies on the brain's capacity for predictive inference. We present a machine learning framework for decoding neural (EEG) responses to dynamic visual language stimuli in Deaf signers. Using coherence between neural signals and optical flow-derived motion features, we construct spatiotemporal representations of predictive neural dynamics. Through entropy-based feature selection, we identify frequency-specific neural signatures that differentiate interpretable linguistic input from linguistically disrupted (time-reversed) stimuli. Our results reveal distributed left-hemispheric and frontal low-frequency coherence as key features in language comprehension, with experience-dependent neural signatures correlating with age. This work demonstrates a novel multimodal approach for probing experience-driven generative models of perception in the brain.