Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding
This enables personalized, emotion-aware telepresence and cognitive interaction in immersive environments, representing a new paradigm for neural-driven avatars.
The paper tackles the problem of generating photorealistic facial expressions from EEG signals, presenting Mind-to-Face, a framework that decodes EEG into high-fidelity avatars with over 65k vertices, demonstrating reliable prediction of dynamic, subject-specific expressions.
Current expressive avatar systems rely heavily on visual cues, failing when faces are occluded or when emotions remain internal. We present Mind-to-Face, the first framework that decodes non-invasive electroencephalogram (EEG) signals directly into high-fidelity facial expressions. We build a dual-modality recording setup to obtain synchronized EEG and multi-view facial video during emotion-eliciting stimuli, enabling precise supervision for neural-to-visual learning. Our model uses a CNN-Transformer encoder to map EEG signals into dense 3D position maps, capable of sampling over 65k vertices, capturing fine-scale geometry and subtle emotional dynamics, and renders them through a modified 3D Gaussian Splatting pipeline for photorealistic, view-consistent results. Through extensive evaluation, we show that EEG alone can reliably predict dynamic, subject-specific facial expressions, including subtle emotional responses, demonstrating that neural signals contain far richer affective and geometric information than previously assumed. Mind-to-Face establishes a new paradigm for neural-driven avatars, enabling personalized, emotion-aware telepresence and cognitive interaction in immersive environments.