EEG-Driven Image Reconstruction with Saliency-Guided Diffusion Models
This work addresses limitations in neural decoding for applications like medical diagnostics and neuroadaptive interfaces, though it is incremental as it builds on existing diffusion models with added conditioning.
The paper tackled the problem of EEG-driven image reconstruction by incorporating spatial attention mechanisms to improve fidelity and semantic coherence, achieving significant improvements in low- and high-level image feature quality over existing methods on the THINGS-EEG dataset.
Existing EEG-driven image reconstruction methods often overlook spatial attention mechanisms, limiting fidelity and semantic coherence. To address this, we propose a dual-conditioning framework that combines EEG embeddings with spatial saliency maps to enhance image generation. Our approach leverages the Adaptive Thinking Mapper (ATM) for EEG feature extraction and fine-tunes Stable Diffusion 2.1 via Low-Rank Adaptation (LoRA) to align neural signals with visual semantics, while a ControlNet branch conditions generation on saliency maps for spatial control. Evaluated on THINGS-EEG, our method achieves a significant improvement in the quality of low- and high-level image features over existing approaches. Simultaneously, strongly aligning with human visual attention. The results demonstrate that attentional priors resolve EEG ambiguities, enabling high-fidelity reconstructions with applications in medical diagnostics and neuroadaptive interfaces, advancing neural decoding through efficient adaptation of pre-trained diffusion models.