Brain-Grasp: Graph-based Saliency Priors for Improved fMRI-based Visual Brain Decoding
For researchers in brain decoding and image reconstruction, this work offers a lightweight, interpretable method that enhances structural fidelity without multiple diffusion stages.
The paper tackles the challenge of preserving object-level structure and semantic fidelity in fMRI-based visual brain decoding. The proposed saliency-driven framework using graph-informed saliency priors improves conceptual alignment and structural similarity to original stimuli, achieving better reconstruction quality with a single frozen diffusion model.
Recent progress in brain-guided image generation has improved the quality of fMRI-based reconstructions; however, fundamental challenges remain in preserving object-level structure and semantic fidelity. Many existing approaches overlook the spatial arrangement of salient objects, leading to conceptually inconsistent outputs. We propose a saliency-driven decoding framework that employs graph-informed saliency priors to translate structural cues from brain signals into spatial masks. These masks, together with semantic information extracted from embeddings, condition a diffusion model to guide image regeneration, helping preserve object conformity while maintaining natural scene composition. In contrast to pipelines that invoke multiple diffusion stages, our approach relies on a single frozen model, offering a more lightweight yet effective design. Experiments show that this strategy improves both conceptual alignment and structural similarity to the original stimuli, while also introducing a new direction for efficient, interpretable, and structurally grounded brain decoding.