Spiking Neural Networks for fMRI-Based Visual Semantic Decoding
For researchers in fMRI-based visual decoding, this work demonstrates that SNN-derived features provide better brain-decodable representations than ANN features, offering a new direction for improving decoding performance.
This paper shows that using spiking neural network (SNN)-derived visual features as targets for fMRI-based visual semantic decoding outperforms traditional ANN-derived features. On the GoD dataset, SNN features reduce feature-prediction error from 0.7707 to 0.0282 and improve top-1 semantic decoding accuracy from 0.1800 to 0.4400.
Functional magnetic resonance imaging (fMRI)-based visual decoding aims to recover visual information from measured brain activity, commonly by mapping fMRI responses into latent visual features for downstream decoding tasks. Most existing methods learn mappings from fMRI responses to visual features extracted by artificial neural networks (ANNs), yet it remains unclear whether ANN-derived features provide suitable targets for brain decoding. In this study, we investigate spiking neural network (SNN)-derived visual features as alternative targets for fMRI-based visual decoding. We compare an ANN baseline with four SNN variants from the same architectural family, which differ in their spiking dynamics. To isolate the effect of the target features, all models use the same L2-regularized linear fMRI-to-feature decoder, while only the feature vectors used as regression targets are varied. Compared with the ANN baseline, SNN-derived features exhibit stronger alignment with fMRI responses and improve visual semantic decoding performance. For instance, on the GoD dataset, SNN-derived features reduce feature-prediction error from 0.7707 to 0.0282 and improve top-1 semantic decoding accuracy from 0.1800 to 0.4400. Ablation results further indicate that both spiking neural dynamics and temporal simulation steps contribute to the observed advantage. These findings support SNN-derived features as effective brain-decodable visual representations and highlight target feature design as an important component of fMRI-based visual decoding.