Cross-Subject Mind Decoding from Inaccurate Representations
This addresses the challenge of cross-subject variability in fMRI-based mind decoding, which is important for neuroscience and brain-computer interfaces, though it appears incremental as it builds on existing generative models like Stable Diffusion.
The paper tackles the problem of cross-subject mind decoding from fMRI signals, where existing methods suffer from error accumulation due to inaccurate representations, and proposes a Bidirectional Autoencoder Intertwining framework that integrates subject bias modulation and refinement modules to improve reconstruction fidelity. The method outperforms state-of-the-art approaches on benchmark datasets and shows strong adaptability to new subjects with minimal training samples.
Decoding stimulus images from fMRI signals has advanced with pre-trained generative models. However, existing methods struggle with cross-subject mappings due to cognitive variability and subject-specific differences. This challenge arises from sequential errors, where unidirectional mappings generate partially inaccurate representations that, when fed into diffusion models, accumulate errors and degrade reconstruction fidelity. To address this, we propose the Bidirectional Autoencoder Intertwining framework for accurate decoded representation prediction. Our approach unifies multiple subjects through a Subject Bias Modulation Module while leveraging bidirectional mapping to better capture data distributions for precise representation prediction. To further enhance fidelity when decoding representations into stimulus images, we introduce a Semantic Refinement Module to improve semantic representations and a Visual Coherence Module to mitigate the effects of inaccurate visual representations. Integrated with ControlNet and Stable Diffusion, our method outperforms state-of-the-art approaches on benchmark datasets in both qualitative and quantitative evaluations. Moreover, our framework exhibits strong adaptability to new subjects with minimal training samples.