ZEBRA: Towards Zero-Shot Cross-Subject Generalization for Universal Brain Visual Decoding
This work addresses scalability and real-world applicability in brain visual decoding for neuroscience and computer vision, representing a practical step toward universal neural decoding.
The paper tackles the problem of subject-specific adaptation in fMRI-to-image reconstruction by introducing ZEBRA, a zero-shot framework that disentangles subject-related and semantic-related components, achieving performance comparable to fully fine-tuned models on several metrics.
Recent advances in neural decoding have enabled the reconstruction of visual experiences from brain activity, positioning fMRI-to-image reconstruction as a promising bridge between neuroscience and computer vision. However, current methods predominantly rely on subject-specific models or require subject-specific fine-tuning, limiting their scalability and real-world applicability. In this work, we introduce ZEBRA, the first zero-shot brain visual decoding framework that eliminates the need for subject-specific adaptation. ZEBRA is built on the key insight that fMRI representations can be decomposed into subject-related and semantic-related components. By leveraging adversarial training, our method explicitly disentangles these components to isolate subject-invariant, semantic-specific representations. This disentanglement allows ZEBRA to generalize to unseen subjects without any additional fMRI data or retraining. Extensive experiments show that ZEBRA significantly outperforms zero-shot baselines and achieves performance comparable to fully finetuned models on several metrics. Our work represents a scalable and practical step toward universal neural decoding. Code and model weights are available at: https://github.com/xmed-lab/ZEBRA.