Multigranular Evaluation for Brain Visual Decoding
This work addresses the need for better evaluation protocols in brain visual decoding research, offering a more interpretable foundation for comparing models, though it is incremental as it focuses on improving metrics rather than proposing a new decoding method.
The paper tackles the problem of evaluating brain visual decoding methods by introducing BASIC, a multigranular framework that quantifies structural fidelity, inferential alignment, and contextual coherence, providing a more discriminative and comprehensive evaluation across multiple datasets.
Existing evaluation protocols for brain visual decoding predominantly rely on coarse metrics that obscure inter-model differences, lack neuroscientific foundation, and fail to capture fine-grained visual distinctions. To address these limitations, we introduce BASIC, a unified, multigranular evaluation framework that jointly quantifies structural fidelity, inferential alignment, and contextual coherence between decoded and ground truth images. For the structural level, we introduce a hierarchical suite of segmentation-based metrics, including foreground, semantic, instance, and component masks, anchored in granularity-aware correspondence across mask structures. For the semantic level, we extract structured scene representations encompassing objects, attributes, and relationships using multimodal large language models, enabling detailed, scalable, and context-rich comparisons with ground-truth stimuli. We benchmark a diverse set of visual decoding methods across multiple stimulus-neuroimaging datasets within this unified evaluation framework. Together, these criteria provide a more discriminative, interpretable, and comprehensive foundation for measuring brain visual decoding methods.