EVA: Bridging Performance and Human Alignment in Hard-Attention Vision Models for Image Classification
For researchers in interpretable AI and active vision, EVA provides a principled framework to reduce the alignment tax between performance and human-likeness, though the improvements are incremental over existing methods.
EVA introduces a hard-attention vision model that balances classification accuracy with human-like scanpath alignment, achieving improved alignment metrics (DTW, NSS) on CIFAR-10 with competitive accuracy, and demonstrating zero-shot human-like scanpaths on COCO-Search18.
Optimizing vision models purely for classification accuracy can impose an alignment tax, degrading human-like scanpaths and limiting interpretability. We introduce EVA, a neuroscience-inspired hard-attention mechanistic testbed that makes the performance-human-likeness trade-off explicit and adjustable. EVA samples a small number of sequential glimpses using a minimal fovea-periphery representation with CNN-based feature extractor and integrates variance control and adaptive gating to stabilize and regulate attention dynamics. EVA is trained with the standard classification objective without gaze supervision. On CIFAR-10 with dense human gaze annotations, EVA improves scanpath alignment under established metrics such as DTW, NSS, while maintaining competitive accuracy. Ablations show that CNN-based feature extraction drives accuracy but suppresses human-likeness, whereas variance control and gating restore human-aligned trajectories with minimal performance loss. We further validate EVA's scalability on ImageNet-100 and evaluate scanpath alignment on COCO-Search18 without COCO-Search18 gaze supervision or finetuning, where EVA yields human-like scanpaths on natural scenes without additional training. Overall, EVA provides a principled framework for trustworthy, human-interpretable active vision.