Visual Language Models show widespread visual deficits on neuropsychological tests
This reveals a critical gap in AI vision systems, highlighting that VLMs can achieve high-level tasks without foundational visual concepts, which is incremental but important for understanding AI limitations.
The study assessed Visual Language Models (VLMs) using neuropsychological tests and found that while they excel at object recognition, they show widespread deficits in low- and mid-level visual abilities, comparable to clinically significant impairments in humans.
Visual Language Models (VLMs) show remarkable performance in visual reasoning tasks, successfully tackling college-level challenges that require high-level understanding of images. However, some recent reports of VLMs struggling to reason about elemental visual concepts like orientation, position, continuity, and occlusion suggest a potential gulf between human and VLM vision. Here we use the toolkit of neuropsychology to systematically assess the capabilities of three state-of-the-art VLMs across visual domains. Using 51 tests drawn from six clinical and experimental batteries, we characterise the visual abilities of leading VLMs relative to normative performance in healthy adults. While the models excel in straightforward object recognition tasks, we find widespread deficits in low- and mid-level visual abilities that would be considered clinically significant in humans. These selective deficits, profiled through validated test batteries, suggest that an artificial system can achieve complex object recognition without developing foundational visual concepts that in humans require no explicit training.