Human perception in computer vision
This work addresses the problem of understanding biological vision for AI researchers and neuroscientists, but it is incremental as it builds on existing comparisons between DNNs and human perception.
The study investigated whether human visual perception correlates with deep neural network (DNN) computations, finding that perceptual sensitivity to image changes correlates with DNN mid-computations, while sensitivity to segmentation, crowding, and shape correlates with DNN end-computations.
Computer vision has made remarkable progress in recent years. Deep neural network (DNN) models optimized to identify objects in images exhibit unprecedented task-trained accuracy and, remarkably, some generalization ability: new visual problems can now be solved more easily based on previous learning. Biological vision (learned in life and through evolution) is also accurate and general-purpose. Is it possible that these different learning regimes converge to similar problem-dependent optimal computations? We therefore asked whether the human system-level computation of visual perception has DNN correlates and considered several anecdotal test cases. We found that perceptual sensitivity to image changes has DNN mid-computation correlates, while sensitivity to segmentation, crowding and shape has DNN end-computation correlates. Our results quantify the applicability of using DNN computation to estimate perceptual loss, and are consistent with the fascinating theoretical view that properties of human perception are a consequence of architecture-independent visual learning.