Extreme Image Transformations Affect Humans and Machines Differently
This work addresses the problem of improving ANN robustness to extreme image distortions for researchers in computer vision and neuroscience, though it is incremental as it builds on known differences in human and machine visual processing.
The study introduced novel image transforms based on neurophysiology to evaluate object recognition in humans and artificial neural networks (ANNs), finding that machines outperform humans on some transforms but struggle on others that are easy for humans, with quantified accuracy differences and a difficulty ranking for humans.
Some recent artificial neural networks (ANNs) claim to model aspects of primate neural and human performance data. Their success in object recognition is, however, dependent on exploiting low-level features for solving visual tasks in a way that humans do not. As a result, out-of-distribution or adversarial input is often challenging for ANNs. Humans instead learn abstract patterns and are mostly unaffected by many extreme image distortions. We introduce a set of novel image transforms inspired by neurophysiological findings and evaluate humans and ANNs on an object recognition task. We show that machines perform better than humans for certain transforms and struggle to perform at par with humans on others that are easy for humans. We quantify the differences in accuracy for humans and machines and find a ranking of difficulty for our transforms for human data. We also suggest how certain characteristics of human visual processing can be adapted to improve the performance of ANNs for our difficult-for-machines transforms.