CNNs and Transformers Perceive Hybrid Images Similar to Humans
This provides incremental evidence for modeling the human visual system's feedforward processing with deep learning, relevant to neuroscience and computer vision researchers.
The study investigated whether deep learning vision models perceive hybrid images similarly to humans, using 63,000 images across 10 fruit categories, and found that CNN and Transformer predictions qualitatively match human perception.
Hybrid images is a technique to generate images with two interpretations that change as a function of viewing distance. It has been utilized to study multiscale processing of images by the human visual system. Using 63,000 hybrid images across 10 fruit categories, here we show that predictions of deep learning vision models qualitatively matches with the human perception of these images. Our results provide yet another evidence in support of the hypothesis that Convolutional Neural Networks (CNNs) and Transformers are good at modeling the feedforward sweep of information in the ventral stream of visual cortex. Code and data is available at https://github.com/aliborji/hybrid_images.git.