Foveated Retinotopy Improves Classification and Localization in CNNs
This work addresses the visual search problem in machine learning, offering an efficient solution inspired by biological vision, though it is incremental as it modifies existing CNN architectures.
The study tackled the problem of improving image classification and object localization in CNNs by incorporating foveated retinotopy, a biological vision mechanism, resulting in maintained classification accuracy with enhanced robustness to scale and rotational perturbations, and enabling object localization through classification probability variations across gaze positions.
From a falcon detecting prey to humans recognizing faces, many species exhibit extraordinary abilities in rapid visual localization and classification. These are made possible by a specialized retinal region called the fovea, which provides high acuity at the center of vision while maintaining lower resolution in the periphery. This distinctive spatial organization, preserved along the early visual pathway through retinotopic mapping, is fundamental to biological vision, yet remains largely unexplored in machine learning. Our study investigates how incorporating foveated retinotopy may benefit deep convolutional neural networks (CNNs) in image classification tasks. By implementing a foveated retinotopic transformation in the input layer of standard ResNet models and re-training them, we maintain comparable classification accuracy while enhancing the network's robustness to scale and rotational perturbations. Although this architectural modification introduces increased sensitivity to fixation point shifts, we demonstrate how this apparent limitation becomes advantageous: variations in classification probabilities across different gaze positions serve as effective indicators for object localization. Our findings suggest that foveated retinotopic mapping encodes implicit knowledge about visual object geometry, offering an efficient solution to the visual search problem - a capability crucial for many living species.