What Does CNN Shift Invariance Look Like? A Visualization Study
This work addresses the problem of understanding and improving shift invariance in CNN features for researchers and practitioners in computer vision, but it is incremental as it focuses on measurement and visualization without proposing new methods.
The study measured and visualized shift invariance in features extracted from popular CNN models, finding that they are not globally invariant and exhibit biases and artifacts, with anti-aliased models improving local invariance but not global invariance.
Feature extraction with convolutional neural networks (CNNs) is a popular method to represent images for machine learning tasks. These representations seek to capture global image content, and ideally should be independent of geometric transformations. We focus on measuring and visualizing the shift invariance of extracted features from popular off-the-shelf CNN models. We present the results of three experiments comparing representations of millions of images with exhaustively shifted objects, examining both local invariance (within a few pixels) and global invariance (across the image frame). We conclude that features extracted from popular networks are not globally invariant, and that biases and artifacts exist within this variance. Additionally, we determine that anti-aliased models significantly improve local invariance but do not impact global invariance. Finally, we provide a code repository for experiment reproduction, as well as a website to interact with our results at https://jakehlee.github.io/visualize-invariance.