Efficient Rotation Invariance in Deep Neural Networks through Artificial Mental Rotation
This addresses the challenge of rotation invariance in computer vision, which is crucial for real-world applications like robotics and autonomous driving, by providing a significant performance gain over existing methods.
The paper tackles the problem of poor performance of CNNs and vision transformers on rotated inputs by introducing artificial mental rotation (AMR), a novel deep learning paradigm inspired by mental rotation. It achieves a 19% improvement in top-1 error over state-of-the-art methods on average across datasets and architectures, and boosts IoU from 32.7 to 55.2 on a semantic segmentation task.
Humans and animals recognize objects irrespective of the beholder's point of view, which may drastically change their appearances. Artificial pattern recognizers also strive to achieve this, e.g., through translational invariance in convolutional neural networks (CNNs). However, both CNNs and vision transformers (ViTs) perform very poorly on rotated inputs. Here we present artificial mental rotation (AMR), a novel deep learning paradigm for dealing with in-plane rotations inspired by the neuro-psychological concept of mental rotation. Our simple AMR implementation works with all common CNN and ViT architectures. We test it on ImageNet, Stanford Cars, and Oxford Pet. With a top-1 error (averaged across datasets and architectures) of $0.743$, AMR outperforms the current state of the art (rotational data augmentation, average top-1 error of $0.626$) by $19\%$. We also easily transfer a trained AMR module to a downstream task to improve the performance of a pre-trained semantic segmentation model on rotated CoCo from $32.7$ to $55.2$ IoU.