Deep Learning for 2D and 3D Rotatable Data: An Overview of Methods
It synthesizes existing research for practitioners and researchers working with 2D and 3D data, but is incremental as it reviews rather than introduces novel methods.
This paper provides an overview of methods for handling rotatable data in deep learning, addressing the need for equivariance/invariance under rotations in tasks like object classification and motion estimation, without presenting new experimental results.
Convolutional networks are successful due to their equivariance/invariance under translations. However, rotatable data such as images, volumes, shapes, or point clouds require processing with equivariance/invariance under rotations in cases where the rotational orientation of the coordinate system does not affect the meaning of the data (e.g. object classification). On the other hand, estimation/processing of rotations is necessary in cases where rotations are important (e.g. motion estimation). There has been recent progress in methods and theory in all these regards. Here we provide an overview of existing methods, both for 2D and 3D rotations (and translations), and identify commonalities and links between them.