Learning with 3D rotations, a hitchhiker's guide to SO(3)
This is an incremental survey that addresses a domain-specific problem for researchers and practitioners in machine learning working with 3D rotations.
The paper tackles the challenge of selecting suitable 3D rotation representations for machine learning by providing a survey and guide that consolidates insights from rotation-based learning, offering guidance based on whether rotations are in the input or output and the data's angle characteristics.
Many settings in machine learning require the selection of a rotation representation. However, choosing a suitable representation from the many available options is challenging. This paper acts as a survey and guide through rotation representations. We walk through their properties that harm or benefit deep learning with gradient-based optimization. By consolidating insights from rotation-based learning, we provide a comprehensive overview of learning functions with rotation representations. We provide guidance on selecting representations based on whether rotations are in the model's input or output and whether the data primarily comprises small angles.