Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection
This work addresses the need for better representation learning in 3D perception tasks like object detection, offering a domain-specific improvement for applications such as autonomous driving.
The paper tackles the problem of improving 3D object detection in LiDAR data by proposing a spatio-temporal equivariant self-supervision framework that encourages feature equivariance under transformations like translation, scaling, flip, rotation, and scene flow, resulting in outperforming existing equivariant and invariant approaches in many settings.
Popular representation learning methods encourage feature invariance under transformations applied at the input. However, in 3D perception tasks like object localization and segmentation, outputs are naturally equivariant to some transformations, such as rotation. Using pre-training loss functions that encourage equivariance of features under certain transformations provides a strong self-supervision signal while also retaining information of geometric relationships between transformed feature representations. This can enable improved performance in downstream tasks that are equivariant to such transformations. In this paper, we propose a spatio-temporal equivariant learning framework by considering both spatial and temporal augmentations jointly. Our experiments show that the best performance arises with a pre-training approach that encourages equivariance to translation, scaling, and flip, rotation and scene flow. For spatial augmentations, we find that depending on the transformation, either a contrastive objective or an equivariance-by-classification objective yields best results. To leverage real-world object deformations and motion, we consider sequential LiDAR scene pairs and develop a novel 3D scene flow-based equivariance objective that leads to improved performance overall. We show our pre-training method for 3D object detection which outperforms existing equivariant and invariant approaches in many settings.