BEVPose: Unveiling Scene Semantics through Pose-Guided Multi-Modal BEV Alignment
This addresses the challenge of data scarcity in diverse environments like off-road or indoor settings for autonomous driving and robotics, though it is incremental as it builds on existing BEV methods.
The paper tackles the problem of data inefficiency in learning Bird's Eye View (BEV) representations for autonomous driving by introducing BEVPose, a framework that uses sensor pose as a supervisory signal to align camera and lidar data, reducing reliance on annotated data and outperforming fully-supervised state-of-the-art methods in BEV map segmentation tasks.
In the field of autonomous driving and mobile robotics, there has been a significant shift in the methods used to create Bird's Eye View (BEV) representations. This shift is characterised by using transformers and learning to fuse measurements from disparate vision sensors, mainly lidar and cameras, into a 2D planar ground-based representation. However, these learning-based methods for creating such maps often rely heavily on extensive annotated data, presenting notable challenges, particularly in diverse or non-urban environments where large-scale datasets are scarce. In this work, we present BEVPose, a framework that integrates BEV representations from camera and lidar data, using sensor pose as a guiding supervisory signal. This method notably reduces the dependence on costly annotated data. By leveraging pose information, we align and fuse multi-modal sensory inputs, facilitating the learning of latent BEV embeddings that capture both geometric and semantic aspects of the environment. Our pretraining approach demonstrates promising performance in BEV map segmentation tasks, outperforming fully-supervised state-of-the-art methods, while necessitating only a minimal amount of annotated data. This development not only confronts the challenge of data efficiency in BEV representation learning but also broadens the potential for such techniques in a variety of domains, including off-road and indoor environments.