MSDPN: Monocular Depth Prediction with Partial Laser Observation using Multi-stage Neural Networks
This work addresses depth prediction for robotics or autonomous systems using realistic sensor data, but it is incremental as it builds on existing multi-stage architectures with a new feature aggregation technique.
The authors tackled the problem of predicting dense depth maps from a monocular camera and a 2D LiDAR, which suffers from partial observations, by proposing MSDPN, a multi-stage neural network with Cross Stage Feature Aggregation, and demonstrated promising performance against state-of-the-art methods on their physically-collected KAIST RGBD-scan dataset.
In this study, a deep-learning-based multi-stage network architecture called Multi-Stage Depth Prediction Network (MSDPN) is proposed to predict a dense depth map using a 2D LiDAR and a monocular camera. Our proposed network consists of a multi-stage encoder-decoder architecture and Cross Stage Feature Aggregation (CSFA). The proposed multi-stage encoder-decoder architecture alleviates the partial observation problem caused by the characteristics of a 2D LiDAR, and CSFA prevents the multi-stage network from diluting the features and allows the network to learn the inter-spatial relationship between features better. Previous works use sub-sampled data from the ground truth as an input rather than actual 2D LiDAR data. In contrast, our approach trains the model and conducts experiments with a physically-collected 2D LiDAR dataset. To this end, we acquired our own dataset called KAIST RGBD-scan dataset and validated the effectiveness and the robustness of MSDPN under realistic conditions. As verified experimentally, our network yields promising performance against state-of-the-art methods. Additionally, we analyzed the performance of different input methods and confirmed that the reference depth map is robust in untrained scenarios.