Sparse SPN: Depth Completion from Sparse Keypoints
This work addresses a specific bottleneck in 3D modeling from sparse point clouds for computer vision applications, but appears incremental in nature.
The paper tackles the problem of depth completion from unevenly distributed sparse keypoints, which existing methods perform poorly on, by extending CSPN with multiscale prediction and a dilated kernel to achieve better completion results.
Our long term goal is to use image-based depth completion to quickly create 3D models from sparse point clouds, e.g. from SfM or SLAM. Much progress has been made in depth completion. However, most current works assume well distributed samples of known depth, e.g. Lidar or random uniform sampling, and perform poorly on uneven samples, such as from keypoints, due to the large unsampled regions. To address this problem, we extend CSPN with multiscale prediction and a dilated kernel, leading to much better completion of keypoint-sampled depth. We also show that a model trained on NYUv2 creates surprisingly good point clouds on ETH3D by completing sparse SfM points.