Joint Prediction of Depths, Normals and Surface Curvature from RGB Images using CNNs
This work addresses the need for accurate 3D scene reconstruction for autonomous robots, introducing a novel approach to estimate surface curvature from color images, which is incremental in combining these tasks.
The paper tackles the problem of 3D scene understanding from a single RGB image by jointly estimating depth, surface normals, and surface curvature using a CNN framework, achieving improved performance over previous state-of-the-art benchmarks for depth and normal estimation while also predicting surface curvature.
Understanding the 3D structure of a scene is of vital importance, when it comes to developing fully autonomous robots. To this end, we present a novel deep learning based framework that estimates depth, surface normals and surface curvature by only using a single RGB image. To the best of our knowledge this is the first work to estimate surface curvature from colour using a machine learning approach. Additionally, we demonstrate that by tuning the network to infer well designed features, such as surface curvature, we can achieve improved performance at estimating depth and normals.This indicates that network guidance is still a useful aspect of designing and training a neural network. We run extensive experiments where the network is trained to infer different tasks while the model capacity is kept constant resulting in different feature maps based on the tasks at hand. We outperform the previous state-of-the-art benchmarks which jointly estimate depths and surface normals while predicting surface curvature in parallel.