Learning to Reconstruct and Segment 3D Objects
This work addresses the fundamental AI challenge of 3D perception for machines, potentially benefiting robotics and computer vision applications, but it appears incremental as it builds on existing deep learning approaches.
The thesis tackles the problem of 3D object reconstruction and segmentation from visual inputs like images or point clouds, aiming to overcome limitations of traditional hand-crafted features by using deep neural networks trained on large-scale 3D data, with contributions ranging from object-level shape estimation to scene-level semantic understanding.
To endow machines with the ability to perceive the real-world in a three dimensional representation as we do as humans is a fundamental and long-standing topic in Artificial Intelligence. Given different types of visual inputs such as images or point clouds acquired by 2D/3D sensors, one important goal is to understand the geometric structure and semantics of the 3D environment. Traditional approaches usually leverage hand-crafted features to estimate the shape and semantics of objects or scenes. However, they are difficult to generalize to novel objects and scenarios, and struggle to overcome critical issues caused by visual occlusions. By contrast, we aim to understand scenes and the objects within them by learning general and robust representations using deep neural networks, trained on large-scale real-world 3D data. To achieve these aims, this thesis makes three core contributions from object-level 3D shape estimation from single or multiple views to scene-level semantic understanding.