Parsing Geometry Using Structure-Aware Shape Templates
This work addresses the challenge of scene understanding for AR and robotics applications by enabling structure-aware parsing from incomplete sensor data, representing an incremental improvement through template-based transfer.
The paper tackles the problem of inferring structural information from partial, noisy RGB-D scans of man-made objects by organizing shape collections into parameterized templates and using a deep neural network to match and fit scans to these templates, resulting in faithful part labeling and guided reconstruction as demonstrated through comparisons to state-of-the-art approaches.
Real-life man-made objects often exhibit strong and easily-identifiable structure, as a direct result of their design or their intended functionality. Structure typically appears in the form of individual parts and their arrangement. Knowing about object structure can be an important cue for object recognition and scene understanding - a key goal for various AR and robotics applications. However, commodity RGB-D sensors used in these scenarios only produce raw, unorganized point clouds, without structural information about the captured scene. Moreover, the generated data is commonly partial and susceptible to artifacts and noise, which makes inferring the structure of scanned objects challenging. In this paper, we organize large shape collections into parameterized shape templates to capture the underlying structure of the objects. The templates allow us to transfer the structural information onto new objects and incomplete scans. We employ a deep neural network that matches the partial scan with one of the shape templates, then match and fit it to complete and detailed models from the collection. This allows us to faithfully label its parts and to guide the reconstruction of the scanned object. We showcase the effectiveness of our method by comparing it to other state-of-the-art approaches.