RICO: Regularizing the Unobservable for Indoor Compositional Reconstruction
This work addresses a specific bottleneck in scene editing and manipulation for indoor environments, representing an incremental improvement over existing methods.
The paper tackles the problem of performance degradation in neural implicit surface methods for indoor compositional reconstruction when objects are partially observed, and proposes RICO to regularize unobservable regions, achieving state-of-the-art results on synthetic and real-world indoor scenes.
Recently, neural implicit surfaces have become popular for multi-view reconstruction. To facilitate practical applications like scene editing and manipulation, some works extend the framework with semantic masks input for the object-compositional reconstruction rather than the holistic perspective. Though achieving plausible disentanglement, the performance drops significantly when processing the indoor scenes where objects are usually partially observed. We propose RICO to address this by regularizing the unobservable regions for indoor compositional reconstruction. Our key idea is to first regularize the smoothness of the occluded background, which then in turn guides the foreground object reconstruction in unobservable regions based on the object-background relationship. Particularly, we regularize the geometry smoothness of occluded background patches. With the improved background surface, the signed distance function and the reversedly rendered depth of objects can be optimized to bound them within the background range. Extensive experiments show our method outperforms other methods on synthetic and real-world indoor scenes and prove the effectiveness of proposed regularizations. The code is available at https://github.com/kyleleey/RICO.