Object-Driven Multi-Layer Scene Decomposition From a Single Image
This work addresses the challenge of scene decomposition for applications like 3D photography and diminished reality, representing an incremental advance over prior methods.
The paper tackles the problem of predicting color and depth behind visible content from a single RGB image by building a Layered Depth Image (LDI) with adaptive layers and semantic encoding, resulting in improved accuracy for occluded objects.
We present a method that tackles the challenge of predicting color and depth behind the visible content of an image. Our approach aims at building up a Layered Depth Image (LDI) from a single RGB input, which is an efficient representation that arranges the scene in layers, including originally occluded regions. Unlike previous work, we enable an adaptive scheme for the number of layers and incorporate semantic encoding for better hallucination of partly occluded objects. Additionally, our approach is object-driven, which especially boosts the accuracy for the occluded intermediate objects. The framework consists of two steps. First, we individually complete each object in terms of color and depth, while estimating the scene layout. Second, we rebuild the scene based on the regressed layers and enforce the recomposed image to resemble the structure of the original input. The learned representation enables various applications, such as 3D photography and diminished reality, all from a single RGB image.