Depth-SIMS: Semi-Parametric Image and Depth Synthesis
This work addresses the need for high-quality synthetic training data for semantic segmentation and depth completion tasks, though it appears incremental as it builds on existing compositing and in-painting techniques.
The paper tackles the problem of generating aligned RGB images and depth maps by developing a compositing synthesis method that creates RGB canvases with segmentation maps and sparse depth, then uses an in-painting network to produce high-quality images and dense depth maps. The result shows a 3.7 percentage point increase in mIoU over state-of-the-art methods and competitive FID scores.
In this paper we present a compositing image synthesis method that generates RGB canvases with well aligned segmentation maps and sparse depth maps, coupled with an in-painting network that transforms the RGB canvases into high quality RGB images and the sparse depth maps into pixel-wise dense depth maps. We benchmark our method in terms of structural alignment and image quality, showing an increase in mIoU over SOTA by 3.7 percentage points and a highly competitive FID. Furthermore, we analyse the quality of the generated data as training data for semantic segmentation and depth completion, and show that our approach is more suited for this purpose than other methods.