MOSAIC: Generating Consistent, Privacy-Preserving Scenes from Multiple Depth Views in Multi-Room Environments
This addresses the need for accurate and privacy-preserving scene generation in multi-room environments, which is incremental as it builds on existing diffusion-based methods with a novel optimization approach.
The paper tackles the problem of generating privacy-preserving digital twins of multi-room indoor environments from depth images, introducing a diffusion-based approach called MOSAIC that explicitly considers cross-view dependencies to avoid error accumulation. The result shows that MOSAIC outperforms state-of-the-art baselines on image fidelity metrics for reconstructing complex multi-room environments.
We introduce a diffusion-based approach for generating privacy-preserving digital twins of multi-room indoor environments from depth images only. Central to our approach is a novel Multi-view Overlapped Scene Alignment with Implicit Consistency (MOSAIC) model that explicitly considers cross-view dependencies within the same scene in the probabilistic sense. MOSAIC operates through a multi-channel inference-time optimization that avoids error accumulation common in sequential or single-room constraints in panorama-based approaches. MOSAIC scales to complex scenes with zero extra training and provably reduces the variance during denoising process when more overlapping views are added, leading to improved generation quality. Experiments show that MOSAIC outperforms state-of-the-art baselines on image fidelity metrics in reconstructing complex multi-room environments. Resources and code are at https://mosaic-cmubig.github.io