Bridging Diffusion Models and 3D Representations: A 3D Consistent Super-Resolution Framework
This addresses the challenge of maintaining spatial coherence in 3D reconstructions for applications like computer vision and graphics, though it is incremental as it builds on existing methods.
The paper tackles the problem of achieving 3D consistency in super-resolution by proposing 3DSR, a framework that uses 3D Gaussian splatting and off-the-shelf 2D diffusion models, resulting in high-resolution, visually compelling outputs on datasets like MipNeRF360 and LLFF.
We propose 3D Super Resolution (3DSR), a novel 3D Gaussian-splatting-based super-resolution framework that leverages off-the-shelf diffusion-based 2D super-resolution models. 3DSR encourages 3D consistency across views via the use of an explicit 3D Gaussian-splatting-based scene representation. This makes the proposed 3DSR different from prior work, such as image upsampling or the use of video super-resolution, which either don't consider 3D consistency or aim to incorporate 3D consistency implicitly. Notably, our method enhances visual quality without additional fine-tuning, ensuring spatial coherence within the reconstructed scene. We evaluate 3DSR on MipNeRF360 and LLFF data, demonstrating that it produces high-resolution results that are visually compelling, while maintaining structural consistency in 3D reconstructions.