Physically Aware 360$^\circ$ View Generation from a Single Image using Disentangled Scene Embeddings
This addresses the need for rapid, photorealistic view synthesis in applications such as medical imaging and immersive content creation, though it appears incremental by building on existing techniques like Gaussian Splatting.
The paper tackles the problem of generating 360° views from a single image by introducing Disentangled360, which integrates direction disentangled volume rendering with a Gaussian Splatting backbone to differentiate isotropic and anisotropic contributions, achieving superior SSIM and LPIPS performance on datasets like Mip-NeRF 360 and RealEstate10K.
We introduce Disentangled360, an innovative 3D-aware technology that integrates the advantages of direction disentangled volume rendering with single-image 360° unique view synthesis for applications in medical imaging and natural scene reconstruction. In contrast to current techniques that either oversimplify anisotropic light behavior or lack generalizability across various contexts, our framework distinctly differentiates between isotropic and anisotropic contributions inside a Gaussian Splatting backbone. We implement a dual-branch conditioning framework, one optimized for CT intensity driven scattering in volumetric data and the other for real-world RGB scenes through normalized camera embeddings. To address scale ambiguity and maintain structural realism, we present a hybrid pose agnostic anchoring method that adaptively samples scene depth and material transitions, functioning as stable pivots during scene distillation. Our design integrates preoperative radiography simulation and consumer-grade 360° rendering into a singular inference pipeline, facilitating rapid, photorealistic view synthesis with inherent directionality. Evaluations on the Mip-NeRF 360, RealEstate10K, and DeepDRR datasets indicate superior SSIM and LPIPS performance, while runtime assessments confirm its viability for interactive applications. Disentangled360 facilitates mixed-reality medical supervision, robotic perception, and immersive content creation, eliminating the necessity for scene-specific finetuning or expensive photon simulations.