Drantal-NeRF: Diffusion-Based Restoration for Anti-aliasing Neural Radiance Field
This addresses a long-standing aliasing problem in 3D implicit representations for computer vision and graphics applications, though it appears incremental as it builds on existing diffusion and NeRF methods.
The paper tackles aliasing artifacts in Neural Radiance Field (NeRF) renderings by proposing Drantal-NeRF, a diffusion-based restoration method that treats aliasing as a degradation model and restores high-quality anti-aliased renderings from low-quality inputs, achieving substantial qualitative and quantitative improvements on challenging urban and 360-degree scenes.
Aliasing artifacts in renderings produced by Neural Radiance Field (NeRF) is a long-standing but complex issue in the field of 3D implicit representation, which arises from a multitude of intricate causes and was mitigated by designing more advanced but complex scene parameterization methods before. In this paper, we present a Diffusion-based restoration method for anti-aliasing Neural Radiance Field (Drantal-NeRF). We consider the anti-aliasing issue from a low-level restoration perspective by viewing aliasing artifacts as a kind of degradation model added to clean ground truths. By leveraging the powerful prior knowledge encapsulated in diffusion model, we could restore the high-realism anti-aliasing renderings conditioned on aliased low-quality counterparts. We further employ a feature-wrapping operation to ensure multi-view restoration consistency and finetune the VAE decoder to better adapt to the scene-specific data distribution. Our proposed method is easy to implement and agnostic to various NeRF backbones. We conduct extensive experiments on challenging large-scale urban scenes as well as unbounded 360-degree scenes and achieve substantial qualitative and quantitative improvements.