Sampling Neural Radiance Fields for Refractive Objects
This addresses the problem of rendering refractive objects in 3D scenes for computer vision and graphics applications, representing an incremental advance over existing NeRF methods.
The paper tackles novel view synthesis for refractive objects by extending neural radiance fields (NeRF) to handle heterogeneous volumes with curved light paths, outperforming state-of-the-art methods with better perceptual similarity metrics and rendering quality on synthetic and real scenes.
Recently, differentiable volume rendering in neural radiance fields (NeRF) has gained a lot of popularity, and its variants have attained many impressive results. However, existing methods usually assume the scene is a homogeneous volume so that a ray is cast along the straight path. In this work, the scene is instead a heterogeneous volume with a piecewise-constant refractive index, where the path will be curved if it intersects the different refractive indices. For novel view synthesis of refractive objects, our NeRF-based framework aims to optimize the radiance fields of bounded volume and boundary from multi-view posed images with refractive object silhouettes. To tackle this challenging problem, the refractive index of a scene is reconstructed from silhouettes. Given the refractive index, we extend the stratified and hierarchical sampling techniques in NeRF to allow drawing samples along a curved path tracked by the Eikonal equation. The results indicate that our framework outperforms the state-of-the-art method both quantitatively and qualitatively, demonstrating better performance on the perceptual similarity metric and an apparent improvement in the rendering quality on several synthetic and real scenes.