Neural Radiance Fields for Transparent Object Using Visual Hull
This addresses a specific problem in computer vision for applications like robotics or AR/VR, but it is incremental as it builds on existing NeRF methods.
The paper tackles novel view synthesis for transparent objects, which is challenging due to light refraction, by proposing a NeRF-based method that uses visual hull reconstruction and Snell's law simulation, resulting in improved performance over conventional NeRF.
Unlike opaque object, novel view synthesis of transparent object is a challenging task, because transparent object refracts light of background causing visual distortions on the transparent object surface along the viewpoint change. Recently introduced Neural Radiance Fields (NeRF) is a view synthesis method. Thanks to its remarkable performance improvement, lots of following applications based on NeRF in various topics have been developed. However, if an object with a different refractive index is included in a scene such as transparent object, NeRF shows limited performance because refracted light ray at the surface of the transparent object is not appropriately considered. To resolve the problem, we propose a NeRF-based method consisting of the following three steps: First, we reconstruct a three-dimensional shape of a transparent object using visual hull. Second, we simulate the refraction of the rays inside of the transparent object according to Snell's law. Last, we sample points through refracted rays and put them into NeRF. Experimental evaluation results demonstrate that our method addresses the limitation of conventional NeRF with transparent objects.