Local Implicit Ray Function for Generalizable Radiance Field Representation
This addresses the issue of resolution mismatches in neural rendering for novel view synthesis, offering a generalizable solution for applications like virtual reality and 3D reconstruction, though it is incremental as it builds on existing NeRF methods.
The paper tackles the problem of blurred or aliased novel view rendering in generalizable neural radiance fields by proposing LIRF, which aggregates information from conical frustums to construct rays and predicts a local volumetric radiance field, resulting in outperforming state-of-the-art methods on unseen scenes at arbitrary scales.
We propose LIRF (Local Implicit Ray Function), a generalizable neural rendering approach for novel view rendering. Current generalizable neural radiance fields (NeRF) methods sample a scene with a single ray per pixel and may therefore render blurred or aliased views when the input views and rendered views capture scene content with different resolutions. To solve this problem, we propose LIRF to aggregate the information from conical frustums to construct a ray. Given 3D positions within conical frustums, LIRF takes 3D coordinates and the features of conical frustums as inputs and predicts a local volumetric radiance field. Since the coordinates are continuous, LIRF renders high-quality novel views at a continuously-valued scale via volume rendering. Besides, we predict the visible weights for each input view via transformer-based feature matching to improve the performance in occluded areas. Experimental results on real-world scenes validate that our method outperforms state-of-the-art methods on novel view rendering of unseen scenes at arbitrary scales.