Multiscale Representation for Real-Time Anti-Aliasing Neural Rendering
This work addresses rendering quality issues in neural graphics for applications like VR/AR, offering a real-time solution that improves over existing methods, though it builds incrementally on prior multiscale approaches like Mip-NeRF.
The paper tackles the problem of aliasing artifacts and blurred rendering in neural radiance fields (NeRF) when dealing with non-uniform scales or distant views, proposing Mip-VoG, a multiscale representation that enables real-time anti-aliasing rendering and outperforms state-of-the-art baselines in experiments.
The rendering scheme in neural radiance field (NeRF) is effective in rendering a pixel by casting a ray into the scene. However, NeRF yields blurred rendering results when the training images are captured at non-uniform scales, and produces aliasing artifacts if the test images are taken in distant views. To address this issue, Mip-NeRF proposes a multiscale representation as a conical frustum to encode scale information. Nevertheless, this approach is only suitable for offline rendering since it relies on integrated positional encoding (IPE) to query a multilayer perceptron (MLP). To overcome this limitation, we propose mip voxel grids (Mip-VoG), an explicit multiscale representation with a deferred architecture for real-time anti-aliasing rendering. Our approach includes a density Mip-VoG for scene geometry and a feature Mip-VoG with a small MLP for view-dependent color. Mip-VoG encodes scene scale using the level of detail (LOD) derived from ray differentials and uses quadrilinear interpolation to map a queried 3D location to its features and density from two neighboring downsampled voxel grids. To our knowledge, our approach is the first to offer multiscale training and real-time anti-aliasing rendering simultaneously. We conducted experiments on multiscale datasets, and the results show that our approach outperforms state-of-the-art real-time rendering baselines.