A New Perspective To Understanding Multi-resolution Hash Encoding For Neural Fields
This provides a foundational insight for researchers in neural fields, addressing a key bottleneck in optimizing hyperparameters, though it is incremental as it builds on existing architectures.
The paper tackles the lack of principled understanding of Instant-NGP's multi-resolution hash grid by proposing a domain manipulation perspective, showing through experiments on 1D signals that it explains how the grid increases neural field expressivity by creating multiple linear segments.
Instant-NGP has been the state-of-the-art architecture of neural fields in recent years. Its incredible signal-fitting capabilities are generally attributed to its multi-resolution hash grid structure and have been used and improved in numerous following works. However, it is unclear how and why such a hash grid structure improves the capabilities of a neural network by such great margins. A lack of principled understanding of the hash grid also implies that the large set of hyperparameters accompanying Instant-NGP could only be tuned empirically without much heuristics. To provide an intuitive explanation of the working principle of the hash grid, we propose a novel perspective, namely domain manipulation. This perspective provides a ground-up explanation of how the feature grid learns the target signal and increases the expressivity of the neural field by artificially creating multiples of pre-existing linear segments. We conducted numerous experiments on carefully constructed 1-dimensional signals to support our claims empirically and aid our illustrations. While our analysis mainly focuses on 1-dimensional signals, we show that the idea is generalizable to higher dimensions.