Versatile Neural Processes for Learning Implicit Neural Representations
This work addresses a bottleneck in function approximation for tasks involving 1D, 2D, and 3D signals, offering a practical improvement for researchers in machine learning and computer vision.
The paper tackles the problem of inferior modeling capability of Neural Processes for complex signals by proposing Versatile Neural Processes (VNP), which introduces a bottleneck encoder and hierarchical global latent variables to efficiently approximate functions, achieving accurate learning of implicit neural representations for 3D scenes without finetuning.
Representing a signal as a continuous function parameterized by neural network (a.k.a. Implicit Neural Representations, INRs) has attracted increasing attention in recent years. Neural Processes (NPs), which model the distributions over functions conditioned on partial observations (context set), provide a practical solution for fast inference of continuous functions. However, existing NP architectures suffer from inferior modeling capability for complex signals. In this paper, we propose an efficient NP framework dubbed Versatile Neural Processes (VNP), which largely increases the capability of approximating functions. Specifically, we introduce a bottleneck encoder that produces fewer and informative context tokens, relieving the high computational cost while providing high modeling capability. At the decoder side, we hierarchically learn multiple global latent variables that jointly model the global structure and the uncertainty of a function, enabling our model to capture the distribution of complex signals. We demonstrate the effectiveness of the proposed VNP on a variety of tasks involving 1D, 2D and 3D signals. Particularly, our method shows promise in learning accurate INRs w.r.t. a 3D scene without further finetuning. Code is available at https://github.com/ZongyuGuo/Versatile-NP .