Sequential Neural Processes
This work addresses the need for efficient modeling of temporal dynamics in stochastic processes, particularly for applications like dynamic 3D scene rendering, though it appears incremental as it extends existing Neural Processes with temporal components.
The paper tackles the problem of modeling dynamic stochastic processes with temporal dependencies, which Neural Processes do not explicitly handle, by proposing Sequential Neural Processes (SNP) that incorporate a temporal state-transition model, resulting in the first 4D model for dynamic 3D scene modeling as demonstrated in experiments on dynamic regression and 4D scene inference.
Neural Processes combine the strengths of neural networks and Gaussian processes to achieve both flexible learning and fast prediction in stochastic processes. However, a large class of problems comprises underlying temporal dependency structures in a sequence of stochastic processes that Neural Processes (NP) do not explicitly consider. In this paper, we propose Sequential Neural Processes (SNP) which incorporates a temporal state-transition model of stochastic processes and thus extends its modeling capabilities to dynamic stochastic processes. In applying SNP to dynamic 3D scene modeling, we introduce the Temporal Generative Query Networks. To our knowledge, this is the first 4D model that can deal with the temporal dynamics of 3D scenes. In experiments, we evaluate the proposed methods in dynamic (non-stationary) regression and 4D scene inference and rendering.