Distance-informed Neural Processes
This work addresses uncertainty calibration issues in Neural Processes, which is important for reliable machine learning applications, but it appears incremental as it builds on existing Neural Process frameworks.
The paper tackled the problem of poor uncertainty estimation and local dependency capture in Neural Processes by proposing the Distance-informed Neural Process (DNP), which combines global and distance-aware local latent structures with bi-Lipschitz regularization, resulting in better-calibrated uncertainty estimates and improved predictive performance across regression and classification tasks.
We propose the Distance-informed Neural Process (DNP), a novel variant of Neural Processes that improves uncertainty estimation by combining global and distance-aware local latent structures. Standard Neural Processes (NPs) often rely on a global latent variable and struggle with uncertainty calibration and capturing local data dependencies. DNP addresses these limitations by introducing a global latent variable to model task-level variations and a local latent variable to capture input similarity within a distance-preserving latent space. This is achieved through bi-Lipschitz regularization, which bounds distortions in input relationships and encourages the preservation of relative distances in the latent space. This modeling approach allows DNP to produce better-calibrated uncertainty estimates and more effectively distinguish in- from out-of-distribution data. Empirical results demonstrate that DNP achieves strong predictive performance and improved uncertainty calibration across regression and classification tasks.