NP-PROV: Neural Processes with Position-Relevant-Only Variances
This addresses uncertainty estimation in function learning for machine learning applications, but is incremental as it modifies an existing Neural Processes framework.
The paper tackles the problem of Neural Processes failing on out-of-domain tasks due to shared latent spaces for mean and variance, by proposing NP-PROV, which separates these into function-value-related and position-only spaces, achieving state-of-the-art likelihood with bounded variance on synthetic and real-world datasets.
Neural Processes (NPs) families encode distributions over functions to a latent representation, given context data, and decode posterior mean and variance at unknown locations. Since mean and variance are derived from the same latent space, they may fail on out-of-domain tasks where fluctuations in function values amplify the model uncertainty. We present a new member named Neural Processes with Position-Relevant-Only Variances (NP-PROV). NP-PROV hypothesizes that a target point close to a context point has small uncertainty, regardless of the function value at that position. The resulting approach derives mean and variance from a function-value-related space and a position-related-only latent space separately. Our evaluation on synthetic and real-world datasets reveals that NP-PROV can achieve state-of-the-art likelihood while retaining a bounded variance when drifts exist in the function value.