Functional Distribution Networks (FDN)
This work addresses the issue of unreliable uncertainty estimation in neural regression for machine learning practitioners, though it is incremental as it builds on existing methods like hypernetworks.
The authors tackled the problem of overconfident probabilistic regressors under distribution shift by introducing Functional Distribution Networks (FDN), which adapt predictive dispersion to inputs, resulting in improved calibration and shift-awareness compared to baselines under matched budgets.
Modern probabilistic regressors often remain overconfident under distribution shift. We present Functional Distribution Networks (FDN), an input-conditioned distribution over network weights that induces predictive mixtures whose dispersion adapts to the input. FDN is trained with a beta-ELBO and Monte Carlo sampling. We further propose an evaluation protocol that cleanly separates interpolation from extrapolation and stresses OOD sanity checks (e.g., that predictive likelihood degrades under shift while in-distribution accuracy and calibration are maintained). On standard regression tasks, we benchmark against strong Bayesian, ensemble, dropout, and hypernetwork baselines under matched parameter and update budgets, and assess accuracy, calibration, and shift-awareness with standard diagnostics. Together, the framework and protocol aim to make OOD-aware, well-calibrated neural regression practical and modular.