Bayesian Evidential Deep Learning with PAC Regularization
This work addresses uncertainty quantification for machine learning practitioners, offering a method that extends existing approaches but is incremental in nature.
The authors tackled the problem of modeling both aleatoric and epistemic uncertainty in neural networks while maintaining a closed-form predictive distribution, achieving improved model fit and uncertainty quantification across regression, classification, and out-of-domain detection benchmarks.
We propose a novel method for closed-form predictive distribution modeling with neural nets. In quantifying prediction uncertainty, we build on Evidential Deep Learning, which has been impactful as being both simple to implement and giving closed-form access to predictive uncertainty. We employ it to model aleatoric uncertainty and extend it to account also for epistemic uncertainty by converting it to a Bayesian Neural Net. While extending its uncertainty quantification capabilities, we maintain its analytically accessible predictive distribution model by performing progressive moment matching for the first time for approximate weight marginalization. The eventual model introduces a prohibitively large number of hyperparameters for stable training. We overcome this drawback by deriving a vacuous PAC bound that comprises the marginal likelihood of the predictor and a complexity penalty. We observe on regression, classification, and out-of-domain detection benchmarks that our method improves model fit and uncertainty quantification.