Safe-Bayesian Generalized Linear Regression
This addresses robustness issues in Bayesian regression for practitioners when models are imperfect approximations of reality.
The paper tackles the problem of Bayesian inference under model misspecification in generalized linear models, showing that generalized Bayes with specific learning rates concentrates around the best approximation of truth even under severely misspecified noise, and demonstrates substantial performance improvements over standard Bayes on simulated and real-world data.
We study generalized Bayesian inference under misspecification, i.e. when the model is 'wrong but useful'. Generalized Bayes equips the likelihood with a learning rate $η$. We show that for generalized linear models (GLMs), $η$-generalized Bayes concentrates around the best approximation of the truth within the model for specific $η\neq 1$, even under severely misspecified noise, as long as the tails of the true distribution are exponential. We derive MCMC samplers for generalized Bayesian lasso and logistic regression and give examples of both simulated and real-world data in which generalized Bayes substantially outperforms standard Bayes.