Posterior concentration and fast convergence rates for generalized Bayesian learning
This work addresses theoretical guarantees for Bayesian learning in non-ideal scenarios, such as irregular hypothesis classes and heavy-tailed losses, which is incremental but important for statistical robustness.
The paper tackles the problem of establishing fast learning rates for generalized Bayes estimators in complex settings, proving that under the multi-scale Bernstein's condition, the posterior concentrates around optimal hypotheses and achieves fast rates, with an application showing robustness of Bayesian linear regression to heavy-tailed distributions.
In this paper, we study the learning rate of generalized Bayes estimators in a general setting where the hypothesis class can be uncountable and have an irregular shape, the loss function can have heavy tails, and the optimal hypothesis may not be unique. We prove that under the multi-scale Bernstein's condition, the generalized posterior distribution concentrates around the set of optimal hypotheses and the generalized Bayes estimator can achieve fast learning rate. Our results are applied to show that the standard Bayesian linear regression is robust to heavy-tailed distributions.