Bayesian Robustness: A Nonasymptotic Viewpoint
This work addresses the problem of robust statistical inference for practitioners dealing with contaminated data, though it is incremental as it builds on existing ULA methods.
The paper tackles robust Bayesian posterior estimation in the presence of adversarial outliers by proposing Rob-ULA, a robust variant of the Unadjusted Langevin Algorithm, and shows that after T = Õ(d/ε_acc) iterations, the sampling distribution achieves an error bound of ε_acc + Õ(ε), where ε is the corruption fraction.
We study the problem of robustly estimating the posterior distribution for the setting where observed data can be contaminated with potentially adversarial outliers. We propose Rob-ULA, a robust variant of the Unadjusted Langevin Algorithm (ULA), and provide a finite-sample analysis of its sampling distribution. In particular, we show that after $T= \tilde{\mathcal{O}}(d/\varepsilon_{\textsf{acc}})$ iterations, we can sample from $p_T$ such that $\text{dist}(p_T, p^*) \leq \varepsilon_{\textsf{acc}} + \tilde{\mathcal{O}}(ε)$, where $ε$ is the fraction of corruptions. We corroborate our theoretical analysis with experiments on both synthetic and real-world data sets for mean estimation, regression and binary classification.