Sharper Perturbed-Kullback-Leibler Exponential Tail Bounds for Beta and Dirichlet Distributions
This is an incremental improvement for statistical theory and machine learning applications involving Bayesian nonparametrics.
The paper tackles the problem of improving exponential tail bounds for Beta distributions by introducing a larger perturbation parameter to tighten the bound, and extends this result to Dirichlet distributions and processes.
This paper presents an improved exponential tail bound for Beta distributions, refining a result in [15]. This improvement is achieved by interpreting their bound as a regular Kullback-Leibler (KL) divergence one, while introducing a specific perturbation $η$ that shifts the mean of the Beta distribution closer to zero within the KL bound. Our contribution is to show that a larger perturbation can be chosen, thereby tightening the bound. We then extend this result from the Beta distribution to Dirichlet distributions and Dirichlet processes (DPs).