PAC-Bayesian Bound for the Conditional Value at Risk
This work provides theoretical guarantees for CVaR-based learning in machine learning, addressing applications like regularization and fairness, but it is incremental as it extends existing PAC-Bayesian methods to a specific risk measure.
The paper tackles the problem of deriving generalization bounds for learning algorithms that minimize the Conditional Value at Risk (CVaR) of empirical loss, presenting a PAC-Bayesian bound that is small when empirical CVaR is small, achieved by reducing CVaR estimation to expectation estimation.
Conditional Value at Risk (CVaR) is a family of "coherent risk measures" which generalize the traditional mathematical expectation. Widely used in mathematical finance, it is garnering increasing interest in machine learning, e.g., as an alternate approach to regularization, and as a means for ensuring fairness. This paper presents a generalization bound for learning algorithms that minimize the CVaR of the empirical loss. The bound is of PAC-Bayesian type and is guaranteed to be small when the empirical CVaR is small. We achieve this by reducing the problem of estimating CVaR to that of merely estimating an expectation. This then enables us, as a by-product, to obtain concentration inequalities for CVaR even when the random variable in question is unbounded.