Flexible risk design using bi-directional dispersion
This work addresses a foundational issue in risk-sensitive machine learning by providing a more flexible and balanced risk design, which is incremental but offers practical improvements for gradient-based learners.
The paper tackles the problem of existing risk measures being overly sensitive to upside loss tails and ignoring downside deviations by introducing a new risk class that penalizes deviations in both directions with flexible tail sensitivity. The result is high-probability learning guarantees without gradient clipping and empirical tests showing control over test loss distribution properties.
Many novel notions of "risk" (e.g., CVaR, tilted risk, DRO risk) have been proposed and studied, but these risks are all at least as sensitive as the mean to loss tails on the upside, and tend to ignore deviations on the downside. We study a complementary new risk class that penalizes loss deviations in a bi-directional manner, while having more flexibility in terms of tail sensitivity than is offered by mean-variance. This class lets us derive high-probability learning guarantees without explicit gradient clipping, and empirical tests using both simulated and real data illustrate a high degree of control over key properties of the test loss distribution incurred by gradient-based learners.