Stochastic Optimization for Spectral Risk Measures
This work addresses optimization challenges for spectral risk measures in machine learning, which is an incremental improvement over existing methods.
The paper tackles the problem of optimizing spectral risk measures (L-risks) in learning systems, which interpolate between average-case and worst-case performance, by developing stochastic algorithms that address biased subgradient estimates and non-smoothness. The result shows that their approach outperforms standard methods like stochastic subgradient and dual averaging, both theoretically and experimentally.
Spectral risk objectives - also called $L$-risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop stochastic algorithms to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging are hindered by bias and that our approach outperforms them.