Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity
This work addresses the problem of improving risk assessment in machine learning systems for scenarios requiring distributional robustness, though it appears incremental as it builds on existing divergence risk measures.
The paper tackles the limitation of expected risk minimization (ERM) by proposing a method to construct risk measures with fine-tuned tail sensitivity, using reference distributions and coherent upper probabilities, and demonstrates its application with divergence risk measures like those based on Kullback-Leibler divergence.
Expected risk minimization (ERM) is at the core of many machine learning systems. This means that the risk inherent in a loss distribution is summarized using a single number - its average. In this paper, we propose a general approach to construct risk measures which exhibit a desired tail sensitivity and may replace the expectation operator in ERM. Our method relies on the specification of a reference distribution with a desired tail behaviour, which is in a one-to-one correspondence to a coherent upper probability. Any risk measure, which is compatible with this upper probability, displays a tail sensitivity which is finely tuned to the reference distribution. As a concrete example, we focus on divergence risk measures based on f-divergence ambiguity sets, which are a widespread tool used to foster distributional robustness of machine learning systems. For instance, we show how ambiguity sets based on the Kullback-Leibler divergence are intricately tied to the class of subexponential random variables. We elaborate the connection of divergence risk measures and rearrangement invariant Banach norms.