Risk averse non-stationary multi-armed bandits
This work addresses risk management in dynamic decision-making for applications like finance or robotics, but it is incremental as it adapts existing methods to non-stationary settings.
The paper tackled the risk-averse multi-armed bandits problem with non-stationary losses by using conditional value-at-risk (CVaR) as the objective and proposed two estimation methods, showing that these policies outperform naive benchmarks in simulations.
This paper tackles the risk averse multi-armed bandits problem when incurred losses are non-stationary. The conditional value-at-risk (CVaR) is used as the objective function. Two estimation methods are proposed for this objective function in the presence of non-stationary losses, one relying on a weighted empirical distribution of losses and another on the dual representation of the CVaR. Such estimates can then be embedded into classic arm selection methods such as epsilon-greedy policies. Simulation experiments assess the performance of the arm selection algorithms based on the two novel estimation approaches, and such policies are shown to outperform naive benchmarks not taking non-stationarity into account.