An Approximate Solution Method for Large Risk-Averse Markov Decision Processes
This addresses the challenge of computational tractability for risk-averse decision-making in stochastic domains, but it is incremental as it builds on existing modeling work without broader SOTA claims.
The paper tackles the problem of solving large risk-averse Markov decision processes with hybrid continuous-discrete state and continuous action spaces, proposing a method that iteratively improves a bound on the value function and demonstrating it on a portfolio optimization problem.
Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse MDPs with hybrid continuous-discrete state spaces and continuous action spaces. The proposed method iteratively improves a bound on the value function using a linearity structure of the MDP. We demonstrate the utility and properties of the method on a portfolio optimization problem.