Risk-Averse Learning by Temporal Difference Methods
This work addresses risk-sensitive decision-making in reinforcement learning, which is incremental as it adapts existing temporal difference methods to incorporate risk measures.
The paper tackles reinforcement learning with dynamic risk measures by proposing risk-averse temporal difference methods and proves their convergence with probability one, demonstrating empirical results on a complex transportation problem.
We consider reinforcement learning with performance evaluated by a dynamic risk measure. We construct a projected risk-averse dynamic programming equation and study its properties. Then we propose risk-averse counterparts of the methods of temporal differences and we prove their convergence with probability one. We also perform an empirical study on a complex transportation problem.