Risk Sensitivity in Markov Games and Multi-Agent Reinforcement Learning: A Systematic Review
It provides a comprehensive overview for researchers in multi-agent systems, but is incremental as it reviews existing work rather than presenting new methods.
This paper systematically reviews the literature on risk sensitivity in Markov games and multi-agent reinforcement learning, addressing the problem of incorporating risk measures to model decision-making in multi-agent systems, with a focus on defining and discussing various risk measures and identifying trends in the field.
Markov games (MGs) and multi-agent reinforcement learning (MARL) are studied to model decision making in multi-agent systems. Traditionally, the objective in MG and MARL has been risk-neutral, i.e., agents are assumed to optimize a performance metric such as expected return, without taking into account subjective or cognitive preferences of themselves or of other agents. However, ignoring such preferences leads to inaccurate models of decision making in many real-world scenarios in finance, operations research, and behavioral economics. Therefore, when these preferences are present, it is necessary to incorporate a suitable measure of risk into the optimization objective of agents, which opens the door to risk-sensitive MG and MARL. In this paper, we systemically review the literature on risk sensitivity in MG and MARL that has been growing in recent years alongside other areas of reinforcement learning and game theory. We define and mathematically describe different risk measures used in MG and MARL and individually for each measure, discuss articles that incorporate it. Finally, we identify recent trends in theoretical and applied works in the field and discuss possible directions of future research.