Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning
This review addresses the need for understanding and quantifying these behaviors in reinforcement learning for applications beyond gaming and simulations, but it is incremental as it synthesizes existing literature without introducing new methods.
The authors conducted a comprehensive literature review to characterize metrics for measuring the stability, robustness, and resilience of reinforcement learning methods, providing a decision tree for selecting these metrics.
Reinforcement learning has received significant interest in recent years, due primarily to the successes of deep reinforcement learning at solving many challenging tasks such as playing Chess, Go and online computer games. However, with the increasing focus on reinforcement learning, applications outside of gaming and simulated environments require understanding the robustness, stability, and resilience of reinforcement learning methods. To this end, we conducted a comprehensive literature review to characterize the available literature on these three behaviors as they pertain to reinforcement learning. We classify the quantitative and theoretical approaches used to indicate or measure robustness, stability, and resilience behaviors. In addition, we identified the action or event to which the quantitative approaches were attempting to be stable, robust, or resilient. Finally, we provide a decision tree useful for selecting metrics to quantify the behaviors. We believe that this is the first comprehensive review of stability, robustness and resilience specifically geared towards reinforcement learning.