An empirical investigation of the challenges of real-world reinforcement learning
This work addresses the gap between theoretical RL advances and practical applications, providing a framework to improve deployability across various real-world problems, though it is incremental as it synthesizes existing issues rather than introducing new methods.
The paper identifies and formalizes key challenges that hinder the deployment of reinforcement learning in real-world systems, analyzing their impact on state-of-the-art algorithms and presenting existing solutions, with the challenges implemented in an open-source benchmark suite called realworldrl-suite.
Reinforcement learning (RL) has proven its worth in a series of artificial domains, and is beginning to show some successes in real-world scenarios. However, much of the research advances in RL are hard to leverage in real-world systems due to a series of assumptions that are rarely satisfied in practice. In this work, we identify and formalize a series of independent challenges that embody the difficulties that must be addressed for RL to be commonly deployed in real-world systems. For each challenge, we define it formally in the context of a Markov Decision Process, analyze the effects of the challenge on state-of-the-art learning algorithms, and present some existing attempts at tackling it. We believe that an approach that addresses our set of proposed challenges would be readily deployable in a large number of real world problems. Our proposed challenges are implemented in a suite of continuous control environments called the realworldrl-suite which we propose an as an open-source benchmark.