Faster and Safer Training by Embedding High-Level Knowledge into Deep Reinforcement Learning
This addresses the problem of inefficient and unsafe training in dynamic decision-making domains for AI practitioners, though it appears incremental as it builds on existing deep reinforcement learning methods.
The paper tackles the slow, resource-intensive, and brittle nature of deep reinforcement learning by proposing Rule-interposing Learning (RIL), a framework that embeds high-level rules to accelerate training and prevent catastrophic explorations, resulting in a more stable system even in early training stages.
Deep reinforcement learning has been successfully used in many dynamic decision making domains, especially those with very large state spaces. However, it is also well-known that deep reinforcement learning can be very slow and resource intensive. The resulting system is often brittle and difficult to explain. In this paper, we attempt to address some of these problems by proposing a framework of Rule-interposing Learning (RIL) that embeds high level rules into the deep reinforcement learning. With some good rules, this framework not only can accelerate the learning process, but also keep it away from catastrophic explorations, thus making the system relatively stable even during the very early stage of training. Moreover, given the rules are high level and easy to interpret, they can be easily maintained, updated and shared with other similar tasks.