Incorporating Relational Background Knowledge into Reinforcement Learning via Differentiable Inductive Logic Programming
This work addresses the challenge of integrating expert knowledge into reinforcement learning for improved efficiency and generalization, representing a novel method for a known bottleneck in relational reinforcement learning.
The paper tackled the problem of incorporating relational background knowledge into reinforcement learning by proposing a novel deep relational reinforcement learning framework based on differentiable inductive logic programming, which effectively learns relational information from images and incorporates expert knowledge, leading to faster learning and better generalization in environments like BoxWorld, GridWorld, and Sort-of-CLEVR.
Relational Reinforcement Learning (RRL) can offers various desirable features. Most importantly, it allows for incorporating expert knowledge into the learning, and hence leading to much faster learning and better generalization compared to the standard deep reinforcement learning. However, most of the existing RRL approaches are either incapable of incorporating expert background knowledge (e.g., in the form of explicit predicate language) or are not able to learn directly from non-relational data such as image. In this paper, we propose a novel deep RRL based on a differentiable Inductive Logic Programming (ILP) that can effectively learn relational information from image and present the state of the environment as first order logic predicates. Additionally, it can take the expert background knowledge and incorporate it into the learning problem using appropriate predicates. The differentiable ILP allows an end to end optimization of the entire framework for learning the policy in RRL. We show the efficacy of this novel RRL framework using environments such as BoxWorld, GridWorld as well as relational reasoning for the Sort-of-CLEVR dataset.