SymDQN: Symbolic Knowledge and Reasoning in Neural Network-based Reinforcement Learning
This work addresses the challenge of combining neural and symbolic approaches in reinforcement learning, offering a novel method for enhancing agent behavior through symbolic guidance, though it is incremental as it builds on existing DuelDQN and LTN frameworks.
The authors tackled the problem of integrating symbolic knowledge into neural network-based reinforcement learning by proposing SymDQN, a modular architecture that augments DuelDQN with Logic Tensor Networks modules, which significantly improved learning performance and agent precision in a grid navigation environment.
We propose a learning architecture that allows symbolic control and guidance in reinforcement learning with deep neural networks. We introduce SymDQN, a novel modular approach that augments the existing Dueling Deep Q-Networks (DuelDQN) architecture with modules based on the neuro-symbolic framework of Logic Tensor Networks (LTNs). The modules guide action policy learning and allow reinforcement learning agents to display behaviour consistent with reasoning about the environment. Our experiment is an ablation study performed on the modules. It is conducted in a reinforcement learning environment of a 5x5 grid navigated by an agent that encounters various shapes, each associated with a given reward. The underlying DuelDQN attempts to learn the optimal behaviour of the agent in this environment, while the modules facilitate shape recognition and reward prediction. We show that our architecture significantly improves learning, both in terms of performance and the precision of the agent. The modularity of SymDQN allows reflecting on the intricacies and complexities of combining neural and symbolic approaches in reinforcement learning.