A Human-Centered Data-Driven Planner-Actor-Critic Architecture via Logic Programming
This addresses the efficiency and data requirements of RL for agents, though it appears incremental as it combines existing methods.
The paper tackles the problem of reinforcement learning being slow and data-hungry by proposing a Planner-Actor-Critic architecture (PACMAN) that integrates symbolic knowledge, RL, and human feedback, resulting in significant jump-start, rapid convergence, and robustness to poor feedback.
Recent successes of Reinforcement Learning (RL) allow an agent to learn policies that surpass human experts but suffers from being time-hungry and data-hungry. By contrast, human learning is significantly faster because prior and general knowledge and multiple information resources are utilized. In this paper, we propose a Planner-Actor-Critic architecture for huMAN-centered planning and learning (PACMAN), where an agent uses its prior, high-level, deterministic symbolic knowledge to plan for goal-directed actions, and also integrates the Actor-Critic algorithm of RL to fine-tune its behavior towards both environmental rewards and human feedback. This work is the first unified framework where knowledge-based planning, RL, and human teaching jointly contribute to the policy learning of an agent. Our experiments demonstrate that PACMAN leads to a significant jump-start at the early stage of learning, converges rapidly and with small variance, and is robust to inconsistent, infrequent, and misleading feedback.