Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learning
This addresses the issue of novelty adaptation for autonomous robots in open-world settings, representing an incremental improvement over existing methods.
The paper tackles the problem of autonomous agents failing to plan in dynamic environments due to novel objects by proposing a neuro-symbolic architecture that integrates symbolic planning, reinforcement learning, and an LLM, resulting in outperforming state-of-the-art methods in operator discovery and learning.
In dynamic open-world environments, autonomous agents often encounter novelties that hinder their ability to find plans to achieve their goals. Specifically, traditional symbolic planners fail to generate plans when the robot's planning domain lacks the operators that enable it to interact appropriately with novel objects in the environment. We propose a neuro-symbolic architecture that integrates symbolic planning, reinforcement learning, and a large language model (LLM) to learn how to handle novel objects. In particular, we leverage the common sense reasoning capability of the LLM to identify missing operators, generate plans with the symbolic AI planner, and write reward functions to guide the reinforcement learning agent in learning control policies for newly identified operators. Our method outperforms the state-of-the-art methods in operator discovery as well as operator learning in continuous robotic domains.