Learning Rational Subgoals from Demonstrations and Instructions
This addresses the challenge of efficient planning in complex environments for AI agents, though it is incremental as it builds on existing planning methods with learned subgoals.
The paper tackles the problem of learning useful subgoals from weakly-annotated demonstrations and instructions to improve long-term planning efficiency for novel goals, resulting in significant performance-time efficiency gains when integrated with off-the-shelf planning algorithms like A* and RRT.
We present a framework for learning useful subgoals that support efficient long-term planning to achieve novel goals. At the core of our framework is a collection of rational subgoals (RSGs), which are essentially binary classifiers over the environmental states. RSGs can be learned from weakly-annotated data, in the form of unsegmented demonstration trajectories, paired with abstract task descriptions, which are composed of terms initially unknown to the agent (e.g., collect-wood then craft-boat then go-across-river). Our framework also discovers dependencies between RSGs, e.g., the task collect-wood is a helpful subgoal for the task craft-boat. Given a goal description, the learned subgoals and the derived dependencies facilitate off-the-shelf planning algorithms, such as A* and RRT, by setting helpful subgoals as waypoints to the planner, which significantly improves performance-time efficiency.