Heuristic Search Planning with Deep Neural Networks using Imitation, Attention and Curriculum Learning
This addresses the challenge of heuristic learning in planning for AI systems, offering a novel approach with significant performance gains, though it is incremental in combining known techniques like attention and curriculum learning.
The paper tackles the problem of learning a well-informed heuristic function for hard task planning domains by introducing a network model that uses attention and optimal plan imitation to relate distant parts of the state space, which drastically improves heuristic quality and far exceeds existing baselines in grid-type PDDL domains.
Learning a well-informed heuristic function for hard task planning domains is an elusive problem. Although there are known neural network architectures to represent such heuristic knowledge, it is not obvious what concrete information is learned and whether techniques aimed at understanding the structure help in improving the quality of the heuristics. This paper presents a network model to learn a heuristic capable of relating distant parts of the state space via optimal plan imitation using the attention mechanism, which drastically improves the learning of a good heuristic function. To counter the limitation of the method in the creation of problems of increasing difficulty, we demonstrate the use of curriculum learning, where newly solved problem instances are added to the training set, which, in turn, helps to solve problems of higher complexities and far exceeds the performances of all existing baselines including classical planning heuristics. We demonstrate its effectiveness for grid-type PDDL domains.