Learning Visual Planning Models from Partially Observed Images
This addresses the problem of enabling planning in real-world scenarios where visual data is incomplete, which is incremental as it builds on prior work for fully observed images.
The paper tackles the problem of learning visual planning models from partially observed images, which is challenging because existing methods often require fully observed data. It introduces a framework called Recplan that learns latent state representations and transition models from image traces, and shows it is more effective than a state-of-the-art approach in environments with incomplete observations.
There has been increasing attention on planning model learning in classical planning. Most existing approaches, however, focus on learning planning models from structured data in symbolic representations. It is often difficult to obtain such structured data in real-world scenarios. Although a number of approaches have been developed for learning planning models from fully observed unstructured data (e.g., images), in many scenarios raw observations are often incomplete. In this paper, we provide a novel framework, \aType{Recplan}, for learning a transition model from partially observed raw image traces. More specifically, by considering the preceding and subsequent images in a trace, we learn the latent state representations of raw observations and then build a transition model based on such representations. Additionally, we propose a neural-network-based approach to learn a heuristic model that estimates the distance toward a given goal observation. Based on the learned transition model and heuristic model, we implement a classical planner for images. We exhibit empirically that our approach is more effective than a state-of-the-art approach of learning visual planning models in the environment with incomplete observations.