Plan2Vec: Unsupervised Representation Learning by Latent Plans
This method addresses the challenge of efficient planning in reinforcement learning for robotics or simulation tasks, though it appears incremental as it builds on existing graph-based and planning techniques.
The paper tackles the problem of unsupervised representation learning for long-horizon control by introducing plan2vec, which constructs a graph from image data and uses path integrals to derive global embeddings, resulting in compute and sample-efficient goal-conditioned value estimates.
In this paper we introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning. Plan2vec constructs a weighted graph on an image dataset using near-neighbor distances, and then extrapolates this local metric to a global embedding by distilling path-integral over planned path. When applied to control, plan2vec offers a way to learn goal-conditioned value estimates that are accurate over long horizons that is both compute and sample efficient. We demonstrate the effectiveness of plan2vec on one simulated and two challenging real-world image datasets. Experimental results show that plan2vec successfully amortizes the planning cost, enabling reactive planning that is linear in memory and computation complexity rather than exhaustive over the entire state space.