Deep Imitative Models for Flexible Inference, Planning, and Control
This work addresses the problem of achieving arbitrary goals in autonomous systems for robotics and AI, representing a novel integration of imitation learning and planning.
The paper tackles the challenge of combining imitation learning with goal-directed planning by introducing Imitative Models, which enable flexible inference and planning to achieve specified goals, and demonstrates substantial performance improvements over existing methods in autonomous driving tasks.
Imitation Learning (IL) is an appealing approach to learn desirable autonomous behavior. However, directing IL to achieve arbitrary goals is difficult. In contrast, planning-based algorithms use dynamics models and reward functions to achieve goals. Yet, reward functions that evoke desirable behavior are often difficult to specify. In this paper, we propose Imitative Models to combine the benefits of IL and goal-directed planning. Imitative Models are probabilistic predictive models of desirable behavior able to plan interpretable expert-like trajectories to achieve specified goals. We derive families of flexible goal objectives, including constrained goal regions, unconstrained goal sets, and energy-based goals. We show that our method can use these objectives to successfully direct behavior. Our method substantially outperforms six IL approaches and a planning-based approach in a dynamic simulated autonomous driving task, and is efficiently learned from expert demonstrations without online data collection. We also show our approach is robust to poorly specified goals, such as goals on the wrong side of the road.