A Model-Based Approach to Imitation Learning through Multi-Step Predictions
This addresses limitations in imitation learning for training agents in complex decision-making tasks, though it appears incremental as it builds on existing model-based approaches.
The paper tackles the problem of compounding errors and limited generalization in imitation learning by introducing a model-based framework using multi-step predictions, which outperforms behavior cloning benchmarks with superior robustness to distribution shift and noise.
Imitation learning is a widely used approach for training agents to replicate expert behavior in complex decision-making tasks. However, existing methods often struggle with compounding errors and limited generalization, due to the inherent challenge of error correction and the distribution shift between training and deployment. In this paper, we present a novel model-based imitation learning framework inspired by model predictive control, which addresses these limitations by integrating predictive modeling through multi-step state predictions. Our method outperforms traditional behavior cloning numerical benchmarks, demonstrating superior robustness to distribution shift and measurement noise both in available data and during execution. Furthermore, we provide theoretical guarantees on the sample complexity and error bounds of our method, offering insights into its convergence properties.