Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior
This provides theoretical guarantees for behavior cloning in robotics or AI control, though it appears incremental as it builds on existing generative modeling techniques.
The paper tackles the problem of behavior cloning for complex expert demonstrations by proposing a theoretical framework that combines low-level stability guarantees with generative modeling, proving that supervised behavior cloning can match expert trajectory distributions in optimal transport cost when certain conditions are met. It validates the approach empirically with diffusion models.
We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. Our framework invokes low-level controllers - either learned or implicit in position-command control - to stabilize imitation around expert demonstrations. We show that with (a) a suitable low-level stability guarantee and (b) a powerful enough generative model as our imitation learner, pure supervised behavior cloning can generate trajectories matching the per-time step distribution of essentially arbitrary expert trajectories in an optimal transport cost. Our analysis relies on a stochastic continuity property of the learned policy we call "total variation continuity" (TVC). We then show that TVC can be ensured with minimal degradation of accuracy by combining a popular data-augmentation regimen with a novel algorithmic trick: adding augmentation noise at execution time. We instantiate our guarantees for policies parameterized by diffusion models and prove that if the learner accurately estimates the score of the (noise-augmented) expert policy, then the distribution of imitator trajectories is close to the demonstrator distribution in a natural optimal transport distance. Our analysis constructs intricate couplings between noise-augmented trajectories, a technique that may be of independent interest. We conclude by empirically validating our algorithmic recommendations, and discussing implications for future research directions for better behavior cloning with generative modeling.