Infeasible optimization problems and the hierarchical augmented Lagrangian method in imitation learning
For robotics practitioners using constrained IL, this work provides a principled remedy for infeasibility issues that can cause training instability.
The paper addresses infeasibility in constrained imitation learning problems and proposes a hierarchical augmented Lagrangian method that drives the policy toward the closest feasible solution. The method is demonstrated on a driving example with acceleration and safety constraints.
Imitation learning (IL) is an effective approach to train complex robotics policies. Recent works have introduced hard constraints into imitation-learning optimization problems to ensure safety, stability, and robustness of the learned policy. However, we argue that these constraints are sometimes infeasible, which can lead to unstable or difficult training dynamics. We study a simple remedy for such situations based on recent theoretical results on the augmented Lagrangian method in infeasible settings. We show that our approach drives the learned policy toward the solution of a closest-feasible constrained IL problem with desirable properties. The method is illustrated on a toy driving example with a total-acceleration constraint and pedestrian-safety constraints, a setting in which infeasibility can naturally arise while still allowing a safe learned policy.