AL-CoLe: Augmented Lagrangian for Constrained Learning
This addresses constrained learning problems in ML, particularly for fairness in classification, with incremental improvements to existing Lagrangian approaches.
The paper tackles constrained learning problems in non-convex machine learning by revisiting Augmented Lagrangian methods, establishing strong duality results and proving convergence to feasible optimal solutions with PAC-style generalization guarantees.
Despite the non-convexity of most modern machine learning parameterizations, Lagrangian duality has become a popular tool for addressing constrained learning problems. We revisit Augmented Lagrangian methods, which aim to mitigate the duality gap in non-convex settings while requiring only minimal modifications, and have remained comparably unexplored in constrained learning settings. We establish strong duality results under mild conditions, prove convergence of dual ascent algorithms to feasible and optimal primal solutions, and provide PAC-style generalization guarantees. Finally, we demonstrate its effectiveness on fairness constrained classification tasks.