Self-Supervised Primal-Dual Learning for Constrained Optimization
This addresses the problem of efficient and feasible model training for constrained optimization in machine learning, offering a novel self-supervised approach that is incremental in its method adaptation.
The paper tackles the challenge of training machine-learning models to approximate optimal solutions of constrained optimization problems without requiring pre-solved instances or solvers, by proposing a self-supervised primal-dual learning method that achieves negligible constraint violations and minor optimality gaps, closely matching the performance of an Augmented Lagrangian Method.
This paper studies how to train machine-learning models that directly approximate the optimal solutions of constrained optimization problems. This is an empirical risk minimization under constraints, which is challenging as training must balance optimality and feasibility conditions. Supervised learning methods often approach this challenge by training the model on a large collection of pre-solved instances. This paper takes a different route and proposes the idea of Primal-Dual Learning (PDL), a self-supervised training method that does not require a set of pre-solved instances or an optimization solver for training and inference. Instead, PDL mimics the trajectory of an Augmented Lagrangian Method (ALM) and jointly trains primal and dual neural networks. Being a primal-dual method, PDL uses instance-specific penalties of the constraint terms in the loss function used to train the primal network. Experiments show that, on a set of nonlinear optimization benchmarks, PDL typically exhibits negligible constraint violations and minor optimality gaps, and is remarkably close to the ALM optimization. PDL also demonstrated improved or similar performance in terms of the optimality gaps, constraint violations, and training times compared to existing approaches.