Pinet: Optimizing hard-constrained neural networks with orthogonal projection layers
This work addresses the challenge of incorporating hard constraints into neural networks for applications like optimization and motion planning, offering a practical tool for researchers and engineers in these domains, though it is incremental as it builds on existing projection and optimization methods.
The paper tackles the problem of ensuring neural network outputs satisfy convex constraints by introducing an output layer that uses operator splitting for fast projections and the implicit function theorem for backpropagation. It results in modest-accuracy solutions faster than traditional solvers for single problems and significantly faster for batches, with improved training time, solution quality, and robustness compared to state-of-the-art learning approaches.
We introduce an output layer for neural networks that ensures satisfaction of convex constraints. Our approach, $Π$net, leverages operator splitting for rapid and reliable projections in the forward pass, and the implicit function theorem for backpropagation. We deploy $Π$net as a feasible-by-design optimization proxy for parametric constrained optimization problems and obtain modest-accuracy solutions faster than traditional solvers when solving a single problem, and significantly faster for a batch of problems. We surpass state-of-the-art learning approaches in terms of training time, solution quality, and robustness to hyperparameter tuning, while maintaining similar inference times. Finally, we tackle multi-vehicle motion planning with non-convex trajectory preferences and provide $Π$net as a GPU-ready package implemented in JAX with effective tuning heuristics.