NCVX: A General-Purpose Optimization Solver for Constrained Machine and Deep Learning
This provides a practical tool for researchers and practitioners in AI and scientific fields who need to impose explicit constraints, such as for trustworthy AI or physical law compliance, though it is incremental as it builds on existing frameworks.
The authors tackled the problem of solving constrained optimization in machine and deep learning by introducing NCVX, a software package that integrates auto-differentiation and GPU acceleration from PyTorch, enabling reliable constraint handling without optimization expertise.
Imposing explicit constraints is relatively new but increasingly pressing in deep learning, stimulated by, e.g., trustworthy AI that performs robust optimization over complicated perturbation sets and scientific applications that need to respect physical laws and constraints. However, it can be hard to reliably solve constrained deep learning problems without optimization expertise. The existing deep learning frameworks do not admit constraints. General-purpose optimization packages can handle constraints but do not perform auto-differentiation and have trouble dealing with nonsmoothness. In this paper, we introduce a new software package called NCVX, whose initial release contains the solver PyGRANSO, a PyTorch-enabled general-purpose optimization package for constrained machine/deep learning problems, the first of its kind. NCVX inherits auto-differentiation, GPU acceleration, and tensor variables from PyTorch, and is built on freely available and widely used open-source frameworks. NCVX is available at https://ncvx.org, with detailed documentation and numerous examples from machine/deep learning and other fields.