An Analysis of Regularized Approaches for Constrained Machine Learning
This is an incremental theoretical analysis for researchers in constrained machine learning, highlighting limitations of existing methods.
The paper tackles the problem of balancing loss and regularization in constrained machine learning, demonstrating that regularization-based approaches cannot guarantee finding all optimal solutions, particularly in non-convex cases where some optima do not correspond to any multiplier value.
Regularization-based approaches for injecting constraints in Machine Learning (ML) were introduced to improve a predictive model via expert knowledge. We tackle the issue of finding the right balance between the loss (the accuracy of the learner) and the regularization term (the degree of constraint satisfaction). The key results of this paper is the formal demonstration that this type of approach cannot guarantee to find all optimal solutions. In particular, in the non-convex case there might be optima for the constrained problem that do not correspond to any multiplier value.