Specifying and Solving Robust Empirical Risk Minimization Problems Using CVXPY
This work provides a practical tool for machine learning practitioners to handle robust ERM problems more easily, but it is incremental as it automates an existing method rather than introducing new algorithmic insights.
The authors tackled the challenge of automating the dualization process in robust empirical risk minimization (ERM), which is typically tedious and error-prone, by developing a framework using CVXPY that allows practitioners to specify and solve such problems with convex losses and uncertainty sets, resulting in a user-friendly tool for robust machine learning.
We consider robust empirical risk minimization (ERM), where model parameters are chosen to minimize the worst-case empirical loss when each data point varies over a given convex uncertainty set. In some simple cases, such problems can be expressed in an analytical form. In general the problem can be made tractable via dualization, which turns a min-max problem into a min-min problem. Dualization requires expertise and is tedious and error-prone. We demonstrate how CVXPY can be used to automate this dualization procedure in a user-friendly manner. Our framework allows practitioners to specify and solve robust ERM problems with a general class of convex losses, capturing many standard regression and classification problems. Users can easily specify any complex uncertainty set that is representable via disciplined convex programming (DCP) constraints.