Consistent Multiclass Algorithms for Complex Metrics and Constraints
This addresses the problem of designing consistent algorithms for multiclass classification with complex metrics and constraints, which is important for practitioners needing tailored performance measures and fairness guarantees, though it appears incremental as it builds on existing optimization frameworks.
The paper tackles multiclass learning with complex performance metrics and constraints defined by arbitrary functions of the confusion matrix, such as G-mean, F1-measure, and fairness constraints. It presents a general framework for designing consistent algorithms that converge to optimal feasible classifiers, with experiments showing favorable comparisons to state-of-the-art baselines.
We present consistent algorithms for multiclass learning with complex performance metrics and constraints, where the objective and constraints are defined by arbitrary functions of the confusion matrix. This setting includes many common performance metrics such as the multiclass G-mean and micro F1-measure, and constraints such as those on the classifier's precision and recall and more recent measures of fairness discrepancy. We give a general framework for designing consistent algorithms for such complex design goals by viewing the learning problem as an optimization problem over the set of feasible confusion matrices. We provide multiple instantiations of our framework under different assumptions on the performance metrics and constraints, and in each case show rates of convergence to the optimal (feasible) classifier (and thus asymptotic consistency). Experiments on a variety of multiclass classification tasks and fairness-constrained problems show that our algorithms compare favorably to the state-of-the-art baselines.