When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
This work addresses the challenge of optimizing pAUC for deep learning, which is important for applications like imbalanced classification, but it is incremental as it builds on existing DRO and pAUC methods.
The paper tackles the problem of maximizing partial AUC (pAUC) in deep learning by proposing gradient-based methods with non-convex convergence guarantees, using distributionally robust optimization (DRO) to formulate surrogate objectives, and demonstrates effectiveness on various datasets.
In this paper, we propose systematic and efficient gradient-based methods for both one-way and two-way partial AUC (pAUC) maximization that are applicable to deep learning. We propose new formulations of pAUC surrogate objectives by using the distributionally robust optimization (DRO) to define the loss for each individual positive data. We consider two formulations of DRO, one of which is based on conditional-value-at-risk (CVaR) that yields a non-smooth but exact estimator for pAUC, and another one is based on a KL divergence regularized DRO that yields an inexact but smooth (soft) estimator for pAUC. For both one-way and two-way pAUC maximization, we propose two algorithms and prove their convergence for optimizing their two formulations, respectively. Experiments demonstrate the effectiveness of the proposed algorithms for pAUC maximization for deep learning on various datasets.