Smooth Ranking SVM via Cutting-Plane Method
This work addresses class imbalance in binary classification, offering a method to improve AUC performance, though it is incremental as it builds on existing SVM-based formulations.
The paper tackles the problem of class imbalance in classification by developing a prototype learning approach that directly optimizes the Area Under the Curve (AUC) using a cutting-plane method, achieving the best test AUC in 25 out of 73 binary classification datasets.
The most popular classification algorithms are designed to maximize classification accuracy during training. However, this strategy may fail in the presence of class imbalance since it is possible to train models with high accuracy by overfitting to the majority class. On the other hand, the Area Under the Curve (AUC) is a widely used metric to compare classification performance of different algorithms when there is a class imbalance, and various approaches focusing on the direct optimization of this metric during training have been proposed. Among them, SVM-based formulations are especially popular as this formulation allows incorporating different regularization strategies easily. In this work, we develop a prototype learning approach that relies on cutting-plane method, similar to Ranking SVM, to maximize AUC. Our algorithm learns simpler models by iteratively introducing cutting planes, thus overfitting is prevented in an unconventional way. Furthermore, it penalizes the changes in the weights at each iteration to avoid large jumps that might be observed in the test performance, thus facilitating a smooth learning process. Based on the experiments conducted on 73 binary classification datasets, our method yields the best test AUC in 25 datasets among its relevant competitors.