Learning by Minimizing the Sum of Ranked Range
This work addresses the need for robust aggregation methods in machine learning, such as for loss functions, but it appears incremental as it builds on existing optimization techniques like the difference of convex algorithm.
The authors tackled the problem of aggregating individual values into learning objectives by introducing the sum of ranked range (SoRR) as a general framework, with empirical results showing effectiveness in binary and multi-label classification tasks using synthetic and real datasets.
In forming learning objectives, one oftentimes needs to aggregate a set of individual values to a single output. Such cases occur in the aggregate loss, which combines individual losses of a learning model over each training sample, and in the individual loss for multi-label learning, which combines prediction scores over all class labels. In this work, we introduce the sum of ranked range (SoRR) as a general approach to form learning objectives. A ranked range is a consecutive sequence of sorted values of a set of real numbers. The minimization of SoRR is solved with the difference of convex algorithm (DCA). We explore two applications in machine learning of the minimization of the SoRR framework, namely the AoRR aggregate loss for binary classification and the TKML individual loss for multi-label/multi-class classification. Our empirical results highlight the effectiveness of the proposed optimization framework and demonstrate the applicability of proposed losses using synthetic and real datasets.