A Simple but Effective Closed-form Solution for Extreme Multi-label Learning
This work addresses the problem of simplifying model design and tuning for researchers and practitioners in extreme multi-label learning, though it is incremental as it adapts an existing method to a new task.
The paper tackles the complexity of hyperparameter tuning and reimplementation in extreme multi-label learning by proposing a simple ridge regression method with a closed-form solution and a single hyperparameter, achieving performance comparable to or exceeding models with numerous hyperparameters and significantly improving prediction for low-frequency labels with a frequency-based weighting.
Extreme multi-label learning (XML) is a task of assigning multiple labels from an extremely large set of labels to each data instance. Many current high-performance XML models are composed of a lot of hyperparameters, which complicates the tuning process. Additionally, the models themselves are adapted specifically to XML, which complicates their reimplementation. To remedy this problem, we propose a simple method based on ridge regression for XML. The proposed method not only has a closed-form solution but also is composed of a single hyperparameter. Since there are no precedents on applying ridge regression to XML, this paper verified the performance of the method by using various XML benchmark datasets. Furthermore, we enhanced the prediction of low-frequency labels in XML, which hold informative content. This prediction is essential yet challenging because of the limited amount of data. Here, we employed a simple frequency-based weighting. This approach greatly simplifies the process compared with existing techniques. Experimental results revealed that it can achieve levels of performance comparable to, or even exceeding, those of models with numerous hyperparameters. Additionally, we found that the frequency-based weighting significantly improved the predictive performance for low-frequency labels, while requiring almost no changes in implementation. The source code for the proposed method is available on github at https://github.com/cars1015/XML-ridge.