SPLBoost: An Improved Robust Boosting Algorithm Based on Self-paced Learning
This work addresses robustness issues in Boosting algorithms for machine learning practitioners, but it is incremental as it builds on existing methods like LogitBoost and SavageBoost.
The authors tackled the problem of AdaBoost's sensitivity to noise and outliers by incorporating Self-paced Learning into the Boosting framework, resulting in SPLBoost, which shows improved robustness in experiments.
It is known that Boosting can be interpreted as a gradient descent technique to minimize an underlying loss function. Specifically, the underlying loss being minimized by the traditional AdaBoost is the exponential loss, which is proved to be very sensitive to random noise/outliers. Therefore, several Boosting algorithms, e.g., LogitBoost and SavageBoost, have been proposed to improve the robustness of AdaBoost by replacing the exponential loss with some designed robust loss functions. In this work, we present a new way to robustify AdaBoost, i.e., incorporating the robust learning idea of Self-paced Learning (SPL) into Boosting framework. Specifically, we design a new robust Boosting algorithm based on SPL regime, i.e., SPLBoost, which can be easily implemented by slightly modifying off-the-shelf Boosting packages. Extensive experiments and a theoretical characterization are also carried out to illustrate the merits of the proposed SPLBoost.