Wavelet Decomposition of Gradient Boosting
This is an incremental improvement for machine learning practitioners using gradient boosting in noisy or imbalanced data settings.
The paper tackles the problem of improving Stochastic Gradient Boosting by incorporating wavelet decomposition of trees, resulting in a method that generally outperforms existing approaches, especially in scenarios with class imbalance and mislabeling.
In this paper we introduce a significant improvement to the popular tree-based Stochastic Gradient Boosting algorithm using a wavelet decomposition of the trees. This approach is based on harmonic analysis and approximation theoretical elements, and as we show through extensive experimentation, our wavelet based method generally outperforms existing methods, particularly in difficult scenarios of class unbalance and mislabeling in the training data.