Local Adaptivity of Gradient Boosting in Histogram Transform Ensemble Learning
This work addresses regression tasks for data scientists by offering a locally adaptive method, though it appears incremental as it builds on existing gradient boosting and histogram transform techniques.
The authors tackled the problem of improving regression performance by developing an adaptive boosting histogram transform (ABHT) algorithm that filters regions with varying smoothness, achieving a convergence rate upper bound strictly smaller than the lower bound of parallel ensemble methods.
In this paper, we propose a gradient boosting algorithm called \textit{adaptive boosting histogram transform} (\textit{ABHT}) for regression to illustrate the local adaptivity of gradient boosting algorithms in histogram transform ensemble learning. From the theoretical perspective, when the target function lies in a locally Hölder continuous space, we show that our ABHT can filter out the regions with different orders of smoothness. Consequently, we are able to prove that the upper bound of the convergence rates of ABHT is strictly smaller than the lower bound of \textit{parallel ensemble histogram transform} (\textit{PEHT}). In the experiments, both synthetic and real-world data experiments empirically validate the theoretical results, which demonstrates the advantageous performance and local adaptivity of our ABHT.