Gradient boosting machine with partially randomized decision trees
This work addresses a specific issue in gradient boosting for regression, offering an incremental improvement by adapting existing tree randomization techniques.
The paper tackled the problem of discontinuity in gradient boosting regression functions due to sparse training data coverage, proposing partially randomized trees to address this and reduce computational complexity, with results demonstrated through synthetic and real data examples.
The gradient boosting machine is a powerful ensemble-based machine learning method for solving regression problems. However, one of the difficulties of its using is a possible discontinuity of the regression function, which arises when regions of training data are not densely covered by training points. In order to overcome this difficulty and to reduce the computational complexity of the gradient boosting machine, we propose to apply the partially randomized trees which can be regarded as a special case of the extremely randomized trees applied to the gradient boosting. The gradient boosting machine with the partially randomized trees is illustrated by means of many numerical examples using synthetic and real data.