JSRT: James-Stein Regression Tree
This is an incremental improvement for regression tree methods in machine learning, addressing a specific bottleneck in performance.
The paper tackles the problem of regression trees neglecting global information by proposing JSRT, which incorporates global mean information using the James-Stein estimator during construction and prediction, showing effectiveness and superiority over other methods in experiments on benchmark datasets.
Regression tree (RT) has been widely used in machine learning and data mining community. Given a target data for prediction, a regression tree is first constructed based on a training dataset before making prediction for each leaf node. In practice, the performance of RT relies heavily on the local mean of samples from an individual node during the tree construction/prediction stage, while neglecting the global information from different nodes, which also plays an important role. To address this issue, we propose a novel regression tree, named James-Stein Regression Tree (JSRT) by considering global information from different nodes. Specifically, we incorporate the global mean information based on James-Stein estimator from different nodes during the construction/predicton stage. Besides, we analyze the generalization error of our method under the mean square error (MSE) metric. Extensive experiments on public benchmark datasets verify the effectiveness and efficiency of our method, and demonstrate the superiority of our method over other RT prediction methods.