Big Data Regression Using Tree Based Segmentation
This addresses the problem of scaling regression for large datasets in applications requiring both predictive accuracy and model interpretability, though it is incremental.
The paper tackles scaling regression to large datasets by proposing a two-step approach: first segmenting data with a regression tree, then fitting separate regression models to each segment. The method achieved predictive performance matching Gradient Boosted Trees while offering better interpretability.
Scaling regression to large datasets is a common problem in many application areas. We propose a two step approach to scaling regression to large datasets. Using a regression tree (CART) to segment the large dataset constitutes the first step of this approach. The second step of this approach is to develop a suitable regression model for each segment. Since segment sizes are not very large, we have the ability to apply sophisticated regression techniques if required. A nice feature of this two step approach is that it can yield models that have good explanatory power as well as good predictive performance. Ensemble methods like Gradient Boosted Trees can offer excellent predictive performance but may not provide interpretable models. In the experiments reported in this study, we found that the predictive performance of the proposed approach matched the predictive performance of Gradient Boosted Trees.