Improved prediction rule ensembling through model-based data generation
This work addresses the need for more interpretable and accurate prediction models in machine learning, though it appears incremental as it builds on existing PRE methods.
The paper tackled the problem of improving prediction rule ensembles (PRE) by using surrogate models for model-based data generation to enhance the Lasso regression step, resulting in substantially improved sparsity while retaining predictive accuracy, especially with a nested surrogacy approach.
Prediction rule ensembles (PRE) provide interpretable prediction models with relatively high accuracy.PRE obtain a large set of decision rules from a (boosted) decision tree ensemble, and achieves sparsitythrough application of Lasso-penalized regression. This article examines the use of surrogate modelsto improve performance of PRE, wherein the Lasso regression is trained with the help of a massivedataset generated by the (boosted) decision tree ensemble. This use of model-based data generationmay improve the stability and consistency of the Lasso step, thus leading to improved overallperformance. We propose two surrogacy approaches, and evaluate them on simulated and existingdatasets, in terms of sparsity and predictive accuracy. The results indicate that the use of surrogacymodels can substantially improve the sparsity of PRE, while retaining predictive accuracy, especiallythrough the use of a nested surrogacy approach.