Selective Cascade of Residual ExtraTrees
This work addresses the need for more robust and explainable machine learning methods, though it appears incremental as it builds upon existing tree-based techniques.
The authors tackled the problem of improving prediction accuracy and reducing generalization errors in tree-based ensemble methods by proposing SCORE, which achieved comparable or superior performance against several benchmark models including ExtraTrees, random forest, gradient boosting machine, and neural networks.
We propose a novel tree-based ensemble method named Selective Cascade of Residual ExtraTrees (SCORE). SCORE draws inspiration from representation learning, incorporates regularized regression with variable selection features, and utilizes boosting to improve prediction and reduce generalization errors. We also develop a variable importance measure to increase the explainability of SCORE. Our computer experiments show that SCORE provides comparable or superior performance in prediction against ExtraTrees, random forest, gradient boosting machine, and neural networks; and the proposed variable importance measure for SCORE is comparable to studied benchmark methods. Finally, the predictive performance of SCORE remains stable across hyper-parameter values, suggesting potential robustness to hyperparameter specification.