A Gradient-Based Split Criterion for Highly Accurate and Transparent Model Trees
This addresses the problem of interpretability in machine learning for users needing transparent models, though it is incremental as it builds on existing model tree methods.
The paper tackles the trade-off between predictive accuracy and model transparency by proposing shallow model trees with a novel split criterion, achieving significantly higher predictive power than state-of-the-art model trees while maintaining simplicity.
Machine learning algorithms aim at minimizing the number of false decisions and increasing the accuracy of predictions. However, the high predictive power of advanced algorithms comes at the costs of transparency. State-of-the-art methods, such as neural networks and ensemble methods, often result in highly complex models that offer little transparency. We propose shallow model trees as a way to combine simple and highly transparent predictive models for higher predictive power without losing the transparency of the original models. We present a novel split criterion for model trees that allows for significantly higher predictive power than state-of-the-art model trees while maintaining the same level of simplicity. This novel approach finds split points which allow the underlying simple models to make better predictions on the corresponding data. In addition, we introduce multiple mechanisms to increase the transparency of the resulting trees.