CART-ELC: Oblique Decision Tree Induction via Exhaustive Search
This addresses the problem of improving classification performance and interpretability for small datasets in machine learning, but it is incremental as it builds on existing oblique tree methods.
The paper tackled the computational challenges of exhaustive search for oblique splits in decision trees by introducing CART-ELC, which restricts the search space, and found that it achieves competitive performance on small datasets with statistically significant accuracy improvements and simpler trees.
Oblique decision trees have attracted attention due to their potential for improved classification performance over traditional axis-aligned decision trees. However, methods that rely on exhaustive search to find oblique splits face computational challenges. As a result, they have not been widely explored. We introduce a novel algorithm, Classification and Regression Tree - Exhaustive Linear Combinations (CART-ELC), for inducing oblique decision trees that performs an exhaustive search on a restricted set of hyperplanes. We then investigate the algorithm's computational complexity and its predictive capabilities. Our results demonstrate that CART-ELC consistently achieves competitive performance on small datasets, often yielding statistically significant improvements in classification accuracy relative to existing decision tree induction algorithms, while frequently producing shallower, simpler, and thus more interpretable trees.