Oblique Bayesian additive regression trees
This work addresses the problem of improving prediction accuracy in tree-based ensemble methods for machine learning practitioners, though it is incremental as it builds on prior oblique tree research.
The authors tackled the limitation of axis-aligned decision rules in Bayesian Additive Regression Trees (BART) by developing an oblique version that uses linear combinations of features, finding it competitive and sometimes much better than existing methods on synthetic and real-world datasets.
Current implementations of Bayesian Additive Regression Trees (BART) are based on axis-aligned decision rules that recursively partition the feature space using a single feature at a time. Several authors have demonstrated that oblique trees, whose decision rules are based on linear combinations of features, can sometimes yield better predictions than axis-aligned trees and exhibit excellent theoretical properties. We develop an oblique version of BART that leverages a data-adaptive decision rule prior that recursively partitions the feature space along random hyperplanes. Using several synthetic and real-world benchmark datasets, we systematically compared our oblique BART implementation to axis-aligned BART and other tree ensemble methods, finding that oblique BART was competitive with -- and sometimes much better than -- those methods.