Canonical Correlation Forests
This provides a more effective and efficient tree ensemble method for machine learning practitioners dealing with correlated input data, though it is incremental in advancing oblique decision tree techniques.
The paper tackles the problem of classification and regression by introducing canonical correlation forests (CCFs), a decision tree ensemble method with hyperplane splits based on local canonical correlation coefficients, which outperforms axis-aligned random forests and other state-of-the-art methods, delivering improved predictive accuracy and faster training times, including outperforming 179 classifiers in a recent survey.
We introduce canonical correlation forests (CCFs), a new decision tree ensemble method for classification and regression. Individual canonical correlation trees are binary decision trees with hyperplane splits based on local canonical correlation coefficients calculated during training. Unlike axis-aligned alternatives, the decision surfaces of CCFs are not restricted to the coordinate system of the inputs features and therefore more naturally represent data with correlated inputs. CCFs naturally accommodate multiple outputs, provide a similar computational complexity to random forests, and inherit their impressive robustness to the choice of input parameters. As part of the CCF training algorithm, we also introduce projection bootstrapping, a novel alternative to bagging for oblique decision tree ensembles which maintains use of the full dataset in selecting split points, often leading to improvements in predictive accuracy. Our experiments show that, even without parameter tuning, CCFs out-perform axis-aligned random forests and other state-of-the-art tree ensemble methods on both classification and regression problems, delivering both improved predictive accuracy and faster training times. We further show that they outperform all of the 179 classifiers considered in a recent extensive survey.