Learning Transformations for Classification Forests
This work addresses the optimization of split functions in classification forests, offering a domain-specific improvement for machine learning practitioners.
The paper tackles the problem of improving classification forests by learning linear transformations at split nodes to enhance class separation and low-rank structure, resulting in improved performance as supported by theoretical and experimental results.
This work introduces a transformation-based learner model for classification forests. The weak learner at each split node plays a crucial role in a classification tree. We propose to optimize the splitting objective by learning a linear transformation on subspaces using nuclear norm as the optimization criteria. The learned linear transformation restores a low-rank structure for data from the same class, and, at the same time, maximizes the separation between different classes, thereby improving the performance of the split function. Theoretical and experimental results support the proposed framework.