Asymmetric Impurity Functions, Class Weighting, and Optimal Splits for Binary Classification Trees
This provides theoretical insights for practitioners building decision trees with class imbalance, though it appears incremental to existing impurity function theory.
The paper investigates how modifying impurity functions to be asymmetric affects optimal splits in binary classification trees, showing this biases splits to isolate specific classes and proving class weighting is equivalent to a specific impurity transformation.
We investigate how asymmetrizing an impurity function affects the choice of optimal node splits when growing a decision tree for binary classification. In particular, we relax the usual axioms of an impurity function and show how skewing an impurity function biases the optimal splits to isolate points of a particular class when splitting a node. We give a rigorous definition of this notion, then give a necessary and sufficient condition for such a bias to hold. We also show that the technique of class weighting is equivalent to applying a specific transformation to the impurity function, and tie all these notions together for a class of impurity functions that includes the entropy and Gini impurity. We also briefly discuss cost-insensitive impurity functions and give a characterization of such functions.