Decision Tree Induction Systems: A Bayesian Analysis
This provides a theoretical foundation for decision tree methods, which is incremental as it builds on existing systems without introducing new empirical results.
The paper tackles the problem of interpreting decision tree induction systems in noisy domains by developing a subjective Bayesian framework, arguing that these systems implicitly favor simpler hypotheses and perform greedy search, and suggests improvements.
Decision tree induction systems are being used for knowledge acquisition in noisy domains. This paper develops a subjective Bayesian interpretation of the task tackled by these systems and the heuristic methods they use. It is argued that decision tree systems implicitly incorporate a prior belief that the simpler (in terms of decision tree complexity) of two hypotheses be preferred, all else being equal, and that they perform a greedy search of the space of decision rules to find one in which there is strong posterior belief. A number of improvements to these systems are then suggested.