Geometric Implications of the Naive Bayes Assumption
This work addresses a gap in understanding for the UAI community regarding the limitations and capabilities of Naive Bayes networks, which is incremental as it builds on prior results.
The paper tackles the problem of understanding the geometric implications of the Naive Bayes assumption, extending the known result of hyperplane separability from binary to m-ary observations and analyzing the effects of observation dependencies on decision surfaces.
A naive (or Idiot) Bayes network is a network with a single hypothesis node and several observations that are conditionally independent given the hypothesis. We recently surveyed a number of members of the UAI community and discovered a general lack of understanding of the implications of the Naive Bayes assumption on the kinds of problems that can be solved by these networks. It has long been recognized [Minsky 61] that if observations are binary, the decision surfaces in these networks are hyperplanes. We extend this result (hyperplane separability) to Naive Bayes networks with m-ary observations. In addition, we illustrate the effect of observation-observation dependencies on decision surfaces. Finally, we discuss the implications of these results on knowledge acquisition and research in learning.