When is the majority-vote classifier beneficial?
This provides theoretical insights for ensemble learning practitioners by clarifying conditions for majority-vote effectiveness, though it is incremental as it builds on prior weak classifier theory.
The paper tackles the problem of determining when a majority-vote classifier improves over individual weak classifiers in binary classification, discovering a phase-transition phenomenon where it requires an average true positive rate of at least 50% and an average false positive rate of at most 50% to be beneficial.
In his seminal work, Schapire (1990) proved that weak classifiers could be improved to achieve arbitrarily high accuracy, but he never implied that a simple majority-vote mechanism could always do the trick. By comparing the asymptotic misclassification error of the majority-vote classifier with the average individual error, we discover an interesting phase-transition phenomenon. For binary classification with equal prior probabilities, our result implies that, for the majority-vote mechanism to work, the collection of weak classifiers must meet the minimum requirement of having an average true positive rate of at least 50% and an average false positive rate of at most 50%.