Controversy Rules - Discovering Regions Where Classifiers (Dis-)Agree Exceptionally
This work provides a method for evaluating classifier agreement/disagreement, which is incremental as it applies an existing framework to classification tasks.
The paper tackles the problem of identifying regions where multiple classifiers show high disagreement, using an algorithm based on Exceptional Model Mining to analyze public datasets and reveal both known and previously unreported insights about classification tasks.
Finding regions for which there is higher controversy among different classifiers is insightful with regards to the domain and our models. Such evaluation can falsify assumptions, assert some, or also, bring to the attention unknown phenomena. The present work describes an algorithm, which is based on the Exceptional Model Mining framework, and enables that kind of investigations. We explore several public datasets and show the usefulness of this approach in classification tasks. We show in this paper a few interesting observations about those well explored datasets, some of which are general knowledge, and other that as far as we know, were not reported before.