On the Trade-off Between Consistency and Coverage in Multi-label Rule Learning Heuristics
This work addresses the lack of studies on rule learning heuristics for multi-label classification, which is important for interpretable models in domains requiring multi-label predictions, but it is incremental as it builds on existing single-label research.
The paper tackles the problem of selecting rule learning heuristics for multi-label classification, showing empirically that trading off consistency and coverage differently is crucial depending on the performance measure to optimize, and emphasizing the need for configurable learners.
Recently, several authors have advocated the use of rule learning algorithms to model multi-label data, as rules are interpretable and can be comprehended, analyzed, or qualitatively evaluated by domain experts. Many rule learning algorithms employ a heuristic-guided search for rules that model regularities contained in the training data and it is commonly accepted that the choice of the heuristic has a significant impact on the predictive performance of the learner. Whereas the properties of rule learning heuristics have been studied in the realm of single-label classification, there is no such work taking into account the particularities of multi-label classification. This is surprising, as the quality of multi-label predictions is usually assessed in terms of a variety of different, potentially competing, performance measures that cannot all be optimized by a single learner at the same time. In this work, we show empirically that it is crucial to trade off the consistency and coverage of rules differently, depending on which multi-label measure should be optimized by a model. Based on these findings, we emphasize the need for configurable learners that can flexibly use different heuristics. As our experiments reveal, the choice of the heuristic is not straight-forward, because a search for rules that optimize a measure locally does usually not result in a model that maximizes that measure globally.