Hierarchical confusion matrix for classification performance evaluation
This work addresses evaluation challenges for hierarchical classification problems, which is important for researchers and practitioners in fields like bioinformatics or document categorization, though it appears incremental as it extends existing confusion matrix concepts.
The authors tackled the problem of evaluating hierarchical classification models by proposing a hierarchical confusion matrix that extends flat evaluation measures to hierarchical structures. They demonstrated its applicability across various hierarchical classification types and validated it on three real-world benchmarks, showing reasonable and useful results.
In this work we propose a novel concept of a hierarchical confusion matrix, opening the door for popular confusion matrix based (flat) evaluation measures from binary classification problems, while considering the peculiarities of hierarchical classification problems. We develop the concept to a generalized form and prove its applicability to all types of hierarchical classification problems including directed acyclic graphs, multi path labelling, and non mandatory leaf node prediction. Finally, we use measures based on the novel confusion matrix to evaluate models within a benchmark for three real world hierarchical classification applications and compare the results to established evaluation measures. The results outline the reasonability of this approach and its usefulness to evaluate hierarchical classification problems. The implementation of hierarchical confusion matrix is available on GitHub.