Making Reliable and Flexible Decisions in Long-tailed Classification
This work addresses the critical issue of error risks in long-tailed classification for applications like medical diagnosis, where misclassifications can have severe impacts, though it is incremental in focusing on decision-making aspects rather than a paradigm shift.
The paper tackles the problem of long-tailed classification by addressing the risk of certain errors, such as misclassifying tail classes as head classes, which can have serious consequences. It introduces the RF-DLC framework, which uses Bayesian Decision Theory and a variational optimization strategy to achieve reliable predictions, with experiments on real-world tasks showing improved performance in tail-sensitivity risk metrics like False Head Rate.
Long-tailed classification is challenging due to its heavy imbalance in class probabilities. While existing methods often focus on overall accuracy or accuracy for tail classes, they overlook a critical aspect: certain types of errors can carry greater risks than others in real-world long-tailed problems. For example, misclassifying patients (a tail class) as healthy individuals (a head class) entails far more serious consequences than the reverse scenario. To address this critical issue, we introduce Making Reliable and Flexible Decisions in Long-tailed Classification (RF-DLC), a novel framework aimed at reliable predictions in long-tailed problems. Leveraging Bayesian Decision Theory, we introduce an integrated gain to seamlessly combine long-tailed data distributions and the decision-making procedure. We further propose an efficient variational optimization strategy for the decision risk objective. Our method adapts readily to diverse utility matrices, which can be designed for specific tasks, ensuring its flexibility for different problem settings. In empirical evaluation, we design a new metric, False Head Rate, to quantify tail-sensitivity risk, along with comprehensive experiments on multiple real-world tasks, including large-scale image classification and uncertainty quantification, to demonstrate the reliability and flexibility of our method.