Risk-based Calibration for Generative Classifiers
This addresses the gap between generative approaches and supervised classification objectives for users of generative classifiers like naïve Bayes and quadratic discriminant analysis, though it is incremental as it refines existing methods.
The paper tackled the problem that generative classifiers are not directly optimized for classification error, proposing risk-based calibration (RC) to iteratively adjust the joint probability distribution based on the 0-1 loss, which significantly improved training and generalization error on 20 datasets compared to closed-form methods.
Generative classifiers are constructed on the basis of a joint probability distribution and are typically learned using closed-form procedures that rely on data statistics and maximize scores related to data fitting. However, these scores are not directly linked to supervised classification metrics such as the error, i.e., the expected 0-1 loss. To address this limitation, we propose a learning procedure called risk-based calibration (RC) that iteratively refines the generative classifier by adjusting its joint probability distribution according to the 0-1 loss in training samples. This is achieved by reinforcing data statistics associated with the true classes while weakening those of incorrect classes. As a result, the classifier progressively assigns higher probability to the correct labels, improving its training error. Results on 20 heterogeneous datasets using both naïve Bayes and quadratic discriminant analysis show that RC significantly outperforms closed-form learning procedures in terms of both training error and generalization error. In this way, RC bridges the gap between traditional generative approaches and learning procedures guided by performance measures, ensuring a closer alignment with supervised classification objectives.