Binary Classification from Positive Data with Skewed Confidence
This work addresses a practical issue in weakly-supervised learning for binary classification, where skewed confidence from biased annotations can degrade model performance, making it incremental by building on existing Pconf classification methods.
The paper tackles the problem of binary classification from positive data with skewed confidence, which deteriorates performance, by introducing a parameterized model for skewed confidence and a hyperparameter selection method that cancels out its negative impact, assuming prior knowledge of the misclassification rate of positive samples. The method's effectiveness is demonstrated through synthetic experiments with linear models, benchmark problems with neural networks, and a real-world application to drivers' drowsiness prediction.
Positive-confidence (Pconf) classification [Ishida et al., 2018] is a promising weakly-supervised learning method which trains a binary classifier only from positive data equipped with confidence. However, in practice, the confidence may be skewed by bias arising in an annotation process. The Pconf classifier cannot be properly learned with skewed confidence, and consequently, the classification performance might be deteriorated. In this paper, we introduce the parameterized model of the skewed confidence, and propose the method for selecting the hyperparameter which cancels out the negative impact of skewed confidence under the assumption that we have the misclassification rate of positive samples as a prior knowledge. We demonstrate the effectiveness of the proposed method through a synthetic experiment with simple linear models and benchmark problems with neural network models. We also apply our method to drivers' drowsiness prediction to show that it works well with a real-world problem where confidence is obtained based on manual annotation.