A Robust Classifier Under Missing-Not-At-Random Sample Selection Bias
This addresses a specific bias issue in machine learning for applications with non-random data sampling, offering an incremental improvement over prior statistical approaches.
The paper tackles the problem of training classifiers under missing-not-at-random (MNAR) sample selection bias, where existing methods like Greene's are ineffective, and proposes BiasCorr, which modifies the training set to achieve robust performance, outperforming state-of-the-art methods on real-world datasets.
The shift between the training and testing distributions is commonly due to sample selection bias, a type of bias caused by non-random sampling of examples to be included in the training set. Although there are many approaches proposed to learn a classifier under sample selection bias, few address the case where a subset of labels in the training set are missing-not-at-random (MNAR) as a result of the selection process. In statistics, Greene's method formulates this type of sample selection with logistic regression as the prediction model. However, we find that simply integrating this method into a robust classification framework is not effective for this bias setting. In this paper, we propose BiasCorr, an algorithm that improves on Greene's method by modifying the original training set in order for a classifier to learn under MNAR sample selection bias. We provide theoretical guarantee for the improvement of BiasCorr over Greene's method by analyzing its bias. Experimental results on real-world datasets demonstrate that BiasCorr produces robust classifiers and can be extended to outperform state-of-the-art classifiers that have been proposed to train under sample selection bias.