Efficient Estimation and Evaluation of Prediction Rules in Semi-Supervised Settings under Stratified Sampling
This addresses the need for semi-supervised learning methods in real-world applications like electronic health records where labeled data is limited and sampling is non-random, though it is incremental as it builds on existing SSL frameworks.
The paper tackles the problem of evaluating prediction rules in semi-supervised learning under non-random stratified sampling, proposing a two-step method that imputes missing labels and augments imputations to ensure consistency, resulting in estimators that outperform supervised counterparts with efficiency gains as shown in asymptotic theory and numerical studies.
In many contemporary applications, large amounts of unlabeled data are readily available while labeled examples are limited. There has been substantial interest in semi-supervised learning (SSL) which aims to leverage unlabeled data to improve estimation or prediction. However, current SSL literature focuses primarily on settings where labeled data is selected randomly from the population of interest. Non-random sampling, while posing additional analytical challenges, is highly applicable to many real world problems. Moreover, no SSL methods currently exist for estimating the prediction performance of a fitted model under non-random sampling. In this paper, we propose a two-step SSL procedure for evaluating a prediction rule derived from a working binary regression model based on the Brier score and overall misclassification rate under stratified sampling. In step I, we impute the missing labels via weighted regression with nonlinear basis functions to account for nonrandom sampling and to improve efficiency. In step II, we augment the initial imputations to ensure the consistency of the resulting estimators regardless of the specification of the prediction model or the imputation model. The final estimator is then obtained with the augmented imputations. We provide asymptotic theory and numerical studies illustrating that our proposals outperform their supervised counterparts in terms of efficiency gain. Our methods are motivated by electronic health records (EHR) research and validated with a real data analysis of an EHR-based study of diabetic neuropathy.