Missing At Random as Covariate Shift: Correcting Bias in Iterative Imputation
This work addresses bias in imputation for machine learning applications, offering a novel correction method that is incremental over existing approaches.
The paper tackled bias in missing data imputation by formulating it as a risk minimization problem with covariate shift, and proposed a method to correct this bias using importance weights. The result showed reductions in root mean squared error by up to 7% and Wasserstein distance by up to 20% compared to unweighted methods.
Accurate imputation of missing data is critical to downstream machine learning performance. We formulate missing data imputation as a risk minimisation problem, which highlights a covariate shift between the observed and unobserved data distributions. This covariate shift induced bias is not accounted for by popular imputation methods and leads to suboptimal performance. In this paper, we derive theoretically valid importance weights that correct for the induced distributional bias. Furthermore, we propose a novel imputation algorithm that jointly estimates both the importance weights and imputation models, enabling bias correction throughout the imputation process. Empirical results across benchmark datasets show reductions in root mean squared error and Wasserstein distance of up to 7% and 20%, respectively, compared to otherwise identical unweighted methods.