Adaptive Optimization for Prediction with Missing Data
This addresses the challenge of handling missing data in predictive modeling, particularly for scenarios where data is not missing at random, offering a more effective alternative to traditional imputation pipelines.
The paper tackles the problem of training predictive models on data with missing entries by proposing adaptive linear regression models that adjust coefficients based on observed features, achieving a 2-10% improvement in out-of-sample accuracy in non-random missing data settings.
When training predictive models on data with missing entries, the most widely used and versatile approach is a pipeline technique where we first impute missing entries and then compute predictions. In this paper, we view prediction with missing data as a two-stage adaptive optimization problem and propose a new class of models, adaptive linear regression models, where the regression coefficients adapt to the set of observed features. We show that some adaptive linear regression models are equivalent to learning an imputation rule and a downstream linear regression model simultaneously instead of sequentially. We leverage this joint-impute-then-regress interpretation to generalize our framework to non-linear models. In settings where data is strongly not missing at random, our methods achieve a 2-10% improvement in out-of-sample accuracy.