Amortized Variational Inference for Logistic Regression with Missing Covariates
This addresses a challenge in statistical inference and machine learning for classification tasks, offering a more efficient solution for handling missing data, though it appears incremental as it builds on existing deep generative models.
The authors tackled the problem of missing covariate data in logistic regression by proposing AV-LR, an amortized variational inference framework that integrates a generative model with an inference network, achieving estimation accuracy comparable to or better than state-of-the-art EM-like algorithms with significantly lower computational cost.
Missing covariate data pose a significant challenge to statistical inference and machine learning, particularly for classification tasks like logistic regression. Classical iterative approaches (EM, multiple imputation) are often computationally intensive, sensitive to high missingness rates, and limited in uncertainty propagation. Recent deep generative models based on VAEs show promise but rely on complex latent representations. We propose Amortized Variational Inference for Logistic Regression (AV-LR), a unified end-to-end framework for binary logistic regression with missing covariates. AV-LR integrates a probabilistic generative model with a simple amortized inference network, trained jointly by maximizing the evidence lower bound. Unlike competing methods, AV-LR performs inference directly in the space of missing data without additional latent variables, using a single inference network and a linear layer that jointly estimate regression parameters and the missingness mechanism. AV-LR achieves estimation accuracy comparable to or better than state-of-the-art EM-like algorithms, with significantly lower computational cost. It naturally extends to missing-not-at-random settings by explicitly modeling the missingness mechanism. Empirical results on synthetic and real-world datasets confirm its effectiveness and efficiency across various missing-data scenarios.