Autoencoders and Probabilistic Inference with Missing Data: An Exact Solution for The Factor Analysis Case
This work addresses a specific technical issue in probabilistic inference for missing data in autoencoders, which is incremental as it builds on existing factor analysis and variational autoencoder frameworks.
The paper tackles the problem of handling missing data in variational autoencoders by providing an exact solution for the latent posterior distribution in the factor analysis model, showing that a different encoder network is needed for each missingness pattern and comparing approximations in experiments.
Latent variable models can be used to probabilistically "fill-in" missing data entries. The variational autoencoder architecture (Kingma and Welling, 2014; Rezende et al., 2014) includes a "recognition" or "encoder" network that infers the latent variables given the data variables. However, it is not clear how to handle missing data variables in this network. The factor analysis (FA) model is a basic autoencoder, using linear encoder and decoder networks. We show how to calculate exactly the latent posterior distribution for the factor analysis (FA) model in the presence of missing data, and note that this solution implies that a different encoder network is required for each pattern of missingness. We also discuss various approximations to the exact solution. Experiments compare the effectiveness of various approaches to filling in the missing data.