Bigeminal Priors Variational auto-encoder
This work addresses the OOD detection problem in unsupervised learning for machine learning practitioners, offering an incremental improvement over existing VAE methods.
The authors tackled the problem of variational auto-encoders (VAEs) incorrectly assigning higher likelihoods to out-of-distribution (OOD) samples than in-distribution ones by introducing the Bigeminal Priors VAE (BPVAE), which uses two independent priors to enhance robustness and improve OOD detection, with quantitative results showing better generalization and stronger robustness compared to standard VAEs.
Variational auto-encoders (VAEs) are an influential and generally-used class of likelihood-based generative models in unsupervised learning. The likelihood-based generative models have been reported to be highly robust to the out-of-distribution (OOD) inputs and can be a detector by assuming that the model assigns higher likelihoods to the samples from the in-distribution (ID) dataset than an OOD dataset. However, recent works reported a phenomenon that VAE recognizes some OOD samples as ID by assigning a higher likelihood to the OOD inputs compared to the one from ID. In this work, we introduce a new model, namely Bigeminal Priors Variational auto-encoder (BPVAE), to address this phenomenon. The BPVAE aims to enhance the robustness of the VAEs by combing the power of VAE with the two independent priors that belong to the training dataset and simple dataset, which complexity is lower than the training dataset, respectively. BPVAE learns two datasets'features, assigning a higher likelihood for the training dataset than the simple dataset. In this way, we can use BPVAE's density estimate for detecting the OOD samples. Quantitative experimental results suggest that our model has better generalization capability and stronger robustness than the standard VAEs, proving the effectiveness of the proposed approach of hybrid learning by collaborative priors. Overall, this work paves a new avenue to potentially overcome the OOD problem via multiple latent priors modeling.