Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection
This addresses a specific problem in machine learning for researchers and practitioners using autoencoders for anomaly detection, though it is incremental as it builds on known limitations.
The paper tackles the failure of autoencoder-based out-of-distribution detection using Bernoulli likelihood, analyzing the issue and proposing Bayesian autoencoders and alternative distributions as fixes, with results showing improved performance on datasets like FashionMNIST vs MNIST.
After an autoencoder (AE) has learnt to reconstruct one dataset, it might be expected that the likelihood on an out-of-distribution (OOD) input would be low. This has been studied as an approach to detect OOD inputs. Recent work showed this intuitive approach can fail for the dataset pairs FashionMNIST vs MNIST. This paper suggests this is due to the use of Bernoulli likelihood and analyses why this is the case, proposing two fixes: 1) Compute the uncertainty of likelihood estimate by using a Bayesian version of the AE. 2) Use alternative distributions to model the likelihood.