Robust Variational Inference
This work addresses robustness in variational inference for machine learning practitioners dealing with noisy data, but it is incremental as it builds on existing variational methods.
The paper tackles the problem of variational inference being sensitive to noisy training data by proposing a robust modification of the evidence and a lower bound, showing improved performance on synthetic noisy datasets from MNIST and OMNIGLOT and small gains on the original datasets.
Variational inference is a powerful tool for approximate inference. However, it mainly focuses on the evidence lower bound as variational objective and the development of other measures for variational inference is a promising area of research. This paper proposes a robust modification of evidence and a lower bound for the evidence, which is applicable when the majority of the training set samples are random noise objects. We provide experiments for variational autoencoders to show advantage of the objective over the evidence lower bound on synthetic datasets obtained by adding uninformative noise objects to MNIST and OMNIGLOT. Additionally, for the original MNIST and OMNIGLOT datasets we observe a small improvement over the non-robust evidence lower bound.