A Kernel Stein Test for Comparing Latent Variable Models
This work addresses the challenge of model comparison for researchers and practitioners using latent variable models, such as in machine learning and statistics, but it is incremental as it extends existing kernel Stein discrepancy tests to a more general class.
The paper tackles the problem of comparing the goodness of fit between two latent variable models with intractable marginal distributions, by proposing a kernel-based nonparametric test that generalizes kernel Stein discrepancy tests to handle latent variables, resulting in a test with controlled type-I error and outperforming a baseline method in cases with low-dimensional latent structure and high-dimensional observations.
We propose a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables, such that the marginal distribution of the observed variables is intractable. The proposed test generalizes the recently proposed kernel Stein discrepancy (KSD) tests (Liu et al., 2016, Chwialkowski et al., 2016, Yang et al., 2018) to the case of latent variable models, a much more general class than the fully observed models treated previously. The new test, with a properly calibrated threshold, has a well-controlled type-I error. In the case of certain models with low-dimensional latent structure and high-dimensional observations, our test significantly outperforms the relative Maximum Mean Discrepancy test, which is based on samples from the models and does not exploit the latent structure.