Kernelized Complete Conditional Stein Discrepancy
This work addresses a problem for machine learning practitioners dealing with distribution comparison in high-dimensional settings, representing an incremental improvement over existing kernelized Stein discrepancies.
The paper tackles the challenge of comparing distributions in high dimensions by introducing kernelized complete conditional Stein discrepancies (KCC-SDs), which use complete conditionals to convert multivariate distributions into univariate ones, and demonstrates their efficacy through a goodness-of-fit test with higher power over baselines and application to assessing sample quality in Markov chain Monte Carlo.
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepancies (KSDs). While kernels make Stein discrepancies tractable, they pose several challenges in high dimensions. We introduce kernelized complete conditional Stein discrepancies (KCC-SDs). Complete conditionals turn a multivariate distribution into multiple univariate distributions. We show that KCC-SDs distinguish distributions. To show the efficacy of KCC-SDs in distinguishing distributions, we introduce a goodness-of-fit test using KCC-SDs. We empirically show that KCC-SDs have higher power over baselines and use KCC-SDs to assess sample quality in Markov chain Monte Carlo.