Total Variation Distance Meets Probabilistic Inference
This work provides an efficient method for probabilistic inference practitioners to estimate TV distances in complex graphical models, representing a significant advance beyond prior incremental results limited to simpler distributions.
The paper tackles the problem of approximating total variation distances between high-dimensional distributions by establishing a reduction to probabilistic inference, resulting in a fully polynomial randomized approximation scheme for distributions defined over Bayes nets of small treewidth, where previously only product distributions were tractable.
In this paper, we establish a novel connection between total variation (TV) distance estimation and probabilistic inference. In particular, we present an efficient, structure-preserving reduction from relative approximation of TV distance to probabilistic inference over directed graphical models. This reduction leads to a fully polynomial randomized approximation scheme (FPRAS) for estimating TV distances between same-structure distributions over any class of Bayes nets for which there is an efficient probabilistic inference algorithm. In particular, it leads to an FPRAS for estimating TV distances between distributions that are defined over a common Bayes net of small treewidth. Prior to this work, such approximation schemes only existed for estimating TV distances between product distributions. Our approach employs a new notion of $partial$ couplings of high-dimensional distributions, which might be of independent interest.