Gaussian Mixture Reduction with Composite Transportation Divergence
This work addresses a key bottleneck in recursive density approximation for applications like Bayesian filtering, offering a flexible and theoretically sound solution that bridges optimization-based and clustering-based techniques.
The paper tackles the problem of Gaussian mixture reduction, where high-order Gaussian mixtures become intractable for inference, by proposing a novel optimization-based method using composite transportation divergence. The result is a unified framework that generalizes existing clustering-based methods, with theoretical convergence guarantees and demonstrated efficiency in empirical experiments.
Gaussian mixtures are widely used for approximating density functions in various applications such as density estimation, belief propagation, and Bayesian filtering. These applications often utilize Gaussian mixtures as initial approximations that are updated recursively. A key challenge in these recursive processes stems from the exponential increase in the mixture's order, resulting in intractable inference. To overcome the difficulty, the Gaussian mixture reduction (GMR), which approximates a high order Gaussian mixture by one with a lower order, can be used. Although existing clustering-based methods are known for their satisfactory performance and computational efficiency, their convergence properties and optimal targets remain unknown. In this paper, we propose a novel optimization-based GMR method based on composite transportation divergence (CTD). We develop a majorization-minimization algorithm for computing the reduced mixture and establish its theoretical convergence under general conditions. Furthermore, we demonstrate that many existing clustering-based methods are special cases of ours, effectively bridging the gap between optimization-based and clustering-based techniques. Our unified framework empowers users to select the most appropriate cost function in CTD to achieve superior performance in their specific applications. Through extensive empirical experiments, we demonstrate the efficiency and effectiveness of our proposed method, showcasing its potential in various domains.