Symmetrizing Bregman Divergence on the Cone of Positive Definite Matrices: Which Mean to Use and Why
For researchers working with positive definite matrices in machine learning and optimization, this work offers a principled guide to selecting means for symmetrized divergences, though it is largely theoretical and incremental.
This paper derives variational principles for symmetrizing Bregman divergences on positive definite matrices, showing that the forward symmetrization yields the arithmetic mean and the reverse symmetrization yields means such as arithmetic, log-Euclidean, or harmonic depending on the mirror map. The results provide a theoretical foundation for choosing the appropriate mean in practice.
This work uncovers variational principles behind symmetrizing the Bregman divergences induced by generic mirror maps over the cone of positive definite matrices. We show that computing the canonical means for this symmetrization can be posed as minimizing the desired symmetrized divergences over a set of mean functionals defined axiomatically to satisfy certain properties. For the forward symmetrization, we prove that the arithmetic mean over the primal space is canonical for any mirror map over the positive definite cone. For the reverse symmetrization, we show that the canonical mean is the arithmetic mean over the dual space, pulled back to the primal space. Applying this result to three common mirror maps used in practice, we show that the canonical means for reverse symmetrization, in those cases, turn out to be the arithmetic, log-Euclidean and harmonic means. Our results improve understanding of existing symmetrization practices in the literature, and can be seen as a navigational chart to help decide which mean to use when.