Group invariance of $f$-divergences and the Fisher--Rao distance
Provides a theoretical unification of invariance properties for divergence measures in statistical models with symmetries, benefiting statisticians and machine learning researchers working with group-structured data.
The paper proves that all f-divergences are invariant under group actions in transformation models, reducing them to functions of a maximal invariant (a double coset). This is applied to location-scale families, showing the same reduction for the Fisher-Rao distance.
Many statistical models have natural symmetries described by a group action. We study how such symmetries affect the comparison of two distributions. We work with a transformation model in which a group acts on both the sample space and the parameter space, and the densities transform with a multiplier. Under this assumption, we show that every $f$-divergence is invariant under the group action. As a consequence, an invariant divergence depends only on a maximal invariant of the pair of parameters. When the action on the parameter space is transitive, this maximal invariant is given by a double coset. We apply this result to multidimensional location-scale families, and we show that the same reduction applies to the Fisher--Rao distance.