Sum decomposition of divergence into three divergences
This work provides a theoretical decomposition framework for divergence functions, which is incremental and relevant to machine learning, statistics, and signal processing.
The paper demonstrates that symmetric Bregman divergences can be decomposed into sums of Jensen divergences and Bregman divergences, and extends this to include f-divergences explicitly in another decomposition.
Divergence functions play a key role as to measure the discrepancy between two points in the field of machine learning, statistics and signal processing. Well-known divergences are the Bregman divergences, the Jensen divergences and the f-divergences. In this paper, we show that the symmetric Bregman divergence can be decomposed into the sum of two types of Jensen divergences and the Bregman divergence. Furthermore, applying this result, we show another sum decomposition of divergence is possible which includes f-divergences explicitly.