Alpha/Beta Divergences and Tweedie Models
This work provides a theoretical foundation for divergence methods in statistics and machine learning, but it is incremental as it extends known relationships to a broader class of models.
The paper establishes a probabilistic interpretation linking beta divergences to Tweedie distributions, showing they arise as special cases of Csiszár's f and Bregman divergences, generalizing the Gaussian-least squares relationship to Tweedie models.
We describe the underlying probabilistic interpretation of alpha and beta divergences. We first show that beta divergences are inherently tied to Tweedie distributions, a particular type of exponential family, known as exponential dispersion models. Starting from the variance function of a Tweedie model, we outline how to get alpha and beta divergences as special cases of Csiszár's $f$ and Bregman divergences. This result directly generalizes the well-known relationship between the Gaussian distribution and least squares estimation to Tweedie models and beta divergence minimization.