Mixtures of Gaussians and Minimum Relative Entropy Techniques for Modeling Continuous Uncertainties
This work addresses the challenge of modeling continuous uncertainties in decision-making, offering an incremental improvement over existing discretization and fitting techniques.
The paper tackles the problem of approximating continuous random variables for probabilistic inference by proposing a method that uses minimum relative entropy to guide transformations and fitting with mixtures of Gaussians, resulting in an automated selection of mixture components that balances accuracy and computational cost.
Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is to fit analytically tractable continuous probability distributions. This approach has potential simplicity and accuracy advantages, especially if variables can be transformed first. This paper shows how a minimum relative entropy criterion can drive both transformation and fitting, illustrating with a power and logarithm family of transformations and mixtures of Gaussian (normal) distributions, which allow use of efficient influence diagram methods. The fitting procedure in this case is the well-known EM algorithm. The selection of the number of components in a fitted mixture distribution is automated with an objective that trades off accuracy and computational cost.