A Rigorous Link Between Self-Organizing Maps and Gaussian Mixture Models
This provides a theoretical foundation for using SOMs in probabilistic applications, but it is incremental as it builds on existing models without introducing new methods.
The paper tackles the problem of formally linking Self-Organizing Maps (SOMs) and Gaussian Mixture Models (GMMs) by showing that energy-based SOMs perform gradient descent to approximate GMM log-likelihood, enabling SOMs to be used as generative probabilistic models for tasks like outlier detection and sampling.
This work presents a mathematical treatment of the relation between Self-Organizing Maps (SOMs) and Gaussian Mixture Models (GMMs). We show that energy-based SOM models can be interpreted as performing gradient descent, minimizing an approximation to the GMM log-likelihood that is particularly valid for high data dimensionalities. The SOM-like decrease of the neighborhood radius can be understood as an annealing procedure ensuring that gradient descent does not get stuck in undesirable local minima. This link allows to treat SOMs as generative probabilistic models, giving a formal justification for using SOMs, e.g., to detect outliers, or for sampling.