Probabilistic Transformers

This offers a theoretical insight for researchers in machine learning, but it is incremental as it reinterprets an existing method without new empirical results.

The paper demonstrates that Transformers can be interpreted as Maximum Posterior Probability estimators for Mixtures of Gaussian Models, providing a probabilistic perspective and suggesting extensions to other probabilistic scenarios.