From Kinetic Theory to AI: a Rediscovery of High-Dimensional Divergences and Their Properties
This is an incremental review that connects kinetic theory concepts to ML/AI, potentially aiding researchers in selecting divergence measures.
The paper reviews divergence measures from kinetic theory, such as the Kullback-Leibler divergence, and explores their theoretical foundations and potential applications in machine learning and AI, without presenting new experimental results or concrete numbers.
Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic theory as a measure of relative entropy between probability distributions. Just as in machine learning, the ability to quantify the proximity of probability distributions plays a central role in kinetic theory. In this paper, we present a comparative review of divergence measures rooted in kinetic theory, highlighting their theoretical foundations and exploring their potential applications in machine learning and artificial intelligence.