TMI: Thermodynamic inference of data manifolds
This work addresses a domain-specific challenge in statistical modeling and dimensionality reduction for researchers in physics and machine learning, offering a novel approach but with incremental extensions to existing thermodynamic frameworks.
The authors tackled the problem of approximating arbitrary probability distributions beyond the Gibbs-Boltzmann form by introducing TMI, a thermodynamic method that learns intensive and extensive variables to achieve interpretable dimensionality reduction, with applications demonstrated on real and artificial datasets.
The Gibbs-Boltzmann distribution offers a physically interpretable way to massively reduce the dimensionality of high dimensional probability distributions where the extensive variables are `features' and the intensive variables are `descriptors'. However, not all probability distributions can be modeled using the Gibbs-Boltzmann form. Here, we present TMI: TMI, {\bf T}hermodynamic {\bf M}anifold {\bf I}nference; a thermodynamic approach to approximate a collection of arbitrary distributions. TMI simultaneously learns from data intensive and extensive variables and achieves dimensionality reduction through a multiplicative, positive valued, and interpretable decomposition of the data. Importantly, the reduced dimensional space of intensive parameters is not homogeneous. The Gibbs-Boltzmann distribution defines an analytically tractable Riemannian metric on the space of intensive variables allowing us to calculate geodesics and volume elements. We discuss the applications of TMI with multiple real and artificial data sets. Possible extensions are discussed as well.