t-SNE, Forceful Colorings and Mean Field Limits
This work provides theoretical insights into t-SNE behavior for homogeneous data, potentially improving interpretability of force-based methods, but is incremental as it builds on existing techniques.
The paper introduces forceful colorings, a method to extract additional features from force-based dimensionality reduction techniques like t-SNE by analyzing force vectors at equilibrium, and derives a mean-field model predicting that t-SNE embeddings of homogeneous clusters form thin annuli with diameters scaling as ~k^{-1/4} n^{-1/4}, supported by numerical results.
t-SNE is one of the most commonly used force-based nonlinear dimensionality reduction methods. This paper has two contributions: the first is forceful colorings, an idea that is also applicable to other force-based methods (UMAP, ForceAtlas2,...). In every equilibrium, the attractive and repulsive forces acting on a particle cancel out: however, both the size and the direction of the attractive (or repulsive) forces acting on a particle are related to its properties: the force vector can serve as an additional feature. Secondly, we analyze the case of t-SNE acting on a single homogeneous cluster (modeled by affinities coming from the adjacency matrix of a random k-regular graph); we derive a mean-field model that leads to interesting questions in classical calculus of variations. The model predicts that, in the limit, the t-SNE embedding of a single perfectly homogeneous cluster is not a point but a thin annulus of diameter $\sim k^{-1/4} n^{-1/4}$. This is supported by numerical results. The mean field ansatz extends to other force-based dimensionality reduction methods.