Large data limits and scaling laws for tSNE
This addresses a theoretical limitation in a widely-used dimension reduction tool, providing a foundation for more stable large-scale applications.
The paper tackled the issue of tSNE embeddings lacking a consistent limit in large data regimes by identifying a continuum limit of the objective function and proposing a rescaled model that achieves a consistent limit.
This work considers large-data asymptotics for t-distributed stochastic neighbor embedding (tSNE), a widely-used non-linear dimension reduction algorithm. We identify an appropriate continuum limit of the tSNE objective function, which can be viewed as a combination of a kernel-based repulsion and an asymptotically-vanishing Laplacian-type regularizer. As a consequence, we show that embeddings of the original tSNE algorithm cannot have any consistent limit as $n \to \infty$. We propose a rescaled model which mitigates the asymptotic decay of the attractive energy, and which does have a consistent limit.