Parametric t-Distributed Stochastic Exemplar-centered Embedding
This addresses a specific bottleneck in data visualization for researchers and practitioners by providing a more stable and efficient embedding method, though it is incremental as it builds upon existing parametric embedding techniques.
The paper tackled the sensitivity of parametric t-SNE to hyperparameters like batch size and perplexity, which cause unstable embeddings, and introduced a method that uses exemplars and noise contrastive samples to achieve linear complexity and improved robustness, outperforming pt-SNE on benchmark datasets in robustness, visual effects, and quantitative evaluations.
Parametric embedding methods such as parametric t-SNE (pt-SNE) have been widely adopted for data visualization and out-of-sample data embedding without further computationally expensive optimization or approximation. However, the performance of pt-SNE is highly sensitive to the hyper-parameter batch size due to conflicting optimization goals, and often produces dramatically different embeddings with different choices of user-defined perplexities. To effectively solve these issues, we present parametric t-distributed stochastic exemplar-centered embedding methods. Our strategy learns embedding parameters by comparing given data only with precomputed exemplars, resulting in a cost function with linear computational and memory complexity, which is further reduced by noise contrastive samples. Moreover, we propose a shallow embedding network with high-order feature interactions for data visualization, which is much easier to tune but produces comparable performance in contrast to a deep neural network employed by pt-SNE. We empirically demonstrate, using several benchmark datasets, that our proposed methods significantly outperform pt-SNE in terms of robustness, visual effects, and quantitative evaluations.