Learning Embeddings into Entropic Wasserstein Spaces
This addresses the problem of representing complex semantic structures in machine learning, offering a novel approach with potential applications in domains like word embeddings, though it appears incremental in advancing embedding techniques.
The paper tackles the limitation of Euclidean embeddings in capturing latent semantic structures by proposing embeddings as discrete probability distributions in Wasserstein spaces, showing they embed metric structures with smaller distortion than Euclidean equivalents and enable direct visualization without dimensionality reduction.
Euclidean embeddings of data are fundamentally limited in their ability to capture latent semantic structures, which need not conform to Euclidean spatial assumptions. Here we consider an alternative, which embeds data as discrete probability distributions in a Wasserstein space, endowed with an optimal transport metric. Wasserstein spaces are much larger and more flexible than Euclidean spaces, in that they can successfully embed a wider variety of metric structures. We exploit this flexibility by learning an embedding that captures semantic information in the Wasserstein distance between embedded distributions. We examine empirically the representational capacity of our learned Wasserstein embeddings, showing that they can embed a wide variety of metric structures with smaller distortion than an equivalent Euclidean embedding. We also investigate an application to word embedding, demonstrating a unique advantage of Wasserstein embeddings: We can visualize the high-dimensional embedding directly, since it is a probability distribution on a low-dimensional space. This obviates the need for dimensionality reduction techniques like t-SNE for visualization.