Generalised Spherical Text Embedding
This work addresses the need for more adaptable text embeddings in natural language processing, though it appears incremental as it builds on existing matrix-based and manifold optimization techniques.
The paper tackles the problem of flexible text embedding representation by jointly encoding words and paragraphs as matrices with unit Frobenius norm, using a novel similarity metric and manifold optimization, resulting in improved performance on document classification, clustering, and semantic similarity benchmarks.
This paper aims to provide an unsupervised modelling approach that allows for a more flexible representation of text embeddings. It jointly encodes the words and the paragraphs as individual matrices of arbitrary column dimension with unit Frobenius norm. The representation is also linguistically motivated with the introduction of a novel similarity metric. The proposed modelling and the novel similarity metric exploits the matrix structure of embeddings. We then go on to show that the same matrices can be reshaped into vectors of unit norm and transform our problem into an optimization problem over the spherical manifold. We exploit manifold optimization to efficiently train the matrix embeddings. We also quantitatively verify the quality of our text embeddings by showing that they demonstrate improved results in document classification, document clustering, and semantic textual similarity benchmark tests.