Are Classes Clusters?
This addresses the problem of evaluating sentence embeddings for clustering tasks, which is incremental as it applies existing models to new data without proposing new methods.
The paper investigates whether sentence embeddings from four models (Universal Sentence Encoder, Sentence-BERT, LASER, DeCLUTR) can map topic classes in text classification datasets (Amazon Reviews, News Category Dataset) to clusters in embedding space, finding that unsupervised classification performs better than random but far from perfect.
Sentence embedding models aim to provide general purpose embeddings for sentences. Most of the models studied in this paper claim to perform well on STS tasks - but they do not report on their suitability for clustering. This paper looks at four recent sentence embedding models (Universal Sentence Encoder (Cer et al., 2018), Sentence-BERT (Reimers and Gurevych, 2019), LASER (Artetxe and Schwenk, 2019), and DeCLUTR (Giorgi et al., 2020)). It gives a brief overview of the ideas behind their implementations. It then investigates how well topic classes in two text classification datasets (Amazon Reviews (Ni et al., 2019) and News Category Dataset (Misra, 2018)) map to clusters in their corresponding sentence embedding space. While the performance of the resulting classification model is far from perfect, it is better than random. This is interesting because the classification model has been constructed in an unsupervised way. The topic classes in these real life topic classification datasets can be partly reconstructed by clustering the corresponding sentence embeddings.