Refining Dimensions for Improving Clustering-based Cross-lingual Topic Models
This addresses cross-lingual topic modeling for multilingual NLP applications, but appears incremental as it refines an existing pipeline.
The paper tackled the problem of language-dependent dimensions in multilingual language models that degrade cross-lingual topic identification, and introduced an SVD-based dimension refinement component that generally outperformed state-of-the-art models on three datasets.
Recent works in clustering-based topic models perform well in monolingual topic identification by introducing a pipeline to cluster the contextualized representations. However, the pipeline is suboptimal in identifying topics across languages due to the presence of language-dependent dimensions (LDDs) generated by multilingual language models. To address this issue, we introduce a novel, SVD-based dimension refinement component into the pipeline of the clustering-based topic model. This component effectively neutralizes the negative impact of LDDs, enabling the model to accurately identify topics across languages. Our experiments on three datasets demonstrate that the updated pipeline with the dimension refinement component generally outperforms other state-of-the-art cross-lingual topic models.