SCDTour: Embedding Axis Ordering and Merging for Interpretable Semantic Change Detection
This addresses the trade-off between interpretability and performance in SCD for NLP researchers, but it is incremental as it builds on existing embedding methods.
The paper tackles the problem in Semantic Change Detection (SCD) where improving interpretability often reduces performance, by proposing SCDTour, a method that orders and merges interpretable axes to preserve SCD performance while maintaining high interpretability, achieving comparable or improved performance against original embeddings.
In Semantic Change Detection (SCD), it is a common problem to obtain embeddings that are both interpretable and high-performing. However, improving interpretability often leads to a loss in the SCD performance, and vice versa. To address this problem, we propose SCDTour, a method that orders and merges interpretable axes to alleviate the performance degradation of SCD. SCDTour considers both (a) semantic similarity between axes in the embedding space, as well as (b) the degree to which each axis contributes to semantic change. Experimental results show that SCDTour preserves performance in semantic change detection while maintaining high interpretability. Moreover, agglomerating the sorted axes produces a more refined set of word senses, which achieves comparable or improved performance against the original full-dimensional embeddings in the SCD task. These findings demonstrate that SCDTour effectively balances interpretability and SCD performance, enabling meaningful interpretation of semantic shifts through a small number of refined axes. Source code is available at https://github.com/LivNLP/svp-tour .