Exploring Interpretability of Independent Components of Word Embeddings with Automated Word Intruder Test
This work addresses interpretability in natural language processing for researchers and practitioners, offering an incremental improvement by automating an existing test.
The paper tackled the problem of interpreting word embeddings by applying Independent Component Analysis (ICA) to find semantic features, showing that most components represent such features and enabling word searches based on combinations. It proposed an automated word intruder test using large language models as a fast, inexpensive method to quantify interpretability without human effort.
Independent Component Analysis (ICA) is an algorithm originally developed for finding separate sources in a mixed signal, such as a recording of multiple people in the same room speaking at the same time. Unlike Principal Component Analysis (PCA), ICA permits the representation of a word as an unstructured set of features, without any particular feature being deemed more significant than the others. In this paper, we used ICA to analyze word embeddings. We have found that ICA can be used to find semantic features of the words, and these features can easily be combined to search for words that satisfy the combination. We show that most of the independent components represent such features. To quantify the interpretability of the components, we use the word intruder test, performed both by humans and by large language models. We propose to use the automated version of the word intruder test as a fast and inexpensive way of quantifying vector interpretability without the need for human effort.