Transforming Hidden States into Binary Semantic Features
This work addresses the problem of understanding the semantic representations within large language models for researchers interested in interpretability.
This paper proposes to re-employ distributional semantics, using Independent Component Analysis, to demonstrate that large language models encode semantic features within their hidden states.
Large language models follow a lineage of many NLP applications that were directly inspired by distributional semantics, but do not seem to be closely related to it anymore. In this paper, we propose to employ the distributional theory of meaning once again. Using Independent Component Analysis to overcome some of its challenging aspects, we show that large language models represent semantic features in their hidden states.