The more polypersonal the better -- a short look on space geometry of fine-tuned layers
This is an incremental study on interpretability for language models, potentially benefiting researchers in NLP and linguistics.
The paper analyzes how adding grammatical modules (polypersonality) to BERT affects its internal representations, finding that a single grammatical layer separates new and old grammatical systems and improves perplexity metrics.
The interpretation of deep learning models is a rapidly growing field, with particular interest in language models. There are various approaches to this task, including training simpler models to replicate neural network predictions and analyzing the latent space of the model. The latter method allows us to not only identify patterns in the model's decision-making process, but also understand the features of its internal structure. In this paper, we analyze the changes in the internal representation of the BERT model when it is trained with additional grammatical modules and data containing new grammatical structures (polypersonality). We find that adding a single grammatical layer causes the model to separate the new and old grammatical systems within itself, improving the overall performance on perplexity metrics.