What Happens To BERT Embeddings During Fine-tuning?
This addresses the problem of understanding model adaptation for researchers in NLP, providing insights into fine-tuning dynamics, but it is incremental as it builds on existing analysis techniques.
The paper investigates how fine-tuning BERT affects its internal representations across different tasks, finding that it primarily changes top layers without catastrophic forgetting of linguistic knowledge, with variation such as dependency parsing reconfiguring most of the model while SQuAD and MNLI involve shallower processing.
While there has been much recent work studying how linguistic information is encoded in pre-trained sentence representations, comparatively little is understood about how these models change when adapted to solve downstream tasks. Using a suite of analysis techniques (probing classifiers, Representational Similarity Analysis, and model ablations), we investigate how fine-tuning affects the representations of the BERT model. We find that while fine-tuning necessarily makes significant changes, it does not lead to catastrophic forgetting of linguistic phenomena. We instead find that fine-tuning primarily affects the top layers of BERT, but with noteworthy variation across tasks. In particular, dependency parsing reconfigures most of the model, whereas SQuAD and MNLI appear to involve much shallower processing. Finally, we also find that fine-tuning has a weaker effect on representations of out-of-domain sentences, suggesting room for improvement in model generalization.