On Surgical Fine-tuning for Language Encoders
This work addresses the efficiency and performance of fine-tuning for NLP practitioners, offering an incremental improvement by localizing task-specific information to reduce computational cost.
The paper tackles the problem of fine-tuning pre-trained language encoders by showing that selectively fine-tuning only a subset of layers, identified using a Fisher information matrix-based metric, achieves performance close to or better than full fine-tuning on GLUE and SuperGLUE tasks, with empirical validation across different encoders.
Fine-tuning all the layers of a pre-trained neural language encoder (either using all the parameters or using parameter-efficient methods) is often the de-facto way of adapting it to a new task. We show evidence that for different downstream language tasks, fine-tuning only a subset of layers is sufficient to obtain performance that is close to and often better than fine-tuning all the layers in the language encoder. We propose an efficient metric based on the diagonal of the Fisher information matrix (FIM score), to select the candidate layers for selective fine-tuning. We show, empirically on GLUE and SuperGLUE tasks and across distinct language encoders, that this metric can effectively select layers leading to a strong downstream performance. Our work highlights that task-specific information corresponding to a given downstream task is often localized within a few layers, and tuning only those is sufficient for strong performance. Additionally, we demonstrate the robustness of the FIM score to rank layers in a manner that remains constant during the optimization process.