Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models
This work addresses the problem of interpretability in pretrained language models for researchers and practitioners, though it appears incremental as it builds on existing white-box metric approaches.
The paper tackles the challenge of designing interpretable metrics for understanding pretrained language models when source models lack inherent interpretability, proposing a novel line of construction and a tree topological probe, with experiments on BERT-large leading to a speculation on model mechanisms and a strategy for enhancing fine-tuning performance.
Pretrained language models are expected to effectively map input text to a set of vectors while preserving the inherent relationships within the text. Consequently, designing a white-box model to compute metrics that reflect the presence of specific internal relations in these vectors has become a common approach for post-hoc interpretability analysis of pretrained language models. However, achieving interpretability in white-box models and ensuring the rigor of metric computation becomes challenging when the source model lacks inherent interpretability. Therefore, in this paper, we discuss striking a balance in this trade-off and propose a novel line to constructing metrics for understanding the mechanisms of pretrained language models. We have specifically designed a family of metrics along this line of investigation, and the model used to compute these metrics is referred to as the tree topological probe. We conducted measurements on BERT-large by using these metrics. Based on the experimental results, we propose a speculation regarding the working mechanism of BERT-like pretrained language models, as well as a strategy for enhancing fine-tuning performance by leveraging the topological probe to improve specific submodules.