BERTology Meets Biology: Interpreting Attention in Protein Language Models
This work addresses interpretability challenges in protein language models for computational biology, but it is incremental as it applies existing interpretability methods to a new domain.
The paper tackled the problem of interpreting protein Transformer models by analyzing attention mechanisms, showing that attention captures protein folding structure, targets binding sites, and focuses on complex biophysical properties with depth, with consistent results across three architectures and two datasets.
Transformer architectures have proven to learn useful representations for protein classification and generation tasks. However, these representations present challenges in interpretability. In this work, we demonstrate a set of methods for analyzing protein Transformer models through the lens of attention. We show that attention: (1) captures the folding structure of proteins, connecting amino acids that are far apart in the underlying sequence, but spatially close in the three-dimensional structure, (2) targets binding sites, a key functional component of proteins, and (3) focuses on progressively more complex biophysical properties with increasing layer depth. We find this behavior to be consistent across three Transformer architectures (BERT, ALBERT, XLNet) and two distinct protein datasets. We also present a three-dimensional visualization of the interaction between attention and protein structure. Code for visualization and analysis is available at https://github.com/salesforce/provis.