Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?
This research addresses the problem of understanding model interpretability for NLP researchers and cognitive scientists, but it is incremental as it builds on existing comparisons between machine and human attention.
The study investigated whether self-attention in large-scale pre-trained language models predicts human eye fixation patterns during task-specific reading, finding that predictiveness depends on rare syntactic contexts and that fine-tuning does not increase correlation.
Learned self-attention functions in state-of-the-art NLP models often correlate with human attention. We investigate whether self-attention in large-scale pre-trained language models is as predictive of human eye fixation patterns during task-reading as classical cognitive models of human attention. We compare attention functions across two task-specific reading datasets for sentiment analysis and relation extraction. We find the predictiveness of large-scale pre-trained self-attention for human attention depends on `what is in the tail', e.g., the syntactic nature of rare contexts. Further, we observe that task-specific fine-tuning does not increase the correlation with human task-specific reading. Through an input reduction experiment we give complementary insights on the sparsity and fidelity trade-off, showing that lower-entropy attention vectors are more faithful.