Predicting Human Visual Attention on Words in Source Code
For software engineering researchers and tool designers, this model offers a way to predict programmer attention, potentially improving UI design and understanding of cognitive processes, though the gains are incremental over existing baselines.
This paper introduces a computational model that predicts human visual attention on words in source code, using a novel loss function to align neural network attention with eye-tracking data. The model outperforms baselines by 64%, 16%, and 467% on three datasets (Java and C) in Pearson correlation, and surpasses Claude and GPT-5 on reading and writing tasks.
This paper presents a computational model to predict human visual attention over words in software source code. The visual attention of software engineers when reading source code has long been studied as a means to understand human cognitive processes during software engineering tasks. Predicting this visual attention is important for perfecting user interface design and understanding what information human programmers need. We propose a model of programmer visual attention in which we design a novel loss function that computes similarity between human attention measured during eye tracking experiments and the internal attention of the artificial neural network. We evaluate our model by comparing its outputs to actual eye tracking data from three separate datasets. Two are in the Java programming language and one is in the C programming language. Our model outperforms the baseline in software engineering by 64%, 16%, and 467% in each of these studies according to Pearson correlation. We used scanpath prediction as an example to demonstrate that our model is more capable of the task that requires the understanding of human thought process. Our model achieves a statistically significant improvement over the close baseline in the reading task according to normalized Levenshtein distance and outperforms both Claude and GPT-5 on both reading and writing tasks.