Learning Efficient Representations of Mouse Movements to Predict User Attention
This work addresses the need for efficient attention prediction in web interfaces, offering an incremental improvement by replacing handcrafted features with neural networks.
The paper tackled the problem of predicting user attention on web pages like search engine results pages (SERPs) by using mouse cursor movements, achieving competitive performance with neural network models trained on raw data.
Tracking mouse cursor movements can be used to predict user attention on heterogeneous page layouts like SERPs. So far, previous work has relied heavily on handcrafted features, which is a time-consuming approach that often requires domain expertise. We investigate different representations of mouse cursor movements, including time series, heatmaps, and trajectory-based images, to build and contrast both recurrent and convolutional neural networks that can predict user attention to direct displays, such as SERP advertisements. Our models are trained over raw mouse cursor data and achieve competitive performance. We conclude that neural network models should be adopted for downstream tasks involving mouse cursor movements, since they can provide an invaluable implicit feedback signal for re-ranking and evaluation.