Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention
This enables personalized visual data communication by addressing individual differences in visualization literacy and attention patterns.
The researchers tackled the problem of individual differences in visualization interpretation by showing that distinct attention patterns correlate with visualization literacy levels, then proposed two computational models: Lit2Sal (a literacy-aware saliency model that outperforms state-of-the-art models) and Sal2Lit (which predicts literacy from attention data with 86% accuracy in under a minute).
Accounting for individual differences can improve the effectiveness of visualization design. While the role of visual attention in visualization interpretation is well recognized, existing work often overlooks how this behavior varies based on visual literacy levels. Based on data from a 235-participant user study covering three visualization tests (mini-VLAT, CALVI, and SGL), we show that distinct attention patterns in visual data exploration can correlate with participants' literacy levels: While experts (high-scorers) generally show a strong attentional focus, novices (low-scorers) focus less and explore more. We then propose two computational models leveraging these insights: Lit2Sal -- a novel visual saliency model that predicts observer attention given their visualization literacy level, and Sal2Lit -- a model to predict visual literacy from human visual attention data. Our quantitative and qualitative evaluation demonstrates that Lit2Sal outperforms state-of-the-art saliency models with literacy-aware considerations. Sal2Lit predicts literacy with 86% accuracy using a single attention map, providing a time-efficient supplement to literacy assessment that only takes less than a minute. Taken together, our unique approach to consider individual differences in salience models and visual attention in literacy assessments paves the way for new directions in personalized visual data communication to enhance understanding.