Factuality Checking in News Headlines with Eye Tracking
This work addresses the problem of automated fact-checking for news consumers by leveraging eye-tracking data, though it is incremental as it applies existing methods to a new domain.
The researchers investigated whether eye movement patterns can predict the factuality of news headlines, finding that false headlines receive significantly less visual attention than true ones, and they developed an ensemble model achieving a mean AUC of 0.688 for this task.
We study whether it is possible to infer if a news headline is true or false using only the movement of the human eyes when reading news headlines. Our study with 55 participants who are eye-tracked when reading 108 news headlines (72 true, 36 false) shows that false headlines receive statistically significantly less visual attention than true headlines. We further build an ensemble learner that predicts news headline factuality using only eye-tracking measurements. Our model yields a mean AUC of 0.688 and is better at detecting false than true headlines. Through a model analysis, we find that eye-tracking 25 users when reading 3-6 headlines is sufficient for our ensemble learner.