Learning to Pay Attention: Unsupervised Modeling of Attentive and Inattentive Respondents in Survey Data
This provides a scalable, domain-agnostic tool for survey platforms to improve data quality without additional respondent burden, though it is incremental as it builds on existing unsupervised methods.
The paper tackled the problem of detecting inattentive respondents in surveys by proposing an unsupervised framework that scores response coherence using geometric reconstruction and probabilistic dependency modeling, finding that detection effectiveness depends more on survey structure than model complexity, with strong results across nine real-world datasets.
The integrity of behavioral and social-science surveys depends on detecting inattentive respondents who provide random or low-effort answers. Traditional safeguards, such as attention checks, are often costly, reactive, and inconsistent. We propose a unified, label-free framework for inattentiveness detection that scores response coherence using complementary unsupervised views: geometric reconstruction (Autoencoders) and probabilistic dependency modeling (Chow-Liu trees). While we introduce a "Percentile Loss" objective to improve Autoencoder robustness against anomalies, our primary contribution is identifying the structural conditions that enable unsupervised quality control. Across nine heterogeneous real-world datasets, we find that detection effectiveness is driven less by model complexity than by survey structure: instruments with coherent, overlapping item batteries exhibit strong covariance patterns that allow even linear models to reliably separate attentive from inattentive respondents. This reveals a critical ``Psychometric-ML Alignment'': the same design principles that maximize measurement reliability (e.g., internal consistency) also maximize algorithmic detectability. The framework provides survey platforms with a scalable, domain-agnostic diagnostic tool that links data quality directly to instrument design, enabling auditing without additional respondent burden.