Balancing Appearance and Context in Sketch Interpretation
This work addresses the challenge of accurately screening for cognitive impairments like dementia in clinical settings, though it is incremental in improving sketch interpretation methods.
The paper tackles the problem of interpreting ambiguous or incomprehensible strokes in clock drawings from the Clock Drawing Test by balancing appearance and context, achieving performance on 2,000 drawings with about 24,000 clock numerals.
We describe a sketch interpretation system that detects and classifies clock numerals created by subjects taking the Clock Drawing Test, a clinical tool widely used to screen for cognitive impairments (e.g., dementia). We describe how it balances appearance and context, and document its performance on some 2,000 drawings (about 24K clock numerals) produced by a wide spectrum of patients. We calibrate the utility of different forms of context, describing experiments with Conditional Random Fields trained and tested using a variety of features. We identify context that contributes to interpreting otherwise ambiguous or incomprehensible strokes. We describe ST-slices, a novel representation that enables "unpeeling" the layers of ink that result when people overwrite, which often produces ink impossible to analyze if only the final drawing is examined. We characterize when ST-slices work, calibrate their impact on performance, and consider their breadth of applicability.