Desirable Unfamiliarity: Insights from Eye Movements on Engagement and Readability of Dictation Interfaces
This addresses the challenge of designing user-friendly dictation interfaces for users, though it is incremental as it builds on existing LLM-based correction methods.
The study tackled the problem of balancing readability, attention, and accuracy in dictation interfaces by conducting an eye-tracking experiment with 20 participants to compare five interfaces, finding that participants spent only 7-11% of their time in active reading during composition and preferred the SUMMARY interface despite its unfamiliar phrasing.
Transcripts displayed on dictation interfaces can be hard to read due to recognition errors and disfluencies. LLM-based text auto-correction could help, but changing the text during production could lead to distraction and unintended phrasing. To understand how to balance readability, attention, and accuracy, we conducted an eye-tracking experiment with 20 participants to compare five dictation interfaces: PLAIN (real-time transcription), AOC (periodic corrections), RAKE (keyword highlights), GP-TSM (grammar-preserving highlights), and SUMMARY (LLM-generated abstractive summary). By analyzing participants' gaze patterns during speech composition and reviewing processes, we found that during composition, participants spent only 7-11% of their time in active reading regardless of the interface. Although SUMMARY introduced unfamiliar words and phrasing during composition, it was easier to read and more preferred by participants. Our findings suggest a high user tolerance for altering spoken words in LLM-enabled diction interfaces.