Typing Behavior in Human-LLM Interaction: Keystroke Dynamics Reveal Cognitive Effort During Prompting
For HCI researchers and LLM interface designers, this work provides initial evidence that keystroke dynamics can indicate cognitive effort in real-time, though it is limited by small sample size and lack of predictive power for perceived quality.
This study investigates keystroke dynamics as a behavioral measure of cognitive effort during human-LLM interaction, finding that hard tasks significantly increase keystrokes, slow typing, and raise workload, but keystrokes do not predict perceived output usefulness.
As Large Language Models (LLMs) become increasingly integrated into daily routines, understanding how users interact with these systems is crucial for effective human-AI collaboration. This work investigates keystroke dynamics as a behavioral measure of user mental effort and perceived output usefulness in human-LLM interaction. We conducted a user study (N = 36) to examine how task difficulty (easy vs. hard) and device type (desktop vs. mobile) influence typing behavior and workload (NASA-TLX) during interactions. Our results indicate that hard tasks led to significantly more keystrokes, slower typing, increased pauses, and higher self-reported workload. Device type had weaker effects, with mobile use slightly reducing input length and typing speed. While keystrokes captured differences in cognitive effort, they did not predict perceived LLM output usefulness. These findings highlight the potential of keystroke dynamics as real-time indicators of cognitive effort during LLM prompting, while also showing their limitations in capturing perceived collaboration success.