Quick Question: Interrupting Users for Microtasks with Reinforcement Learning
This work addresses the challenge of optimizing microtask interruptions for users in computing environments, though it is incremental as it builds on prior contextual and behavioral methods.
The paper tackled the problem of scheduling microtask interruptions to minimize user annoyance by using reinforcement learning, finding that while response rates were similar to supervised methods, RL reduced notification dismissals and improved user experience over a 5-week study with 30 participants.
Human attention is a scarce resource in modern computing. A multitude of microtasks vie for user attention to crowdsource information, perform momentary assessments, personalize services, and execute actions with a single touch. A lot gets done when these tasks take up the invisible free moments of the day. However, an interruption at an inappropriate time degrades productivity and causes annoyance. Prior works have exploited contextual cues and behavioral data to identify interruptibility for microtasks with much success. With Quick Question, we explore use of reinforcement learning (RL) to schedule microtasks while minimizing user annoyance and compare its performance with supervised learning. We model the problem as a Markov decision process and use Advantage Actor Critic algorithm to identify interruptible moments based on context and history of user interactions. In our 5-week, 30-participant study, we compare the proposed RL algorithm against supervised learning methods. While the mean number of responses between both methods is commensurate, RL is more effective at avoiding dismissal of notifications and improves user experience over time.