Interacting with next-phrase suggestions: How suggestion systems aid and influence the cognitive processes of writing
This addresses the problem of understanding cognitive impacts for writers using AI tools, but it is incremental as it builds on existing qualitative research.
The study investigated how amateur writers interact with next-phrase suggestion systems, finding that writers incorporate parts of suggestions even when disagreeing, and the system alters writing plans and increases distraction.
Writing with next-phrase suggestions powered by large language models is becoming more pervasive by the day. However, research to understand writers' interaction and decision-making processes while engaging with such systems is still emerging. We conducted a qualitative study to shed light on writers' cognitive processes while writing with next-phrase suggestion systems. To do so, we recruited 14 amateur writers to write two reviews each, one without suggestions and one with suggestions. Additionally, we also positively and negatively biased the suggestion system to get a diverse range of instances where writers' opinions and the bias in the language model align or misalign to varying degrees. We found that writers interact with next-phrase suggestions in various complex ways: Writers abstracted and extracted multiple parts of the suggestions and incorporated them within their writing, even when they disagreed with the suggestion as a whole; along with evaluating the suggestions on various criteria. The suggestion system also had various effects on the writing process, such as altering the writer's usual writing plans, leading to higher levels of distraction etc. Based on our qualitative analysis using the cognitive process model of writing by Hayes as a lens, we propose a theoretical model of 'writer-suggestion interaction' for writing with GPT-2 (and causal language models in general) for a movie review writing task, followed by directions for future research and design.