The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
This study addresses the impact of task framing on writing style for researchers in computational linguistics and natural language processing, but it is incremental as it builds on existing tasks and methods.
The paper tackled the problem of how different writing tasks affect linguistic style, showing that a simple linear classifier using stylistic features can distinguish among three task variants with different constraints, and combining these features with language model predictions achieves state-of-the-art performance on the story cloze challenge.
A writer's style depends not just on personal traits but also on her intent and mental state. In this paper, we show how variants of the same writing task can lead to measurable differences in writing style. We present a case study based on the story cloze task (Mostafazadeh et al., 2016a), where annotators were assigned similar writing tasks with different constraints: (1) writing an entire story, (2) adding a story ending for a given story context, and (3) adding an incoherent ending to a story. We show that a simple linear classifier informed by stylistic features is able to successfully distinguish among the three cases, without even looking at the story context. In addition, combining our stylistic features with language model predictions reaches state of the art performance on the story cloze challenge. Our results demonstrate that different task framings can dramatically affect the way people write.