Modeling Story Expectations to Understand Engagement: A Generative Framework Using LLMs
This addresses the challenge for content creators and platforms in predicting engagement by complementing existing methods, though it is incremental as it builds on feature extraction techniques.
The paper tackled the problem of modeling audience forward-looking beliefs in stories to understand engagement, introducing a framework that uses LLMs to generate continuations and extract expectation-related features, and demonstrated that it amplifies the marginal explanatory power of existing feature engineering techniques by 31% on average over 30,000 book chapters.
Understanding when and why consumers engage with stories is crucial for content creators and platforms. While existing theories suggest that audience beliefs of what is going to happen should play an important role in engagement decisions, empirical work has mostly focused on developing techniques to directly extract features from actual content, rather than capturing forward-looking beliefs, due to the lack of a principled way to model such beliefs in unstructured narrative data. To complement existing feature extraction techniques, this paper introduces a novel framework that leverages large language models to model audience forward-looking beliefs about how stories might unfold. Our method generates multiple potential continuations for each story and extracts features related to expectations, uncertainty, and surprise using established content analysis techniques. Applying our method to over 30,000 book chapters, we demonstrate that our framework complements existing feature engineering techniques by amplifying their marginal explanatory power on average by 31%. The results reveal that different types of engagement-continuing to read, commenting, and voting-are driven by distinct combinations of current and anticipated content features. Our framework provides a novel way to study and explore how audience forward-looking beliefs shape their engagement with narrative media, with implications for marketing strategy in content-focused industries.