Societal Impacts Research Requires Benchmarks for Creative Composition Tasks
This work addresses the societal risks of AI-generated content for users and information ecosystems, but it is incremental as it builds on existing concerns by proposing benchmark improvements.
The paper identifies creative composition tasks as a prevalent use case for foundation models, where current benchmarks fail to align with real usage patterns, and argues that developing new benchmarks for these tasks is essential to understand societal harms like formulaic or misleading AI-generated content.
Foundation models that are capable of automating cognitive tasks represent a pivotal technological shift, yet their societal implications remain unclear. These systems promise exciting advances, yet they also risk flooding our information ecosystem with formulaic, homogeneous, and potentially misleading synthetic content. Developing benchmarks grounded in real use cases where these risks are most significant is therefore critical. Through a thematic analysis using 2 million language model user prompts, we identify creative composition tasks as a prevalent usage category where users seek help with personal tasks that require everyday creativity. Our fine-grained analysis identifies mismatches between current benchmarks and usage patterns among these tasks. Crucially, we argue that the same use cases that currently lack thorough evaluations can lead to negative downstream impacts. This position paper argues that benchmarks focused on creative composition tasks is a necessary step towards understanding the societal harms of AI-generated content. We call for greater transparency in usage patterns to inform the development of new benchmarks that can effectively measure both the progress and the impacts of models with creative capabilities.