We're Different, We're the Same: Creative Homogeneity Across LLMs
This reveals a potential limitation for users relying on LLMs for creative tasks, as it may drive all users toward a limited set of outputs, highlighting an incremental concern in AI-assisted creativity.
The study investigated whether using large language models (LLMs) as creative assistants leads to narrowed creativity, finding that LLM responses are significantly more similar to each other than human responses, indicating homogeneity across models.
Numerous powerful large language models (LLMs) are now available for use as writing support tools, idea generators, and beyond. Although these LLMs are marketed as helpful creative assistants, several works have shown that using an LLM as a creative partner results in a narrower set of creative outputs. However, these studies only consider the effects of interacting with a single LLM, begging the question of whether such narrowed creativity stems from using a particular LLM -- which arguably has a limited range of outputs -- or from using LLMs in general as creative assistants. To study this question, we elicit creative responses from humans and a broad set of LLMs using standardized creativity tests and compare the population-level diversity of responses. We find that LLM responses are much more similar to other LLM responses than human responses are to each other, even after controlling for response structure and other key variables. This finding of significant homogeneity in creative outputs across the LLMs we evaluate adds a new dimension to the ongoing conversation about creativity and LLMs. If today's LLMs behave similarly, using them as a creative partners -- regardless of the model used -- may drive all users towards a limited set of "creative" outputs.