Creativity Has Left the Chat: The Price of Debiasing Language Models
This highlights a trade-off between consistency and creativity in aligned models, which is important for marketers and others using LLMs for creative tasks like copywriting and ad creation, though it is incremental in exploring side effects of existing alignment techniques.
The study investigated the unintended consequences of Reinforcement Learning from Human Feedback (RLHF) on the creativity of Large Language Models, specifically the Llama-2 series, finding that aligned models exhibit lower entropy in token predictions, form distinct clusters in embedding space, and gravitate towards 'attractor states', indicating reduced output diversity.
Large Language Models (LLMs) have revolutionized natural language processing but can exhibit biases and may generate toxic content. While alignment techniques like Reinforcement Learning from Human Feedback (RLHF) reduce these issues, their impact on creativity, defined as syntactic and semantic diversity, remains unexplored. We investigate the unintended consequences of RLHF on the creativity of LLMs through three experiments focusing on the Llama-2 series. Our findings reveal that aligned models exhibit lower entropy in token predictions, form distinct clusters in the embedding space, and gravitate towards "attractor states", indicating limited output diversity. Our findings have significant implications for marketers who rely on LLMs for creative tasks such as copywriting, ad creation, and customer persona generation. The trade-off between consistency and creativity in aligned models should be carefully considered when selecting the appropriate model for a given application. We also discuss the importance of prompt engineering in harnessing the creative potential of base models.