Avoidance Decoding for Diverse Multi-Branch Story Generation
This addresses the issue of limited creative diversity in story generation for users of LLMs, representing an incremental improvement in decoding methods.
The paper tackles the problem of repetitive and monotonous outputs in story generation with LLMs by proposing Avoidance Decoding, a decoding strategy that penalizes similarity to previous outputs to encourage diverse multi-branch stories, achieving up to 2.6 times higher diversity and reducing repetition by 30% compared to baselines.
Large Language Models (LLMs) often generate repetitive and monotonous outputs, especially in tasks like story generation, due to limited creative diversity when given the same input prompt. To address this challenge, we propose a novel decoding strategy, Avoidance Decoding, that modifies token logits by penalizing similarity to previously generated outputs, thereby encouraging more diverse multi-branch stories. This penalty adaptively balances two similarity measures: (1) Concept-level Similarity Penalty, which is prioritized in early stages to diversify initial story concepts, and (2) Narrative-level Similarity Penalty, which is increasingly emphasized later to ensure natural yet diverse plot development. Notably, our method achieves up to 2.6 times higher output diversity and reduces repetition by an average of 30% compared to strong baselines, while effectively mitigating text degeneration. Furthermore, we reveal that our method activates a broader range of neurons, demonstrating that it leverages the model's intrinsic creativity.