Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction
This work addresses the challenge of enhancing creativity in language models for open-ended tasks like wordplay, analogies, and design, though it appears incremental in proposing new methods for known bottlenecks.
The authors tackled the problem of next-token prediction being myopic for creative tasks by designing minimal algorithmic tasks to quantify these limits, finding that multi-token approaches like teacherless training and diffusion models excel in producing diverse and original output, and that seed-conditioning noise injection works as well as or better than temperature sampling.
We design a suite of minimal algorithmic tasks that are a loose abstraction of open-ended real-world tasks. This allows us to cleanly and controllably quantify the creative limits of the present-day language model. Much like real-world tasks that require a creative, far-sighted leap of thought, our tasks require an implicit, open-ended stochastic planning step that either (a) discovers new connections in an abstract knowledge graph (like in wordplay, drawing analogies, or research) or (b) constructs new patterns (like in designing math problems or new proteins). In these tasks, we empirically and conceptually argue how next-token learning is myopic; multi-token approaches, namely teacherless training and diffusion models, comparatively excel in producing diverse and original output. Secondly, to elicit randomness without hurting coherence, we find that injecting noise at the input layer (dubbed seed-conditioning) works surprisingly as well as (and in some conditions, better than) temperature sampling from the output layer. Thus, our work offers a principled, minimal test-bed for analyzing open-ended creative skills, and offers new arguments for going beyond next-token learning and temperature sampling. We make part of the code available under https://github.com/chenwu98/algorithmic-creativity