Learning to Plan Long-Term for Language Modeling
This addresses the issue of inefficient long-term text generation in language models for applications like content creation or dialogue systems, though it appears incremental as it builds on existing attention-based models.
The paper tackles the problem of language models lacking explicit long-term planning mechanisms, which leads to suboptimal token prediction, by proposing a planner that predicts latent plans for future sentences and samples multiple plans to condition the language model, resulting in improved next token prediction accuracy.
Modern language models predict the next token in the sequence by considering the past text through a powerful function such as attention. However, language models have no explicit mechanism that allows them to spend computation time for planning long-distance future text, leading to a suboptimal token prediction. In this paper, we propose a planner that predicts a latent plan for many sentences into the future. By sampling multiple plans at once, we condition the language model on an accurate approximation of the distribution of text continuations, which leads to better next token prediction accuracy. In effect, this allows trading computation time for prediction accuracy.