The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More
This addresses hallucinations and unreliable information retrieval in language models, which is a critical problem for AI reliability, though it is incremental as it builds on known issues like the reversal curse.
The paper identifies the reversal curse in language models as a factorization curse, showing that next-token prediction inherently fails to learn joint distributions under different factorizations, and finds that factorization-agnostic objectives can significantly mitigate this issue across five tasks.
Today's best language models still struggle with hallucinations: factually incorrect generations, which impede their ability to reliably retrieve information seen during training. The reversal curse, where models cannot recall information when probed in a different order than was encountered during training, exemplifies this in information retrieval. We reframe the reversal curse as a factorization curse - a failure of models to learn the same joint distribution under different factorizations. Through a series of controlled experiments with increasing levels of realism including WikiReversal, a setting we introduce to closely simulate a knowledge intensive finetuning task, we find that the factorization curse is an inherent failure of the next-token prediction objective used in popular large language models. Moreover, we demonstrate reliable information retrieval cannot be solved with scale, reversed tokens, or even naive bidirectional-attention training. Consequently, various approaches to finetuning on specialized data would necessarily provide mixed results on downstream tasks, unless the model has already seen the right sequence of tokens. Across five tasks of varying levels of complexity, our results uncover a promising path forward: factorization-agnostic objectives can significantly mitigate the reversal curse and hint at improved knowledge storage and planning capabilities.