When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
This provides a principled framework for handling long contexts in LLMs, which is important for applications like retrieval and summarization, though it appears incremental as it builds on existing chunking approaches.
The authors tackled the challenge of applying LLMs to long texts by proposing a noise decomposition framework that identifies three failure modes, and found that multi-agent chunking with careful management can enable weaker models to surpass advanced models like GPT4o on large inputs.
We investigate the challenge of applying Large Language Models (LLMs) to long texts. We propose a theoretical framework that distinguishes the failure modes of long context tasks into three categories: cross-chunk dependence (task noise), confusion that grows with context size (model noise), and the imperfect integration of partial results (aggregator noise). Under this view, we analyze when it is effective to use multi-agent chunking, i.e., dividing a length sequence into smaller chunks and aggregating the processed results of each chunk. Our experiments on tasks such as retrieval, question answering, and summarization confirm both the theoretical analysis and the conditions that favor multi-agent chunking. By exploring superlinear model noise growth with input length, we also explain why, for large inputs, a weaker model configured with chunk-based processing can surpass a more advanced model like GPT4o applied in a single shot. Overall, we present a principled understanding framework and our results highlight a direct pathway to handling long contexts in LLMs with carefully managed chunking and aggregator strategies.