Lost in the Passage: Passage-level In-context Learning Does Not Necessarily Need a "Passage"
This reveals a critical limitation in how LLMs process long-context demonstrations, impacting researchers and practitioners in NLP and AI.
The study found that in passage-level in-context learning for generation tasks, large language models fail to utilize demonstration passages effectively, as meaningless passages perform better than original ones, with experiments showing up to 1/4 length meaningless passages achieving superior results.
By simply incorporating demonstrations into the context, in-context learning (ICL) enables large language models (LLMs) to yield awesome performance on many tasks. In this study, we focus on passage-level long-context ICL for generation tasks and find that LLMs cannot learn the intrinsic relationship between the demonstration passage and the generation output. We conduct experiments with different LLMs on two typical generation tasks including single-document question answering and distractor generation, demonstrating that even a completely meaningless demonstration passage with 1/4 length achieves much better performance than the original full passage. Analysis via attention and information flow reveals that LLMs pay little attention to passages compared to other components in the prompt and little information flows from the passage to other parts of the demonstration, which further confirms our finding. Additionally, experiments on context compression indicate that compression approaches proven effective on other long-context tasks are not suitable for passage-level ICL, since simply using shorter meaningless demonstration passages already achieves competitive performance.