FragRel: Exploiting Fragment-level Relations in the External Memory of Large Language Models
This work addresses the challenge of handling structurally connected long texts for applications like story analysis and code generation, representing an incremental improvement over existing hierarchical memory approaches.
The paper tackles the problem of processing long texts with intensive inter-relations, such as coherent stories and code repositories, in Large Language Models (LLMs) by exploiting fragment-level relations in external memory, resulting in improved performance on tasks like long story understanding, repository-level code generation, and long-term chatting.
To process contexts with unlimited length using Large Language Models (LLMs), recent studies explore hierarchically managing the long text. Only several text fragments are taken from the external memory and passed into the temporary working memory, i.e., LLM's context window. However, existing approaches isolatedly handle the text fragments without considering their structural connections, thereby suffering limited capability on texts with intensive inter-relations, e.g., coherent stories and code repositories. This work attempts to resolve this by exploiting the fragment-level relations in external memory. First, we formulate the fragment-level relations and present several instantiations for different text types. Next, we introduce a relation-aware fragment assessment criteria upon previous independent fragment assessment. Finally, we present the fragment-connected Hierarchical Memory based LLM. We validate the benefits of involving these relations on long story understanding, repository-level code generation, and long-term chatting.