GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge?
This addresses the need for LLMs to adapt to growing knowledge in open-domain QA and dialogue tasks, offering a novel solution to a known bottleneck.
The paper tackles the problem of large language models (LLMs) becoming outdated as real-world knowledge evolves, by proposing a retrieval-interactive framework that allows models to evaluate and re-retrieve information, resulting in significant improvements over existing methods, performing comparably to or surpassing continuously trained models.
In the real world, knowledge is constantly evolving, which can render existing knowledge-based datasets outdated. This unreliability highlights the critical need for continuous updates to ensure both accuracy and relevance in knowledge-intensive tasks. To address this, we propose GrowOVER-QA and GrowOVER-Dialogue, dynamic open-domain QA and dialogue benchmarks that undergo a continuous cycle of updates, keeping pace with the rapid evolution of knowledge. Our research indicates that retrieval-augmented language models (RaLMs) struggle with knowledge that has not been trained on or recently updated. Consequently, we introduce a novel retrieval-interactive language model framework, where the language model evaluates and reflects on its answers for further re-retrieval. Our exhaustive experiments demonstrate that our training-free framework significantly improves upon existing methods, performing comparably to or even surpassing continuously trained language models.