Reimagining Retrieval Augmented Language Models for Answering Queries
This work addresses the challenge of making language models more effective for question answering tasks, though it appears incremental as it builds on existing semi-parametric architectures.
The paper tackles the problem of improving question answering by enhancing retrieval-augmented language models with views, a query analyzer/planner, and provenance, resulting in a system that is significantly more powerful in terms of accuracy and efficiency.
We present a reality check on large language models and inspect the promise of retrieval augmented language models in comparison. Such language models are semi-parametric, where models integrate model parameters and knowledge from external data sources to make their predictions, as opposed to the parametric nature of vanilla large language models. We give initial experimental findings that semi-parametric architectures can be enhanced with views, a query analyzer/planner, and provenance to make a significantly more powerful system for question answering in terms of accuracy and efficiency, and potentially for other NLP tasks