Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs
This addresses the challenge of high computational costs in knowledge acquisition for LLMs, offering a more efficient solution for AI systems that integrate retrieval, though it is incremental as it builds on existing retrieval-augmented methods.
The paper tackles the problem of efficiently determining when and what knowledge to retrieve for large language models (LLMs) by using a slim proxy model to detect missing knowledge, resulting in improved end-to-end performance on question-answering tasks with lower inference costs, achieving or surpassing state-of-the-art models on five datasets with two LLMs.
The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies. However, determining the knowledge that an LLM already possesses and the knowledge that requires the help of a search engine remains an unresolved issue. Most existing methods solve this problem through the results of preliminary answers or reasoning done by the LLM itself, but this incurs excessively high computational costs. This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model, to enhance the LLM's knowledge acquisition process. We employ a proxy model which has far fewer parameters, and take its answers as heuristic answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM. We only conduct retrieval for the missing knowledge in questions that the LLM does not know. Extensive experimental results on five datasets with two LLMs demonstrate a notable improvement in the end-to-end performance of LLMs in question-answering tasks, achieving or surpassing current state-of-the-art models with lower LLM inference costs.