Towards Building a Robust Knowledge Intensive Question Answering Model with Large Language Models
This work addresses robustness issues in LLMs for question answering, but it is incremental as it builds on existing retrieval enhancement and fine-tuning techniques.
The authors tackled the problem of noise and errors in retrieved information affecting the robustness of large language models (LLMs) in knowledge-intensive question answering, proposing a data augmentation-based fine-tuning method and contrastive learning to enhance robustness and discrimination capability, with results evaluated by GPT-4 showing improvements.
The development of LLMs has greatly enhanced the intelligence and fluency of question answering, while the emergence of retrieval enhancement has enabled models to better utilize external information. However, the presence of noise and errors in retrieved information poses challenges to the robustness of LLMs. In this work, to evaluate the model's performance under multiple interferences, we first construct a dataset based on machine reading comprehension datasets simulating various scenarios, including critical information absence, noise, and conflicts. To address the issue of model accuracy decline caused by noisy external information, we propose a data augmentation-based fine-tuning method to enhance LLM's robustness against noise. Additionally, contrastive learning approach is utilized to preserve the model's discrimination capability of external information. We have conducted experiments on both existing LLMs and our approach, the results are evaluated by GPT-4, which indicates that our proposed methods improve model robustness while strengthening the model's discrimination capability.