Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based Question Answering
This addresses the need for more reliable and in-depth answers in open-ended QA, though it appears incremental by combining existing models.
The paper tackles the problem of open-ended question answering by proposing a Chain-of-Discussion framework that leverages multiple open-source LLMs to improve answer quality, with experiments showing enhanced correctness and comprehensiveness.
Open-ended question answering requires models to find appropriate evidence to form wellreasoned, comprehensive and helpful answers. In practical applications, models also need to engage in extended discussions on potential scenarios closely relevant to the question. With augmentation of retrieval module, open-source Large Language Models (LLMs) can produce coherent answers often with different focuses, but are still sub-optimal in terms of reliable evidence selection and in-depth question analysis. In this paper, we propose a novel Chain-ofDiscussion framework to leverage the synergy among multiple open-source LLMs aiming to provide more correct and more comprehensive answers for open-ended QA, although they are not strong enough individually. Our experiments show that discussions among multiple LLMs play a vital role in enhancing the quality of answers.