Brainstorming Brings Power to Large Language Models of Knowledge Reasoning
This provides a new solution for distributed deployment of LLMs, addressing instability in multi-model collaboration for knowledge reasoning tasks.
The paper tackles the problem of biased and unstable results from single large language models in knowledge reasoning by proposing a multi-model brainstorming method based on prompts, which significantly improves effectiveness in logical reasoning and fact extraction across three datasets, with two small models achieving accuracy approximating larger models.
Large Language Models (LLMs) have demonstrated amazing capabilities in language generation, text comprehension, and knowledge reasoning. While a single powerful model can already handle multiple tasks, relying on a single perspective can lead to biased and unstable results. Recent studies have further improved the model's reasoning ability on a wide range of tasks by introducing multi-model collaboration. However, models with different capabilities may produce conflicting answers on the same problem, and how to reasonably obtain the correct answer from multiple candidate models has become a challenging problem. In this paper, we propose the multi-model brainstorming based on prompt. It incorporates different models into a group for brainstorming, and after multiple rounds of reasoning elaboration and re-inference, a consensus answer is reached within the group. We conducted experiments on three different types of datasets, and demonstrate that the brainstorming can significantly improve the effectiveness in logical reasoning and fact extraction. Furthermore, we find that two small-parameter models can achieve accuracy approximating that of larger-parameter models through brainstorming, which provides a new solution for distributed deployment of LLMs.