When One LLM Drools, Multi-LLM Collaboration Rules
It addresses the problem of underrepresentation and unreliability in AI systems for developers and researchers, but it is incremental as it organizes existing methods and motivates future work without presenting new empirical results.
This position paper argues that relying on a single large language model is insufficient for complex, contextualized, and subjective scenarios, proposing multi-LLM collaboration to improve reliability, democratization, and pluralism by better representing diverse data, skills, and populations.
This position paper argues that in many realistic (i.e., complex, contextualized, subjective) scenarios, one LLM is not enough to produce a reliable output. We challenge the status quo of relying solely on a single general-purpose LLM and argue for multi-LLM collaboration to better represent the extensive diversity of data, skills, and people. We first posit that a single LLM underrepresents real-world data distributions, heterogeneous skills, and pluralistic populations, and that such representation gaps cannot be trivially patched by further training a single LLM. We then organize existing multi-LLM collaboration methods into a hierarchy, based on the level of access and information exchange, ranging from API-level, text-level, logit-level, to weight-level collaboration. Based on these methods, we highlight how multi-LLM collaboration addresses challenges that a single LLM struggles with, such as reliability, democratization, and pluralism. Finally, we identify the limitations of existing multi-LLM methods and motivate future work. We envision multi-LLM collaboration as an essential path toward compositional intelligence and collaborative AI development.