From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systems
It addresses the lack of unified frameworks for analyzing and evaluating CAIS, which is crucial for advancing system-level AI in academia and industry, though it is incremental as a survey.
This survey tackles the fragmented landscape of Compound AI Systems (CAIS), which integrate LLMs with external components to enhance capabilities, by defining the concept, proposing a taxonomy, and analyzing paradigms like RAG and LLM Agents to provide a comprehensive foundation for researchers and practitioners.
Compound Al Systems (CAIS) is an emerging paradigm that integrates large language models (LLMs) with external components, such as retrievers, agents, tools, and orchestrators, to overcome the limitations of standalone models in tasks requiring memory, reasoning, real-time grounding, and multimodal understanding. These systems enable more capable and context-aware behaviors by composing multiple specialized modules into cohesive workflows. Despite growing adoption in both academia and industry, the CAIS landscape remains fragmented, lacking a unified framework for analysis, taxonomy, and evaluation. In this survey, we define the concept of CAIS, propose a multi-dimensional taxonomy based on component roles and orchestration strategies, and analyze four foundational paradigms: Retrieval-Augmented Generation (RAG), LLM Agents, Multimodal LLMs (MLLMs), and orchestration-centric architectures. We review representative systems, compare design trade-offs, and summarize evaluation methodologies across these paradigms. Finally, we identify key challenges-including scalability, interoperability, benchmarking, and coordination-and outline promising directions for future research. This survey aims to provide researchers and practitioners with a comprehensive foundation for understanding, developing, and advancing the next generation of system-level artificial intelligence.