Towards Rationality in Language and Multimodal Agents: A Survey
It addresses the issue of unreliable outputs in AI systems for researchers and developers, but is incremental as it synthesizes existing work rather than proposing new methods.
This survey examines the problem of improving rationality in language and multimodal agents, which often lack consistency and evidence-based decision-making, by reviewing recent advancements and identifying open challenges.
This work discusses how to build more rational language and multimodal agents and what criteria define rationality in intelligent systems. Rationality is the quality of being guided by reason, characterized by decision-making that aligns with evidence and logical principles. It plays a crucial role in reliable problem-solving by ensuring well-grounded and consistent solutions. Despite their progress, large language models (LLMs) often fall short of rationality due to their bounded knowledge space and inconsistent outputs. In response, recent efforts have shifted toward developing multimodal and multi-agent systems, as well as integrating modules like external tools, programming codes, symbolic reasoners, utility function, and conformal risk controls rather than relying solely on a single LLM for decision-making. This paper surveys state-of-the-art advancements in language and multimodal agents, assesses their role in enhancing rationality, and outlines open challenges and future research directions. We maintain an open repository at https://github.com/bowen-upenn/Agent_Rationality.