Programmable Cognitive Bias in Social Agents
This work addresses the challenge of creating reliable social agents for researchers and developers, though it is incremental as it builds on existing simulation methods.
The paper tackles the problem of inconsistent and imprecise agent behaviors in LLM-based social simulations by introducing CoBRA, a toolkit that explicitly programs cognitive biases using classic social science experiments, resulting in precise, model-agnostic control over agent behaviors.
This paper introduces CoBRA, a novel toolkit for systematically specifying agent behavior in LLM-based social simulation. We found that conventional approaches that specify agent behaviors through implicit natural language descriptions cannot yield consistent behaviors across models, and the produced agent behaviors do not capture the nuances of the descriptions. In contrast, CoBRA presents a new approach to program agents' cognitive biases explicitly, by grounding agents' expected behaviors using classic social science experiments. CoBRA has two components: (1) Cognitive Bias Index that measures the cognitive bias of a social agent, by quantifying the agent's reactions in a set of validated classical social science experiments; (2) Behavioral Regulation Engine that aligns the agent's behavior to demonstrate controlled cognitive bias. We evaluated CoBRA as an HCI toolkit through demonstration and technical benchmarks. Our results suggest that CoBRA can precisely program the cognitive bias demonstrated in a social agent in a model-agnostic manner.