WhatIf: Interactive Exploration of LLM-Powered Social Simulations for Policy Reasoning
For policymakers in high-uncertainty domains, WhatIf provides a novel interactive simulation paradigm that supports real-time exploration and collaborative reasoning, addressing limitations of existing static tools.
WhatIf enables policymakers to interactively steer, inspect, and compare LLM-powered social simulations in real time, supporting iterative exploration and vulnerability discovery in domains like emergency management. Evaluated with five professionals, it facilitated branching comparisons and surfaced tacit assumptions.
Policymakers in domains such as emergency management, public health, and urban planning must make decisions under deep uncertainty, where outcomes depend on how large populations interpret information, coordinate, and adopt over time. Existing tools only partially support this process: tabletop exercises enable collaborative discussion but lack dynamic feedback, while computational simulations capture population dynamics but are designed for offline analysis. We present WhatIf, an interactive system that enables policymakers to steer, inspect, and compare LLM-powered social simulations in real time. Informed by a formative study in emergency preparedness planning, we derive four design requirements for interactive policy simulations: fluid steering, real-time scale, collaborative exploration, and multi-level interpretability. We developed WhatIf guided by these requirements and evaluated it with five preparedness professionals across three disaster evacuation scenarios. Our findings show that participants used the system as a space for iterative branching and comparison rather than evaluating fixed plans; reflected on tacit planning assumptions when agent behavior violated expectations; surfaced previously unrecognized planning vulnerabilities; and grounded their reasoning in inspectable agent-level cases rather than aggregate outputs alone. These findings suggest broader design implications for LLM-powered social simulation systems: designing such systems as interactive, shared reasoning environments -- rather than offline predictive tools -- can better support expert decision-making under deep uncertainty.