Models we Can Trust: Toward a Systematic Discipline of (Agent-Based) Model Interpretation and Validation
This addresses the need for more rigorous validation and interpretation methods in social science modeling, particularly for agent-based models, but is incremental as it outlines foundational directions rather than presenting new results.
The paper advocates for establishing a systematic discipline to interpret and validate agent-based and mathematical models in social science, proposing directions such as logical frameworks for specifying stylized facts, adapting reactive systems tools for behavioral equivalence, and developing adversarial perturbation theories to assess robustness.
We advocate the development of a discipline of interacting with and extracting information from models, both mathematical (e.g. game-theoretic ones) and computational (e.g. agent-based models). We outline some directions for the development of a such a discipline: - the development of logical frameworks for the systematic formal specification of stylized facts and social mechanisms in (mathematical and computational) social science. Such frameworks would bring to attention new issues, such as phase transitions, i.e. dramatical changes in the validity of the stylized facts beyond some critical values in parameter space. We argue that such statements are useful for those logical frameworks describing properties of ABM. - the adaptation of tools from the theory of reactive systems (such as bisimulation) to obtain practically relevant notions of two systems "having the same behavior". - the systematic development of an adversarial theory of model perturbations, that investigates the robustness of conclusions derived from models of social behavior to variations in several features of the social dynamics. These may include: activation order, the underlying social network, individual agent behavior.