Why Model Credibility Isn't Enough: -Rethinking Trust in Simulation Architectures
This work addresses the problem of evaluating trust in complex simulation architectures for practitioners and researchers in simulation and modeling, but it is an incremental survey without novel results or concrete numbers.
The paper argues that assessing the credibility of individual simulation models is insufficient for evaluating the credibility of the overall simulation architecture, and reviews current approaches for assembly credibility, comparing sensitivity analysis, expert qualitative analysis, AI explainability, and networks. It provides an assessment of these approaches based on rigor, generalization, and resource requirements.
Credibility of a simulation model is an important topic. Several approaches try to quantify the credibility of simulation. However, models are mostly assembled within a simulation architecture. Can the credibility of a simulation architecture be assessed based on the credibility of the models that comprise it? This paper aims to address this issue by providing an overview of the current state of the art in the field of assembly credibility. It will compare sensitivity analysis techniques, qualitative analysis by experts, explainability in AI, and networks. Finally, an assessment of the proposed approaches, based on criteria such as rigor, generalization, and resource requirements, will reveal the strengths and weaknesses of each approach.