Bayesian Optimal Experimental Design for Simulator Models of Cognition
This work addresses the challenge of designing informative experiments for realistic but intractable cognitive models, which is an incremental advance for cognitive science and related fields.
The paper tackled the problem of applying Bayesian optimal experimental design to intractable simulator models of human cognition, combining BOED with approximate inference to achieve improved model discrimination and parameter estimation in multi-armed bandit tasks.
Bayesian optimal experimental design (BOED) is a methodology to identify experiments that are expected to yield informative data. Recent work in cognitive science considered BOED for computational models of human behavior with tractable and known likelihood functions. However, tractability often comes at the cost of realism; simulator models that can capture the richness of human behavior are often intractable. In this work, we combine recent advances in BOED and approximate inference for intractable models, using machine-learning methods to find optimal experimental designs, approximate sufficient summary statistics and amortized posterior distributions. Our simulation experiments on multi-armed bandit tasks show that our method results in improved model discrimination and parameter estimation, as compared to experimental designs commonly used in the literature.