Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Models
This addresses a bottleneck in experimental design for fields like healthcare and robotics, where experiments are costly or hazardous, but it is incremental as it builds on prior policy-based BOED methods.
The paper tackled the problem of designing experiments for non-differentiable implicit models, where traditional methods fail due to intractable likelihoods or differentiability requirements, and introduced RL-DAD, which performs competitively with baselines on three benchmarks.
For applications in healthcare, physics, energy, robotics, and many other fields, designing maximally informative experiments is valuable, particularly when experiments are expensive, time-consuming, or pose safety hazards. While existing approaches can sequentially design experiments based on prior observation history, many of these methods do not extend to implicit models, where simulation is possible but computing the likelihood is intractable. Furthermore, they often require either significant online computation during deployment or a differentiable simulation system. We introduce Reinforcement Learning for Deep Adaptive Design (RL-DAD), a method for simulation-based optimal experimental design for non-differentiable implicit models. RL-DAD extends prior work in policy-based Bayesian Optimal Experimental Design (BOED) by reformulating it as a Markov Decision Process with a reward function based on likelihood-free information lower bounds, which is used to learn a policy via deep reinforcement learning. The learned design policy maps prior histories to experiment designs offline and can be quickly deployed during online execution. We evaluate RL-DAD and find that it performs competitively with baselines on three benchmarks.