Optimizing Social Utility in Sequential Experiments
For regulators and product developers in high-stakes domains like drug development, this work addresses the inefficiency of costly trials that may deter socially valuable 'moonshot' products.
This paper introduces a sequential experimentation protocol where a regulator partially subsidizes a developer's trial costs, modeled as a belief Markov decision process. Simulations on antibiotic development data show the protocol can increase social utility by over 35% compared to standard non-sequential protocols.
Regulatory approval of products in high-stakes domains such as drug development requires statistical evidence of safety and efficacy through large-scale randomized controlled trials. However, the high financial cost of these trials may deter developers who lack absolute certainty in their product's efficacy, ultimately stifling the development of `moonshot' products that could offer high social utility. To address this inefficiency, in this paper, we introduce a statistical protocol for experimentation where the product developer (the agent) conducts a randomized controlled trial sequentially and the regulator (the principal) partially subsidizes its cost. By modeling the protocol using a belief Markov decision process, we show that the agent's optimal strategy can be found efficiently using dynamic programming. Further, we show that the social utility is a piecewise linear and convex function over the subsidy level the principal selects, and thus the socially optimal subsidy can also be found efficiently using divide-and-conquer. Simulation experiments using publicly available data on antibiotic development and approval demonstrate that our statistical protocol can be used to increase social utility by more than $35$$\%$ relative to standard, non-sequential protocols.