Optimal Learning for Sequential Decisions in Laboratory Experimentation
This is an incremental tutorial aimed at experimental scientists to improve decision-making in laboratory settings.
The tutorial addresses the slow and failure-prone process of scientific experimentation by introducing the concept of a learning policy, specifically the knowledge gradient, to maximize information value from each experiment, emphasizing the role of belief models and uncertainty reduction.
The process of discovery in the physical, biological and medical sciences can be painstakingly slow. Most experiments fail, and the time from initiation of research until a new advance reaches commercial production can span 20 years. This tutorial is aimed to provide experimental scientists with a foundation in the science of making decisions. Using numerical examples drawn from the experiences of the authors, the article describes the fundamental elements of any experimental learning problem. It emphasizes the important role of belief models, which include not only the best estimate of relationships provided by prior research, previous experiments and scientific expertise, but also the uncertainty in these relationships. We introduce the concept of a learning policy, and review the major categories of policies. We then introduce a policy, known as the knowledge gradient, that maximizes the value of information from each experiment. We bring out the importance of reducing uncertainty, and illustrate this process for different belief models.