A Decision-Theoretic Model for Using Scientific Data
This work addresses the challenge of integrating scientific evidence into decision-making for AI systems, particularly in medicine, offering a foundational model that could support advisory systems, statistical workstations, and meta-analyses, though it is incremental in applying existing decision analysis to a specific domain.
The paper tackles the problem of how AI agents should update beliefs using scientific data, presenting a decision-theoretic framework that structures this process for medical research, extending influence diagrams to model parameters like patient, population, and study sample with biases. It provides justification for clinical practices like randomization and blindfolding, covering designs such as case-control studies and clinical trials.
Many Artificial Intelligence systems depend on the agent's updating its beliefs about the world on the basis of experience. Experiments constitute one type of experience, so scientific methodology offers a natural environment for examining the issues attendant to using this class of evidence. This paper presents a framework which structures the process of using scientific data from research reports for the purpose of making decisions, using decision analysis as the basis for the structure and using medical research as the general scientific domain. The structure extends the basic influence diagram for updating belief in an object domain parameter of interest by expanding the parameter into four parts: those of the patient, the population, the study sample, and the effective study sample. The structure uses biases to perform the transformation of one parameter into another, so that, for instance, selection biases, in concert with the population parameter, yield the study sample parameter. The influence diagram structure provides decision theoretic justification for practices of good clinical research such as randomized assignment and blindfolding of care providers. The model covers most research designs used in medicine: case-control studies, cohort studies, and controlled clinical trials, and provides an architecture to separate clearly between statistical knowledge and domain knowledge. The proposed general model can be the basis for clinical epidemiological advisory systems, when coupled with heuristic pruning of irrelevant biases; of statistical workstations, when the computational machinery for calculation of posterior distributions is added; and of meta-analytic reviews, when multiple studies may impact on a single population parameter.