EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE
This addresses decision-making in cost-sensitive scenarios like healthcare, where incremental improvements in efficiency can reduce expenses or enhance outcomes.
The paper tackles the problem of efficiently acquiring additional information for decision-making under cost constraints, proposing the EDDI framework which uses a Partial VAE and acquisition function to optimize information gain. Results show cost reduction at the same decision quality and improved decision quality at the same cost in benchmarks and healthcare applications.
Many real-life decision-making situations allow further relevant information to be acquired at a specific cost, for example, in assessing the health status of a patient we may decide to take additional measurements such as diagnostic tests or imaging scans before making a final assessment. Acquiring more relevant information enables better decision making, but may be costly. How can we trade off the desire to make good decisions by acquiring further information with the cost of performing that acquisition? To this end, we propose a principled framework, named EDDI (Efficient Dynamic Discovery of high-value Information), based on the theory of Bayesian experimental design. In EDDI, we propose a novel partial variational autoencoder (Partial VAE) to predict missing data entries problematically given any subset of the observed ones, and combine it with an acquisition function that maximizes expected information gain on a set of target variables. We show cost reduction at the same decision quality and improved decision quality at the same cost in multiple machine learning benchmarks and two real-world health-care applications.