Reinforcement Learning with Efficient Active Feature Acquisition
This addresses the challenge of costly information acquisition in real-life domains like healthcare, offering an incremental improvement over existing methods.
The paper tackles the problem of sequential decision-making under partial observability where acquiring information is costly, proposing a model-based reinforcement learning framework with a sequential variational auto-encoder to learn active feature acquisition policies that maximize task reward efficiently. The method outperforms conventional baselines in control and medical simulator tasks, achieving greater cost efficiency.
Solving real-life sequential decision making problems under partial observability involves an exploration-exploitation problem. To be successful, an agent needs to efficiently gather valuable information about the state of the world for making rewarding decisions. However, in real-life, acquiring valuable information is often highly costly, e.g., in the medical domain, information acquisition might correspond to performing a medical test on a patient. This poses a significant challenge for the agent to perform optimally for the task while reducing the cost for information acquisition. In this paper, we propose a model-based reinforcement learning framework that learns an active feature acquisition policy to solve the exploration-exploitation problem during its execution. Key to the success is a novel sequential variational auto-encoder that learns high-quality representations from partially observed states, which are then used by the policy to maximize the task reward in a cost efficient manner. We demonstrate the efficacy of our proposed framework in a control domain as well as using a medical simulator. In both tasks, our proposed method outperforms conventional baselines and results in policies with greater cost efficiency.