Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples
This addresses the need for accountability in offline RL for high-stakes domains like healthcare, representing an incremental improvement by focusing on explainability within existing frameworks.
The paper tackles the problem of decision accountability in offline reinforcement learning, particularly in responsibility-sensitive settings like healthcare, by introducing the Accountable Offline Controller (AOC) that uses a dataset as a Decision Corpus for accountable control based on selected examples, and demonstrates its effectiveness in simulated and real-world healthcare scenarios with high performance.
Learning controllers with offline data in decision-making systems is an essential area of research due to its potential to reduce the risk of applications in real-world systems. However, in responsibility-sensitive settings such as healthcare, decision accountability is of paramount importance, yet has not been adequately addressed by the literature. This paper introduces the Accountable Offline Controller (AOC) that employs the offline dataset as the Decision Corpus and performs accountable control based on a tailored selection of examples, referred to as the Corpus Subset. AOC operates effectively in low-data scenarios, can be extended to the strictly offline imitation setting, and displays qualities of both conservation and adaptability. We assess AOC's performance in both simulated and real-world healthcare scenarios, emphasizing its capability to manage offline control tasks with high levels of performance while maintaining accountability.