Off-Policy Selection for Initiating Human-Centric Experimental Design
This addresses the challenge of policy selection in heterogeneous human-centric systems like healthcare and education, representing an incremental improvement over existing off-policy selection methods.
The paper tackles the problem of selecting personalized policies for new participants in human-centric systems without prior offline data, by introducing First-Glance Off-Policy Selection (FPS), which uses sub-group segmentation to improve outcomes in intelligent tutoring and sepsis treatment applications.
In human-centric tasks such as healthcare and education, the heterogeneity among patients and students necessitates personalized treatments and instructional interventions. While reinforcement learning (RL) has been utilized in those tasks, off-policy selection (OPS) is pivotal to close the loop by offline evaluating and selecting policies without online interactions, yet current OPS methods often overlook the heterogeneity among participants. Our work is centered on resolving a pivotal challenge in human-centric systems (HCSs): how to select a policy to deploy when a new participant joining the cohort, without having access to any prior offline data collected over the participant? We introduce First-Glance Off-Policy Selection (FPS), a novel approach that systematically addresses participant heterogeneity through sub-group segmentation and tailored OPS criteria to each sub-group. By grouping individuals with similar traits, FPS facilitates personalized policy selection aligned with unique characteristics of each participant or group of participants. FPS is evaluated via two important but challenging applications, intelligent tutoring systems and a healthcare application for sepsis treatment and intervention. FPS presents significant advancement in enhancing learning outcomes of students and in-hospital care outcomes.