Dialogue-Contextualized Re-ranking for Medical History-Taking
This addresses the problem of data scarcity and mismatch in medical history-taking for AI virtual care applications, representing an incremental improvement over existing methods.
The paper tackled the training-inference gap in AI-driven medical history-taking by proposing a two-stage re-ranking approach with a dialogue-contextualized model, achieving a 30% higher nDCG and 77% higher mAP compared to an expert system.
AI-driven medical history-taking is an important component in symptom checking, automated patient intake, triage, and other AI virtual care applications. As history-taking is extremely varied, machine learning models require a significant amount of data to train. To overcome this challenge, existing systems are developed using indirect data or expert knowledge. This leads to a training-inference gap as models are trained on different kinds of data than what they observe at inference time. In this work, we present a two-stage re-ranking approach that helps close the training-inference gap by re-ranking the first-stage question candidates using a dialogue-contextualized model. For this, we propose a new model, global re-ranker, which cross-encodes the dialogue with all questions simultaneously, and compare it with several existing neural baselines. We test both transformer and S4-based language model backbones. We find that relative to the expert system, the best performance is achieved by our proposed global re-ranker with a transformer backbone, resulting in a 30% higher normalized discount cumulative gain (nDCG) and a 77% higher mean average precision (mAP).