The accuracy vs. coverage trade-off in patient-facing diagnosis models
This addresses the need for accurate and comprehensive online symptom checkers for patients and physicians, but it is incremental as it focuses on quantifying a known trade-off.
The paper tackles the trade-off between coverage and accuracy in patient-facing diagnosis models, finding that top-3 accuracy drops by 1% for every 10 diseases added to coverage, and linear models perform as well as neural networks.
A third of adults in America use the Internet to diagnose medical concerns, and online symptom checkers are increasingly part of this process. These tools are powered by diagnosis models similar to clinical decision support systems, with the primary difference being the coverage of symptoms and diagnoses. To be useful to patients and physicians, these models must have high accuracy while covering a meaningful space of symptoms and diagnoses. To the best of our knowledge, this paper is the first in studying the trade-off between the coverage of the model and its performance for diagnosis. To this end, we learn diagnosis models with different coverage from EHR data. We find a 1\% drop in top-3 accuracy for every 10 diseases added to the coverage. We also observe that complexity for these models does not affect performance, with linear models performing as well as neural networks.