DC3 -- A Diagnostic Case Challenge Collection for Clinical Decision Support
This addresses the need for transparent and standardized evaluation methods in clinical decision support to reduce diagnostic errors, though it is incremental as it provides a new dataset rather than a novel method.
The paper tackles the problem of evaluating diagnostic decision support systems by introducing DC3, a collection of 31 difficult diagnostic case challenges compiled by clinical experts, which includes physician observations and relevance judgments to PubMed articles.
In clinical care, obtaining a correct diagnosis is the first step towards successful treatment and, ultimately, recovery. Depending on the complexity of the case, the diagnostic phase can be lengthy and ridden with errors and delays. Such errors have a high likelihood to cause patients severe harm or even lead to their death and are estimated to cost the U.S. healthcare system several hundred billion dollars each year. To avoid diagnostic errors, physicians increasingly rely on diagnostic decision support systems drawing from heuristics, historic cases, textbooks, clinical guidelines and scholarly biomedical literature. The evaluation of such systems, however, is often conducted in an ad-hoc fashion, using non-transparent methodology, and proprietary data. This paper presents DC3, a collection of 31 extremely difficult diagnostic case challenges, manually compiled and solved by clinical experts. For each case, we present a number of temporally ordered physician-generated observations alongside the eventually confirmed true diagnosis. We additionally provide inferred dense relevance judgments for these cases among the PubMed collection of 27 million scholarly biomedical articles.