The Three Ghosts of Medical AI: Can the Black-Box Present Deliver?
This paper addresses the critical problem of transparency in medical AI for clinicians and patients, highlighting fundamental issues with current black-box models.
This paper argues that current opaque AI models in healthcare fail to meet transparency needs for clinicians and patients. It identifies three key implications: lack of quality assurance, failure to elicit trust, and restriction of physician-patient dialogue.
Our title alludes to the three Christmas ghosts encountered by Ebenezer Scrooge in \textit{A Christmas Carol}, who guide Ebenezer through the past, present, and future of Christmas holiday events. Similarly, our article will take readers through a journey of the past, present, and future of medical AI. In doing so, we focus on the crux of modern machine learning: the reliance on powerful but intrinsically opaque models. When applied to the healthcare domain, these models fail to meet the needs for transparency that their clinician and patient end-users require. We review the implications of this failure, and argue that opaque models (1) lack quality assurance, (2) fail to elicit trust, and (3) restrict physician-patient dialogue. We then discuss how upholding transparency in all aspects of model design and model validation can help ensure the reliability of medical AI.