minoHealth.ai: A Clinical Evaluation Of Deep Learning Systems For the Diagnosis of Pleural Effusion and Cardiomegaly In Ghana, Vietnam and the United States of America
This addresses the need for rapid and accurate diagnosis of these conditions to reduce mortality and costs, but it is incremental as it applies existing AI methods to new clinical data across different regions.
The study evaluated deep learning systems for diagnosing cardiomegaly and pleural effusion on chest x-rays from Ghana, Vietnam, and the USA, finding that the AI models achieved AUC-ROC scores up to 0.97 and outperformed radiologists by about 10%.
A rapid and accurate diagnosis of cardiomegaly and pleural effusion is of the utmost importance to reduce mortality and medical costs. Artificial Intelligence has shown promise in diagnosing medical conditions. With this study, we seek to evaluate how well Artificial Intelligence (AI) systems, developed my minoHealth AI Labs, will perform at diagnosing cardiomegaly and pleural effusion, using chest x-rays from Ghana, Vietnam and the USA, and how well AI systems will perform when compared with radiologists working in Ghana. The evaluation dataset used in this study contained 100 images randomly selected from three datasets. The Deep Learning models were further tested on a larger Ghanaian dataset containing five hundred and sixty one (561) samples. Two AI systems were then evaluated on the evaluation dataset, whilst we also gave the same chest x-ray images within the evaluation dataset to 4 radiologists, with 5 - 20 years experience, to diagnose independently. For cardiomegaly, minoHealth-ai systems scored Area under the Receiver operating characteristic Curve (AUC-ROC) of 0.9 and 0.97 while the AUC-ROC of individual radiologists ranged from 0.77 to 0.87. For pleural effusion, the minoHealth-ai systems scored 0.97 and 0.91 whereas individual radiologists scored between 0.75 and 0.86. On both conditions, the best performing AI model outperforms the best performing radiologist by about 10%. We also evaluate the specificity, sensitivity, negative predictive value (NPV), and positive predictive value (PPV) between the minoHealth-ai systems and radiologists.