Perfectly predicting ICU length of stay: too good to be true
This is an incremental critique exposing errors in a medical AI study, important for researchers and clinicians to prevent misleading applications in hospital management.
The paper critiques a study that claimed perfect prediction of ICU length of stay for lung cancer patients using machine learning, but identifies methodological flaws that lead to overly optimistic results, providing a corrected AUROC of 88.91%.
A paper of Alsinglawi et al was recently accepted and published in Scientific Reports. In this paper, the authors aim to predict length of stay (LOS), discretized into either long (> 7 days) or short stays (< 7 days), of lung cancer patients in an ICU department using various machine learning techniques. The authors claim to achieve perfect results with an Area Under the Receiver Operating Characteristic curve (AUROC) of 100% with a Random Forest (RF) classifier with ADASYN class balancing over sampling technique, which if accurate could have significant implications for hospital management. However, we have identified several methodological flaws within the manuscript which cause the results to be overly optimistic and would have serious consequences if used in a clinical practice. Moreover, the reporting of the methodology is unclear and many important details are missing from the manuscript, which makes reproduction extremely difficult. We highlight the effect these oversights have had on the result and provide a more believable result of 88.91% AUROC when these oversights are corrected.