Abdulaziz Ahmed

Abdulaziz Ahmed, Mohammed Al-Maamari, Mohammad Firouz et al.

Patient triage at emergency departments (EDs) is necessary to prioritize care for patients with critical and time-sensitive conditions. Different tools are used for patient triage and one of the most common ones is the emergency severity index (ESI), which has a scale of five levels, where level 1 is the most urgent and level 5 is the least urgent. This paper proposes a framework for utilizing machine learning to develop an e-triage tool that can be used at EDs. A large retrospective dataset of ED patient visits is obtained from the electronic health record of a healthcare provider in the Midwest of the US for three years. However, the main challenge of using machine learning algorithms is that most of them have many parameters and without optimizing these parameters, developing a high-performance model is not possible. This paper proposes an approach to optimize the hyperparameters of machine learning. The metaheuristic optimization algorithms simulated annealing (SA) and adaptive simulated annealing (ASA) are proposed to optimize the parameters of extreme gradient boosting (XGB) and categorical boosting (CaB). The newly proposed algorithms are SA-XGB, ASA-XGB, SA-CaB, ASA-CaB. Grid search (GS), which is a traditional approach used for machine learning fine-tunning is also used to fine-tune the parameters of XGB and CaB, which are named GS-XGB and GS-CaB. The six algorithms are trained and tested using eight data groups obtained from the feature selection phase. The results show ASA-CaB outperformed all the proposed algorithms with accuracy, precision, recall, and f1 of 83.3%, 83.2%, 83.3%, 83.2%, respectively.

Abdulaziz Ahmed, Khalid Y. Aram, Salih Tutun

The issue of left before treatment complete (LBTC) patients is common in emergency departments (EDs). This issue represents a medico-legal risk and may cause a revenue loss. Thus, understanding the factors that cause patients to leave before treatment is complete is vital to mitigate and potentially eliminate these adverse effects. This paper proposes a framework for studying the factors that affect LBTC outcomes in EDs. The framework integrates machine learning, metaheuristic optimization, and model interpretation techniques. Metaheuristic optimization is used for hyperparameter optimization--one of the main challenges of machine learning model development. Three metaheuristic optimization algorithms are employed for optimizing the parameters of extreme gradient boosting (XGB), which are simulated annealing (SA), adaptive simulated annealing (ASA), and adaptive tabu simulated annealing (ATSA). The optimized XGB models are used to predict the LBTC outcomes for the patients under treatment in ED. The designed algorithms are trained and tested using four data groups resulting from the feature selection phase. The model with the best predictive performance is interpreted using SHaply Additive exPlanations (SHAP) method. The findings show that ATSA-XGB outperformed other mode configurations with an accuracy, area under the curve (AUC), sensitivity, specificity, and F1-score of 86.61%, 87.50%, 85.71%, 87.51%, and 86.60%, respectively. The degree and the direction of effects of each feature were determined and explained using the SHAP method.

Hieu Nghiem, Hemanth Reddy Singareddy, Zhuqi Miao et al.

Objective: Develop a cost-effective, large language model (LLM)-based pipeline for automatically extracting Review of Systems (ROS) entities from clinical notes. Materials and Methods: The pipeline extracts ROS sections using SecTag, followed by few-shot LLMs to identify ROS entity spans, their positive/negative status, and associated body systems. We implemented the pipeline using open-source LLMs (Mistral, Llama, Gemma) and ChatGPT. The evaluation was conducted on 36 general medicine notes containing 341 annotated ROS entities. Results: When integrating ChatGPT, the pipeline achieved the lowest error rates in detecting ROS entity spans and their corresponding statuses/systems (28.2% and 14.5%, respectively). Open-source LLMs enable local, cost-efficient execution of the pipeline while delivering promising performance with similarly low error rates (span: 30.5-36.7%; status/system: 24.3-27.3%). Discussion and Conclusion: Our pipeline offers a scalable and locally deployable solution to reduce ROS documentation burden. Open-source LLMs present a viable alternative to commercial models in resource-limited healthcare environments.

Orhun Vural, Bunyamin Ozaydin, Khalid Y. Aram et al.

Background: Emergency department (ED) overcrowding remains a major challenge, causing delays in care and increased operational strain. Hospital management often reacts to congestion after it occurs. Machine learning predictive modeling offers a proactive approach by forecasting patient flow metrics, such as waiting count, to improve resource planning and hospital efficiency. Objective: This study develops machine learning models to predict ED waiting room occupancy at two time scales. The hourly model forecasts the waiting count six hours ahead (e.g., a 1 PM prediction for 7 PM), while the daily model estimates the average waiting count for the next 24 hours (e.g., a 5 PM prediction for the following day's average). These tools support staffing decisions and enable earlier interventions to reduce overcrowding. Methods: Data from a partner hospital's ED in the southeastern United States were used, integrating internal metrics and external features. Eleven machine learning algorithms, including traditional and deep learning models, were trained and evaluated. Feature combinations were optimized, and performance was assessed across varying patient volumes and hours. Results: TSiTPlus achieved the best hourly prediction (MAE: 4.19, MSE: 29.32). The mean hourly waiting count was 18.11, with a standard deviation of 9.77. Accuracy varied by hour, with MAEs ranging from 2.45 (11 PM) to 5.45 (8 PM). Extreme case analysis at one, two, and three standard deviations above the mean showed MAEs of 6.16, 10.16, and 15.59, respectively. For daily predictions, XCMPlus performed best (MAE: 2.00, MSE: 6.64), with a daily mean of 18.11 and standard deviation of 4.51. Conclusions: These models accurately forecast ED waiting room occupancy and support proactive resource allocation. Their implementation has the potential to improve patient flow and reduce overcrowding in emergency care settings.

Abdulaziz Ahmed, Mohammad Saleem, Mohammed Alzeen et al.

Importance: Emergency department (ED) returns for mental health conditions pose a major healthcare burden, with 24-27% of patients returning within 30 days. Traditional machine learning models for predicting these returns often lack interpretability for clinical use. Objective: To assess whether integrating large language models (LLMs) with machine learning improves predictive accuracy and clinical interpretability of ED mental health return risk models. Methods: This retrospective cohort study analyzed 42,464 ED visits for 27,904 unique mental health patients at an academic medical center in the Deep South from January 2018 to December 2022. Main Outcomes and Measures: Two primary outcomes were evaluated: (1) 30-day ED return prediction accuracy and (2) model interpretability using a novel LLM-enhanced framework integrating SHAP (SHapley Additive exPlanations) values with clinical knowledge. Results: For chief complaint classification, LLaMA 3 (8B) with 10-shot learning outperformed traditional models (accuracy: 0.882, F1-score: 0.86). In SDoH classification, LLM-based models achieved 0.95 accuracy and 0.96 F1-score, with Alcohol, Tobacco, and Substance Abuse performing best (F1: 0.96-0.89), while Exercise and Home Environment showed lower performance (F1: 0.70-0.67). The LLM-based interpretability framework achieved 99% accuracy in translating model predictions into clinically relevant explanations. LLM-extracted features improved XGBoost AUC from 0.74 to 0.76 and AUC-PR from 0.58 to 0.61. Conclusions and Relevance: Integrating LLMs with machine learning models yielded modest but consistent accuracy gains while significantly enhancing interpretability through automated, clinically relevant explanations. This approach provides a framework for translating predictive analytics into actionable clinical insights.

Orhun Vural, Bunyamin Ozaydin, James Booth et al.

This study presents a deep learning-based framework for predicting emergency department (ED) boarding counts six hours in advance using only operational and contextual data, without patient-level information. Data from ED tracking systems, inpatient census, weather, holidays, and local events were aggregated hourly and processed with comprehensive feature engineering. The mean ED boarding count was 28.7 (standard deviation = 11.2). Multiple deep learning models, including ResNetPlus, TSTPlus, and TSiTPlus, were trained and optimized using Optuna, with TSTPlus achieving the best results (mean absolute error = 4.30, mean squared error = 29.47, R2 = 0.79). The framework accurately forecasted boarding counts, including during extreme periods, and demonstrated that broader input features improve predictive accuracy. This approach supports proactive hospital management and offers a practical method for mitigating ED overcrowding.

Abdulaziz Ahmed, Omar Ashour, Haneen Ali et al.

This work proposes a framework for optimizing machine learning algorithms. The practicality of the framework is illustrated using an important case study from the healthcare domain, which is predicting the admission status of emergency department (ED) patients (e.g., admitted vs. discharged) using patient data at the time of triage. The proposed framework can mitigate the crowding problem by proactively planning the patient boarding process. A large retrospective dataset of patient records is obtained from the electronic health record database of all ED visits over three years from three major locations of a healthcare provider in the Midwest of the US. Three machine learning algorithms are proposed: T-XGB, T-ADAB, and T-MLP. T-XGB integrates extreme gradient boosting (XGB) and Tabu Search (TS), T-ADAB integrates Adaboost and TS, and T-MLP integrates multi-layer perceptron (MLP) and TS. The proposed algorithms are compared with the traditional algorithms: XGB, ADAB, and MLP, in which their parameters are tunned using grid search. The three proposed algorithms and the original ones are trained and tested using nine data groups that are obtained from different feature selection methods. In other words, 54 models are developed. Performance was evaluated using five measures: Area under the curve (AUC), sensitivity, specificity, F1, and accuracy. The results show that the newly proposed algorithms resulted in high AUC and outperformed the traditional algorithms. The T-ADAB performs the best among the newly developed algorithms. The AUC, sensitivity, specificity, F1, and accuracy of the best model are 95.4%, 99.3%, 91.4%, 95.2%, 97.2%, respectively.