Alvaro Ulloa

Alvaro Ulloa, Linyuan Jing, Christopher W Good et al.

Predicting future clinical events helps physicians guide appropriate intervention. Machine learning has tremendous promise to assist physicians with predictions based on the discovery of complex patterns from historical data, such as large, longitudinal electronic health records (EHR). This study is a first attempt to demonstrate such capabilities using raw echocardiographic videos of the heart. We show that a large dataset of 723,754 clinically-acquired echocardiographic videos (~45 million images) linked to longitudinal follow-up data in 27,028 patients can be used to train a deep neural network to predict 1-year mortality with good accuracy (area under the curve (AUC) in an independent test set = 0.839). Prediction accuracy was further improved by adding EHR data (AUC = 0.858). Finally, we demonstrate that the trained neural network was more accurate in mortality prediction than two expert cardiologists. These results highlight the potential of neural networks to add new power to clinical predictions.

Alvaro Ulloa, Sergey Plis, Vince Calhoun

The wide variety of brain imaging technologies allows us to exploit information inherent to different data modalities. The richness of multimodal datasets may increase predictive power and reveal latent variables that otherwise would have not been found. However, the analysis of multimodal data is often conducted by assuming linear interactions which impact the accuracy of the results. We propose the use of a multimodal multi-layer perceptron model to enhance the predictive power of structural and functional magnetic resonance imaging (sMRI and fMRI) combined. We also use a synthetic data generator to pre-train each modality input layers, alleviating the effects of the small sample size that is often the case for brain imaging modalities. The proposed model improved the average and uncertainty of the area under the ROC curve to 0.850+-0.051 compared to the best results on individual modalities (0.741+-0.075 for sMRI, and 0.833+-0.050 for fMRI).