UMMS Affiliation

Department of Medicine, Division of Cardiovascular Medicine; Department of Population and Quantitative Health Sciences; Meyers Primary Care Institute

Publication Date


Document Type



Artificial Intelligence and Robotics | Epidemiology | Health Information Technology | Health Services Administration | Health Services Research


BACKGROUND: Scalable and accurate health outcome prediction using electronic health record (EHR) data has gained much attention in research recently. Previous machine learning models mostly ignore relations between different types of clinical data (ie, laboratory components, International Classification of Diseases codes, and medications).

OBJECTIVE: This study aimed to model such relations and build predictive models using the EHR data from intensive care units. We developed innovative neural network models and compared them with the widely used logistic regression model and other state-of-the-art neural network models to predict the patient's mortality using their longitudinal EHR data.

METHODS: We built a set of neural network models that we collectively called as long short-term memory (LSTM) outcome prediction using comprehensive feature relations or in short, CLOUT. Our CLOUT models use a correlational neural network model to identify a latent space representation between different types of discrete clinical features during a patient's encounter and integrate the latent representation into an LSTM-based predictive model framework. In addition, we designed an ablation experiment to identify risk factors from our CLOUT models. Using physicians' input as the gold standard, we compared the risk factors identified by both CLOUT and logistic regression models.

RESULTS: Experiments on the Medical Information Mart for Intensive Care-III dataset (selected patient population: 7537) show that CLOUT (area under the receiver operating characteristic curve=0.89) has surpassed logistic regression (0.82) and other baseline NN models ( < 0.86). In addition, physicians' agreement with the CLOUT-derived risk factor rankings was statistically significantly higher than the agreement with the logistic regression model.

CONCLUSIONS: Our results support the applicability of CLOUT for real-world clinical use in identifying patients at high risk of mortality.


ablation, neural networks, patient mortality, predictive modeling

Rights and Permissions

© Subendhu Rongali, Adam J Rose, David D McManus, Adarsha S Bajracharya, Alok Kapoor, Edgard Granillo, Hong Yu. Originally published in the Journal of Medical Internet Research (, 23.03.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.

DOI of Published Version



Rongali S, Rose AJ, McManus DD, Bajracharya AS, Kapoor A, Granillo E, Yu H. Learning Latent Space Representations to Predict Patient Outcomes: Model Development and Validation. J Med Internet Res. 2020 Mar 23;22(3):e16374. doi: 10.2196/16374. PMID: 32202503; PMCID: PMC7136840. Link to article on publisher's site

Journal/Book/Conference Title

Journal of medical Internet research

Related Resources

Link to Article in PubMed

PubMed ID


Creative Commons License

Creative Commons Attribution 4.0 License
This work is licensed under a Creative Commons Attribution 4.0 License.