UMMS Affiliation

Department of Quantitative Health Sciences; Division of Cardiovascular Medicine, Department of Medicine

Publication Date


Document Type



Artificial Intelligence and Robotics | Cardiovascular Diseases | Databases and Information Systems | Health Information Technology | Health Services Administration | Pathological Conditions, Signs and Symptoms


BACKGROUND: Bleeding events are common and critical and may cause significant morbidity and mortality. High incidences of bleeding events are associated with cardiovascular disease in patients on anticoagulant therapy. Prompt and accurate detection of bleeding events is essential to prevent serious consequences. As bleeding events are often described in clinical notes, automatic detection of bleeding events from electronic health record (EHR) notes may improve drug-safety surveillance and pharmacovigilance.

OBJECTIVE: We aimed to develop a natural language processing (NLP) system to automatically classify whether an EHR note sentence contains a bleeding event.

METHODS: We expert annotated 878 EHR notes (76,577 sentences and 562,630 word-tokens) to identify bleeding events at the sentence level. This annotated corpus was used to train and validate our NLP systems. We developed an innovative hybrid convolutional neural network (CNN) and long short-term memory (LSTM) autoencoder (HCLA) model that integrates a CNN architecture with a bidirectional LSTM (BiLSTM) autoencoder model to leverage large unlabeled EHR data.

RESULTS: HCLA achieved the best area under the receiver operating characteristic curve (0.957) and F1 score (0.938) to identify whether a sentence contains a bleeding event, thereby surpassing the strong baseline support vector machines and other CNN and autoencoder models.

CONCLUSIONS: By incorporating a supervised CNN model and a pretrained unsupervised BiLSTM autoencoder, the HCLA achieved high performance in detecting bleeding events.


BiLSTM, autoencoder, bleeding, convolutional neural networks, electronic health record

Rights and Permissions

© Rumeng Li, Baotian Hu, Feifan Liu, Weisong Liu, Francesca Cunningham, David D McManus, Hong Yu. Originally published in JMIR Medical Informatics (, 02.02.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.

DOI of Published Version



JMIR Med Inform. 2019 Feb 8;7(1):e10788. doi: 10.2196/10788. Link to article on publisher's site

Journal/Book/Conference Title

JMIR medical informatics

Related Resources

Link to Article in PubMed

PubMed ID


Creative Commons License

Creative Commons Attribution 4.0 License
This work is licensed under a Creative Commons Attribution 4.0 License.