UMMS Affiliation

Department of Population and Quantitative Health Sciences

Publication Date


Document Type

Conference Proceeding


Artificial Intelligence and Robotics | Biostatistics | Chemicals and Drugs | Epidemiology | Health Services Research | Patient Safety


Clinical judgment studies are an integral part of drug safety surveillance and pharmacovigilance frameworks. They help quantify the causal relationship between medication and its adverse drug reactions (ADRs). To conduct such studies, physicians need to review patients' charts manually to answer Naranjo questionnaire(1). In this paper, we propose a methodology to automatically infer causal relations from patients' discharge summaries by combining the capabilities of deep learning and statistical learning models. We use Bidirectional Encoder Representations from Transformers (BERT)(2) to extract relevant paragraphs for each Naranjo question and then use a statistical learning model such as logistic regression to predict the Naranjo score and the causal relation between the medication and an ADR. Our methodology achieves a macro-averaged f1-score of 0.50 and weighted f1-score of 0.63.


adverse drug reactions, Naranjo questionnaire, patient discharge summaries, deep learning, statistical learning

Rights and Permissions

Copyright ©2020 AMIA. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose.


Rawat BPS, Jagannatha A, Liu F, Yu H. Inferring ADR causality by predicting the Naranjo Score from Clinical Notes. AMIA Annu Symp Proc. 2021 Jan 25;2020:1041-1049. PMID: 33936480; PMCID: PMC8075501.

Journal/Book/Conference Title

AMIA ... Annual Symposium proceedings. AMIA Symposium

PubMed ID


Related Resources

Link to Article in PubMed