UMMS Affiliation

Department of Population and Quantitative Health Sciences; Meyers Primary Care Institute; Department of Medicine; Department of Pediatrics

Publication Date


Document Type



Health Communication | Health Information Technology | Health Services Administration | Health Services Research | Influenza Virus Vaccines


BACKGROUND: Patient reminders for influenza vaccination, delivered via an electronic health record patient portal and interactive voice response calls, offer an innovative approach to engaging patients and improving patient care.

OBJECTIVE: The goal of this study was to test the effectiveness of portal and interactive voice response outreach in improving rates of influenza vaccination by targeting patients in early September, shortly after vaccinations became available.

METHODS: Using electronic health record portal messages and interactive voice response calls promoting influenza vaccination, outreach was conducted in September 2015. Participants included adult patients within a large multispecialty group practice in central Massachusetts. Our main outcome was electronic health record-documented early influenza vaccination during the 2015-2016 influenza season, measured in November 2015. We randomly assigned all active portal users to 1 of 2 groups: (1) receiving a portal message promoting influenza vaccinations, listing upcoming clinics, and offering online scheduling of vaccination appointments (n=19,506) or (2) receiving usual care (n=19,505). We randomly assigned all portal nonusers to 1 of 2 groups: (1) receiving interactive voice response call (n=15,000) or (2) receiving usual care (n=43,596). The intervention also solicited patient self-reports on influenza vaccinations completed outside the clinic. Self-reported influenza vaccination data were uploaded into the electronic health records to increase the accuracy of existing provider-directed electronic health record clinical decision support (vaccination alerts) but were excluded from main analyses.

RESULTS: Among portal users, 28.4% (5549/19,506) of those randomized to receive messages and 27.1% (5294/19,505) of the usual care group had influenza vaccinations documented by November 2015 (P=.004). In multivariate analysis of portal users, message recipients were slightly more likely to have documented vaccinations when compared to the usual care group (OR 1.07, 95% CI 1.02-1.12). Among portal nonusers, 8.4% (1262/15,000) of those randomized to receive calls and 8.2% (3586/43,596) of usual care had documented vaccinations (P=.47), and multivariate analysis showed nonsignificant differences. Over half of portal messages sent were opened (10,112/19,479; 51.9%), and over half of interactive voice response calls placed (7599/14,984; 50.7%) reached their intended target, thus we attained similar levels of exposure to the messaging for both interventions. Among portal message recipients, 25.4% of message openers (2570/10,112) responded to a subsequent question on receipt of influenza vaccination; among interactive voice response recipients, 72.5% of those reached (5513/7599) responded to a similar question.

CONCLUSIONS: Portal message outreach to a general primary care population achieved a small but statistically significant improvement in rates of influenza vaccination (OR 1.07, 95% CI 1.02-1.12). Interactive voice response calls did not significantly improve vaccination rates among portal nonusers (OR 1.03, 95% CI 0.96-1.10). Rates of patient engagement with both modalities were favorable.

TRIAL REGISTRATION: NCT02266277; Lloyd Fisher, Peggy Preusse, Devi Sundaresan, Lawrence Garber, Kathleen M Mazor, Sarah L Cutrona. Originally published in the Journal of Medical Internet Research (, 25.09.2020.


electronic health records, influenza vaccination, patient care, patient engagement

Rights and Permissions

Copyright © Jessica G Wijesundara, Mayuko Ito Fukunaga, Jessica Ogarek, Bruce Barton, Lloyd Fisher, Peggy Preusse, Devi Sundaresan, Lawrence Garber, Kathleen M Mazor, Sarah L Cutrona. Originally published in the Journal of Medical Internet Research (, 25.09.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.

DOI of Published Version



Wijesundara JG, Ito Fukunaga M, Ogarek J, Barton B, Fisher L, Preusse P, Sundaresan D, Garber L, Mazor KM, Cutrona SL. Electronic Health Record Portal Messages and Interactive Voice Response Calls to Improve Rates of Early Season Influenza Vaccination: Randomized Controlled Trial. J Med Internet Res. 2020 Sep 25;22(9):e16373. doi: 10.2196/16373. PMID: 32975529; PMCID: PMC7547389. Link to article on publisher's site

Journal/Book/Conference Title

Journal of medical Internet research

Related Resources

Link to Article in PubMed

PubMed ID


Creative Commons License

Creative Commons Attribution 4.0 License
This work is licensed under a Creative Commons Attribution 4.0 License.