Department of Emergency Medicine; School of Medicine
Emergency Medicine | Quality Improvement | Radiology
BACKGROUND: Audit and feedback (AandF) has been used as a strategy to modify clinician behavior with moderate success. Although AandF is theorized to work by improving the accuracy of clinicians' estimates of their own behavior, few interventions have included assessment of clinicians' estimates at baseline to examine whether they account for intervention success or failure. We tested an AandF intervention to reduce computed tomography (CT) ordering by emergency physicians, while also examining the physicians' baseline estimates of their own behavior compared to peers.
METHODS: Our study was a prospective, multi-site, 20-month, randomized trial to examine the effect of an AandF intervention on CT ordering rates, overall and by test subtype. From the electronic health record, we obtained 12 months of baseline CT ordering per 100 patients treated for every physician from four emergency departments. Those who were randomized to receive AandF were shown a de-identified graph of the group's baseline CT utilization, asked to estimate wherein the distribution of their own CT order practices fell, and then shown their actual performance. All participants also received a brief educational intervention. CT ordering rates were collected for all physicians for 6 months after the intervention. Pre-post ordering rates were compared using independent and repeated measures t tests.
RESULTS: Fifty-one of 52 eligible physicians participated. The mean CT ordering rate increased significantly in both experimental conditions after the intervention (intervention pre = 35.7, post = 40.3, t = 4.13, p < 0.001; control pre = 33.9, post = 38.9, t = 3.94, p = 0.001), with no significant between-group difference observed at follow-up (t = 0.43, p = 0.67). Within the intervention group, physicians had poor accuracy in estimating their own ordering behavior at baseline: most overestimated and all guessed that they were in the upper half of the distribution of their peers. CT ordering increased regardless of self-estimate accuracy.
CONCLUSIONS: Our AandF intervention failed to reduce physician CT ordering: our feedback to the physicians showed most of them that they had overestimated their CT ordering behavior, and they were therefore unlikely to reduce it as a result. After "audit," it may be prudent to assess baseline clinician awareness of behavior before moving toward a feedback intervention.
Audit and feedback, Emergency medicine, Physician ordering
Rights and Permissions
Copyright © The Author(s) 2021. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
DOI of Published Version
Larkin C, Sanseverino AM, Joseph J, Eisenhauer L, Reznek MA. Accuracy of emergency physicians' self-estimates of CT scan utilization and its potential effect on an audit and feedback intervention: a randomized trial. Implement Sci Commun. 2021 Jul 27;2(1):83. doi: 10.1186/s43058-021-00182-1. PMID: 34315533; PMCID: PMC8317272. Link to article on publisher's site
Implementation science communications
Larkin C, Sanseverino A, Joseph J, Eisenhauer L, Reznek MA. (2021). Accuracy of emergency physicians' self-estimates of CT scan utilization and its potential effect on an audit and feedback intervention: a randomized trial. UMass Chan Medical School Faculty Publications. https://doi.org/10.1186/s43058-021-00182-1. Retrieved from https://escholarship.umassmed.edu/faculty_pubs/2140
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.