Multi-Institution Validation of an Emergency Ultrasound Image Rating Scale-A Pilot Study
Department of Emergency Medicine
BACKGROUND: As bedside ultrasound (BUS) is being increasingly taught and incorporated into emergency medicine practice, measurement of BUS competency is becoming more important. The commonly adopted experiential approach to BUS competency has never been validated on a large scale, and has some limitations by design.
OBJECTIVE: Our aim was to introduce and report preliminary testing of a novel emergency BUS image rating scale (URS).
METHODS: Gallbladder BUS was selected as the test case. Twenty anonymous BUS image sets (still images and clips) were forwarded electronically to 16 reviewers (13 attendings, 3 fellows) at six training sites across the United States. Each reviewer rated the BUS sets using the pilot URS that consisted of three components, with numerical values assigned to each of the following aspects: Landmarks, Image Quality, and Annotations. Reviewers also decided whether or not each BUS set would be "Clinically Useful." Kendall taus were calculated as a measure of concordance among the reviewers.
RESULTS: Among the 13 attendings, image review experience ranged from 2-15 years, 5-300 scans per week, and averaged 7.8 years and 60 images. Kendall taus for each aspect of the URS were: Landmarks: 0.55; Image Quality: 0.57; Annotation: 0.26; Total Score: 0.63, and Clinical Usefulness: 0.45. All URS elements correlated significantly with clinical usefulness (p < 0.001). The correlation coefficient between each attending reviewer and the entire group ranged from 0.48-0.69, and was independent of image review experience beyond fellowship training.
CONCLUSION: Our novel URS had moderate-to-good inter-rater agreement in this pilot study. Based on these results, the URS will be modified for use in future investigations.
DOI of Published Version
J Emerg Med. 2015 Jul;49(1):32-39.e1. doi: 10.1016/j.jemermed.2015.01.010. Epub 2015 Apr 20. Link to article on publisher's site
The Journal of emergency medicine
Lam, Samuel H. F; Bailitz, John; Blehar, David J.; Becker, Brent A.; Hoffmann, Beatrice; Liteplo, Andrew S.; Rajan, Kumar B.; and Lambert, Michael, "Multi-Institution Validation of an Emergency Ultrasound Image Rating Scale-A Pilot Study" (2015). Emergency Medicine Publications and Presentations. 140.