The feasibility of a multi-format Web-based assessment of physicians' communication skills
Meyers Primary Care Institute
Clinical Competence; *Communication; Educational Measurement; Educational Status; Feasibility Studies; Feedback; Female; Health Care Surveys; Humans; *Internet; Male; Online Systems; Physician's Practice Patterns; *Physician-Patient Relations; Physicians; Questionnaires; Reproducibility of Results; Statistics as Topic; United States; Videotape Recording
Health Services Research | Primary Care
OBJECTIVE: Little is known about the best approaches and format for measuring physicians' communication skills in an online environment. This study examines the reliability and validity of scores from two Web-based communication skill assessment formats.
METHODS: We created two online communication skill assessment formats: (a) MCQ (multiple-choice questions) consisting of video-based multiple-choice questions; (b) multi-format including video-based multiple-choice questions with rationales, Likert-type scales, and free text responses of what physicians would say to a patient. We randomized 100 general internists to each test format. Peer and patient ratings collected via the American Board of Internal Medicine (ABIM) served as validity sources.
RESULTS: Seventy-seven internists completed the tests (MCQ: 38; multi-format: 39). The adjusted reliability was 0.74 for both formats. Excellent communicators, as based on their peer and patient ratings, performed slightly better on both tests than adequate communicators, though this difference was not statistically significant. Physicians in both groups rated test format innovative (4.2 out of 5.0).
CONCLUSION: The acceptable reliability and participants' overall positive experiences point to the value of ongoing research into rigorous Web-based communication skills assessment.
PRACTICE IMPLICATIONS: With efficient and reliable scoring, the Web offers an important way to measure and potentially enhance physicians' communication skills.
DOI of Published Version
Patient Educ Couns. 2011 Sep;84(3):359-67. Epub 2011 May 6. Link to article on publisher's site
Patient education and counseling
Kim, Sara; Brock, Douglas M.; Hess, Brian J.; Holmboe, Eric S.; Gallagher, Thomas H.; Lipner, Rebecca S.; and Mazor, Kathleen M., "The feasibility of a multi-format Web-based assessment of physicians' communication skills" (2011). Meyers Primary Care Institute Publications and Presentations. 510.