Using probabilistic corrections to account for abstractor agreement in medical record reviews.

UMMS Affiliation

Meyers Primary Care Institute

Publication Date


Document Type



Abstracting and Indexing as Topic; Aged; Aged, 80 and over; Breast Neoplasms; Comorbidity; Data Interpretation, Statistical; Female; Humans; Mastectomy; Medical Records; Neoplasm Recurrence, Local; Probability; United States


Health Services Research | Medicine and Health Sciences


The quality of medical record abstracts is often characterized in a reliability substudy. These results usually indicate agreement, but not the extent to which lack of agreement affects associations observed in the complete data. In this study, medical records were reviewed and abstracted for patients diagnosed with stage I or stage II breast cancer between 1990 and 1994 at one of six US Cancer Research Network sites. For a subsample, interrater reliability data were available. The authors calculated conventional hazard ratios and 95% confidence intervals for the association of demographic, tumor, and treatment characteristics with recurrence rate. These conventional estimates of effect were compared with three sets of estimates and 95% simulation intervals that took account of the uncertainty assessed by lack of agreement in the reliability substudy. The rate of recurrence was associated with increasing cancer stage and with treatment modality but not with demographic characteristics. The hazard ratios and simulation intervals that took account of the reliability data showed that the simulation interval grew wider as the sources of uncertainty taken into account grew more complete, but the associations expected a priori remained readily apparent. While many investigators use reliability data only as a metric for data quality, a more thorough approach can also quantitatively depict the uncertainty in the observed associations.


Am J Epidemiol. 2007 Jun 15;165(12):1454-61. Epub 2007 Apr 3.

Journal/Book/Conference Title

American journal of epidemiology

Related Resources

Link to article in PubMed

PubMed ID