Scoring Subjectivity and Item Performance on Measures Used to Assess Violence Risk: The PCL-R and HCR-20 as Exemplars
Department of Psychiatry
Violence; Risk Assessment; Reproducibility of Results
Health Services Research | Mental and Social Health | Psychiatric and Mental Health | Psychiatry | Psychiatry and Psychology
Although reliability is essential to validity, most research on violence risk assessment tools has paid little attention to strategies for improving rater agreement. The authors evaluated the degree to which perceived subjectivity in scoring guidelines for items from two measures-the Psychopathy Checklist-Revised (PCL-R) and the Historical, Clinical, Risk Management-20 (HCR-20)-were related to indices of interrater agreement reported in the research literature. Nine doctoral students with experience scoring the PCL-R and HCR-20 showed a high level of agreement with respect to the amount of subjectivity required to score items. Decreased subjectivity was associated with increased rater agreement for both tools. For the PCL-R, items on aggregate measures that have the strongest effects in predictive validity research (i.e., Factor 2, Facet 4) were rated as the least subjective. For the HCR-20, items on the Historical scale were rated as the least subjective. Implications for measure development and refinement are discussed.
DOI of Published Version
Assessment. 2010 Aug 4. Link to article on publisher's site
Rufino KA, Guy LS, Boccaccini MT. (2010). Scoring Subjectivity and Item Performance on Measures Used to Assess Violence Risk: The PCL-R and HCR-20 as Exemplars. Implementation Science and Practice Advances Research Center Publications. https://doi.org/10.1177/1073191110378482. Retrieved from https://escholarship.umassmed.edu/psych_cmhsr/320