Readability of informed-consent forms used in mental health research
Department of Psychiatry
Appelbaum, Paul S.; Foti, Mary Ellen
Consent Forms; Reading; Mentally Ill Persons; Educational Status; Research Subjects; Informed Consent
Life Sciences | Medicine and Health Sciences
Background: The validity of the informed consent process rests in part on whether potential study participants are able to understand what the purpose of a study is as well as its risks and benefits. This information is generally disclosed in a written informed-consent form. However, in a nationally representative survey, seventy percent of adults with a self-reported mental health problem were found to be either functionally illiterate or marginally literate. Furthermore, those mentally ill adults who are literate tend to read an average of 3-5 grade levels below that expected by their level of education. It is possible that those consenting to participate in mental health research are not able to read the informed-consent form they are asked to sign.
Objectives: We sought to determine whether the informed-consent forms used in research approved by the Massachusetts Department of Mental Health were written at a grade level higher than that achieved by subjects eligible to participate in the studies. To investigate this possibility we compared the readability of informed-consent forms used in MA-DMH research with the maximum attained grade level of clients receiving MA-DMH services. We performed a sub-analysis of the readability of informed-consent forms based on the risk level for each study (I, II, and III).
Methods: We obtained the maximum completed U.S. grade level for all adults receiving MA-DMH services from their computerized database. The informed-consent forms for one hundred and fifty-four studies were separated into three groups according to risk levels I, II, and III. The readability of each form was calculated using four different readability formulas widely used in medical research: the Flesch Reading Ease Score (FRES), the Flesch-Kincaid Grade Level (FKGL), the FOG Index, and the Fry Graph. The FRES provides a score from 0-100, with a greater score indicating easier reading; the other three formulas provide scores that correspond to the U.S. grade level needed to read the text.
We performed a descriptive analysis of the range, mean, and standard deviation for the readability scores of the four formulas used. We then performed a Kruskal-Wallis test to determine if the readability scores of each formula varied by risk level. For each formula, the Kruskal-Wallis test first assigned a rank to the readability score of each informed-consent form. It then provided the mean rank of scores for each risk level. A chi-squared analysis measured the degree of difference between risk level ranks and the mean of all the ranks.
Results: Of the 34,095 adult clients receiving MA-DMH services, the maximum attained educational level was known for 33,537 (98%) individuals. 13,323 clients (39.7%) completed between the 1st and 11th grades. 12,701 (37.9%) completed the 12th grade but had no further education and 7,513 (22.4%) had some education beyond the 12th grade.
The mean FRES for 154 tested forms was 47.62 (33 to 74), with a corresponding FKGL of 12 (6.4 to 15.4). The mean FOG Index readability for 154 forms was 14.5 (8.4 to 18.4). Ten informed-consent forms fell beyond the 16th grade level on the Fry graph; the mean for the remaining 144 informed-consent forms was 13.93 (7 to 16).
Of the 154 informed-consent forms, sixty-eight studies were from risk level I studies, seventy-two from risk level II studies, and twelve from risk level III studies. For each formula tested, the mean readability scores increased with increasing risk level, indicating that those consent forms used in higher risk studies were, on average, harder to read.
Conclusion: Even by the most conservative estimate (i.e., FKGL, which had the lowest mean readability score: 12.055), approximately forty percent of those eligible to participate in MA-DMH research studies lack the ability to read the average informed-consent form. The magnitude of this finding is underscored by the fact that persons with mental illness tend to read 3-5 grade levels below their maximum level of education. Furthermore, those studies which pose a greater level of risk to the study participant are, on average, harder to read.
DOI of Published Version
This research was eventually published as follows: Christopher, Paul P., Foti, Mary Ellen, Roy-Bujnowski, Kristen, Appelbaum, Paul S. Consent Form Readability and Educational Levels of Potential Participants in Mental Health Research. Psychiatr Serv 2007 58: 227-232. Link to article on publisher's website
Christopher, Paul P., "Readability of informed-consent forms used in mental health research" (2005). University of Massachusetts Medical School. Senior Scholars Program. Paper 24.