Department of Quantitative Health Sciences
Medical Subject Headings
Adult; Analysis of Variance; Confidence Intervals; Female; Health Status; Humans; Male; Models, Statistical; Outcome Assessment (Health Care); Quality of Life; Renal Insufficiency, Chronic; Self Report
Biostatistics | Epidemiology | Health Services Research
BACKGROUND: Relative validity (RV), a ratio of ANOVA F-statistics, is often used to compare the validity of patient-reported outcome (PRO) measures. We used the bootstrap to establish the statistical significance of the RV and to identify key factors affecting its significance.
METHODS: Based on responses from 453 chronic kidney disease (CKD) patients to 16 CKD-specific and generic PRO measures, RVs were computed to determine how well each measure discriminated across clinically-defined groups of patients compared to the most discriminating (reference) measure. Statistical significance of RV was quantified by the 95% bootstrap confidence interval. Simulations examined the effects of sample size, denominator F-statistic, correlation between comparator and reference measures, and number of bootstrap replicates.
RESULTS: The statistical significance of the RV increased as the magnitude of denominator F-statistic increased or as the correlation between comparator and reference measures increased. A denominator F-statistic of 57 conveyed sufficient power (80%) to detect an RV of 0.6 for two measures correlated at r = 0.7. Larger denominator F-statistics or higher correlations provided greater power. Larger sample size with a fixed denominator F-statistic or more bootstrap replicates (beyond 500) had minimal impact.
CONCLUSIONS: The bootstrap is valuable for establishing the statistical significance of RV estimates. A reasonably large denominator F-statistic (F > 57) is required for adequate power when using the RV to compare the validity of measures with small or moderate correlations (r < 0.7). Substantially greater power can be achieved when comparing measures of a very high correlation (r > 0.9).
Deng, Nina; Allison, Jeroan J.; Fang, Hua (Julia); Ash, Arlene S.; and Ware, John E. Jr., "Using the bootstrap to establish statistical significance for relative validity comparisons among patient-reported outcome measures" (2013). Quantitative Health Sciences Publications and Presentations. 1111.