Design, analysis, power, and sample size calculation for three-phase interrupted time series analysis in evaluation of health policy interventions
Department of Population and Quantitative Health Sciences; Department of Medicine; Meyers Primary Care Institute
Biostatistics | Health Policy | Health Services Administration | Health Services Research | Investigative Techniques | Statistics and Probability
OBJECTIVE: To discuss the study design and data analysis for three-phase interrupted time series (ITS) studies to evaluate the impact of health policy, systems, or environmental interventions. Simulation methods are used to conduct power and sample size calculation for these studies.
METHODS: We consider the design and analysis of three-phase ITS studies using a study funded by National Institutes of Health as an exemplar. The design and analysis of both one-arm and two-arm three-phase ITS studies are introduced.
RESULTS: A simulation-based approach, with ready-to-use computer programs, was developed to determine the power for two types of three-phase ITS studies. Simulations were conducted to estimate the power of segmented autoregressive (AR) error models when autocorrelation ranged from -0.9 to 0.9 with various effect sizes. The power increased as the sample size or the effect size increased. The power to detect the same effect sizes varied largely, depending on testing level change, trend changes, or both.
CONCLUSION: This article provides a convenient tool for investigators to generate sample sizes to ensure sufficient statistical power when three-phase ITS study design is implemented.
interrupted time series, policy evaluation, power, quasi-experimental design, sample size calculation, segmented regression
DOI of Published Version
J Eval Clin Pract. 2019 Aug 19. doi: 10.1111/jep.13266. [Epub ahead of print] Link to article on publisher's site
Journal of evaluation in clinical practice
Zhang B, Liu W, Lemon SC, Barton BA, Fischer MA, Lawrence C, Rahn EJ, Danila MI, Saag KG, Harris PA, Allison JJ. (2019). Design, analysis, power, and sample size calculation for three-phase interrupted time series analysis in evaluation of health policy interventions. Population and Quantitative Health Sciences Publications. https://doi.org/10.1111/jep.13266. Retrieved from https://escholarship.umassmed.edu/qhs_pp/1292