Simulation Analysis and Meta-Analysis of Single-Case Experimental Designs

Single-case experimental designs remain outside of mainstream methodology despite their substantial contributions to our understanding of human behavior. An obstacle to wider adoption is the lack of consensus regarding the analysis and meta-analysis of single-case data. Many single-case statistical methods have been proposed; nearly all are limited by the incompleteness of their models or their lack of formal statistical development, both limitations that inhibit research synthesis and knowledge building. This dissertation, presented in three manuscripts, introduces a simulation-based method of analysis and meta-analysis for single-case experimental designs. Interrupted Times-Series Simulation (ITSSIM) estimates treatment effect sizes by modeling level, trend, variance, and autocorrelation parameters. Parameter estimates are naturally imprecise in brief time-series. ITSSIM compensates for this imprecision by using an iterative procedure to model many plausible parameter values given the observed data. ITSSIM calculates an effect size by comparing a distribution of plausible “null effects”—the no-treatment predictions based on baseline data—to a distribution of plausible treatment effects. ITSSIM effect size estimates, reported as correlation coefficients, standardized mean differences, or unstandardized effects, are interpretable for both clinical practice and quantitative research synthesis. Three studies provide evidence for the content validity, construct validity, and criterion validity of ITSSIM effect size estimates using theoretical, comparative, and deductive strategies, respectively. The first study establishes the theoretical rationale for single-case simulation methods generally, and ITSSIM specifically. ITSSIM produced effect size estimates comparable to five sophisticated multilevel methods when a study of disruptive classroom behavior was reanalyzed. In the second study, ITSSIM produced mean effect size estimates consistent with similar meta-analyses of group-design research. The third study field tests a new software tool for simulation research of single-case statistics. ITSSIM performed reliably under a variety of simulation conditions, controlling for baseline trend and autocorrelation. The results from these three studies indicate that ITSSIM is a powerful, comprehensive method for analysis and meta-analysis of single-case experimental designs. ITSSIM effect size estimates are consistent with other, previously published single-case statistics, and it yields reasonable results even under extreme simulation conditions. ITSSIM is recommended to single-case investigators who wish to better understand their single-case data and treatment effects