Evaluating Count Outcomes in Synthesized Single-Case Designs with Multilevel Modeling: A Simulation Study

Complex statistical techniques such as multilevel modeling (MLM) ideally require substantial sample sizes in order to avoid assumption violations. Unfortunately, large between-subjects sample sizes can be impractical and, in some cases, impossible in real-world applications. The use of single-case designs (SCD) allow researchers to overcome this issue. The ability to handle non-normal outcomes appropriately in such single-case designs, however, remains unclear, especially when the outcome reflects recurrent event (count) data. The purpose of this study is to evaluate the utility of MLM for evaluating recurrent event outcomes in synthesized single-case designs. More specifically, this study seeks to determine the effects of analysis and analytic design decisions when distributional assumptions also vary as a result of the count outcomes. The ability to properly model non-normal distributions in school-based or clinical research settings is critical for two reasons: (1) count data are one of the most prevalent outcomes used in common single-case designs, and (2) it is necessary to avoid biased point estimates and standard errors. Monte Carlosimulation was used to examine relative bias, mean square error, and confidence interval coverage rates across four simulation conditions: distributional assumption, degree of freedom methods, sample size, and time-series lengths, where the synthesis of two empirical data sets were utilized to represent the population parameters. As hypothesized, the Negative Binomial distribution performed better, in comparison to the normal distribution and Poisson distribution on relative bias, mean square error, and coverage. The Kenward-Roger and Satterthwaite degree of freedom methods resulted in coverage rates that were closer to the nominal .95 level than the between-within, residual, and containment methods. The results from the sample sizes and time-series lengths were less straightforward than the other conditions. The results from this study should be used to provide guidance for methodological decisions of synthesized single-case design research. However, researchers should consider their own purpose and research context prior to making methodological decisions, as a single analysis is insufficient for all applied situations. Adviser: James A. Bovair