The performance of Multilevel Structural Equation Modeling (MSEM) in comparison to Multilevel Modeling (MLM) in multilevel mediation analysis with non-normal data

The mediation analysis has been used to test if the effect of one variable on another variable is mediated by the third variable. The mediation analysis answers a question of how a predictor influences an outcome variable. Such information helps to gain understanding of mechanism underlying the variation of the outcome. When the mediation analysis is conducted on hierarchical data, the structure of data needs to be taken into account. Krull and MacKinnon (1999) recommended using Multilevel Modeling (MLM) with nested data and showed that the MLM approach has more power and flexibility over the standard Ordinary Least Squares (OLS) approach in multilevel data. However the MLM mediation model still has some limitations such as incapability of analyzing outcome variables measured at the upper level. Preacher, Zyphur, and Zhang (2010) proposed that the Multilevel Structural Equation Modeling (MSEM) will overcome the limitation of MLM approach in multilevel mediation analysis. The purpose of this study was to examine the performance of the MSEM approach on non-normal hierarchical data. This study also aimed to compare the MSEM method with the MLM method proposed by MacKinnon (2008) and Zhang, Zyphur, and Preacher (2009). The study focused on the null hypothesis testing which were presented by Type I error, statistical power, and convergence rate. Using Monte Carlo method, this study systematically investigates the effect of several factors on the performance of the MSEM and MLM methods. Designed factors considered were: the magnitude of the population indirect effect, the population distribution shape, sample size at level 1 and level 2, and the intra-class correlation (ICC) level. The results of this study showed no significant effect of the degree of non-normality on any performance criteria of either MSEM or MLM models. While the Type I error rates of the MLM model reached the expected alpha level as the group number was 300 or higher, the MSEM model showed very conservative performance in term of controlling for the Type I error with the rejection rates of null conditions were zero or closed to zero across all conditions. It was evident that the MLM model outperformed the MSEM model in term of power for most simulated conditions. Among the simulation factors examined in this dissertation, the mediation effect size emerged as the most important one since it is highly associated with each of the considered performance criteria. This study also supported the finding of previous studies (Preacher, Zhang, & Zyphur, 2011; Zhang, 2005) about the relationship between sample size, especially the number of group, and the performance of either the MLM or MSEM models. The accuracy and precision of the MLM and MSEM methods were also investigated partially in this study in term of relative bias and confidence interval (CI) width. The MSEM model outperformed the MLM model in term of relative bias while the MLM model had better CI width than the MSEM model. Sample size, effect size, and ICC value were the factors that significantly associate with the performance of these methods in term of relative bias and CI width