Rethinking analyses of crossed effects experiments in marketing communications research

Although Repeated Measures ANOVA is generally used to analyze experimental designs, this technique does not suffice to describe all variance in a crossed effects experiment. Responses are generated from the same subjects and simultaneously those responses will be collected for the same stimuli, exposing the independence of the observations and the generalizability of the results. We address this methodological concern by fitting a mixed-effects model to reanalyze the outcomes of an experiment in marketing communications. In that experiment, a RM ANOVA was used to analyze the impact of a condition and a treatment factor on the recall of products displayed on a computer screen and where the within-subject variance was a random effect (Janssens et al., 2011). Although no major impact on the fixed effects was identified, the interaction between the experimental condition and the treatment remained significant, the mixed-effects model with two random effect terms outweighs a RM ANOVA with only one random effect term for subject. It significantly reduces the overall variance and significantly improves the predictive power of the model. Additionally, the intra-class correlation reveals that the random effect term for the stimuli explains 49.14 % of the variance compared to only 7.93 % for the subjects