Computer-based simulations in learning environments: A meta-analysis of outcomes

This study is a meta-analysis of the true- and quasi-experimental research concerning student achievement and attitudes in instructional environments utilizing computer-based simulations. Weighted effect size analysis procedures were used in which independence of the sampling groups was maintained. Three general research areas were investigated: (1) the effectiveness of computer-based simulation instruction as compared to traditional instruction; (2) the impact of instructional, implementation, and student characteristics on outcomes; and (3) areas in which further research is warranted. Specific search criteria and procedures led to the identification of 54 studies in which computer-based simulations were used in an instructional environment. Of these studies, 43 supplied sufficient information for the computation of effect sizes. A mean effect size of 0.29 (standard error = 0.04) was found for cognitive achievement outcomes when computer-based simulation instruction was compared to traditional instruction. No difference was found among effect sizes of low-level thinking, high-level thinking, or retention outcomes. A paucity of research assessing attitudinal outcomes prohibited any generalizations concerning student attitude toward instruction or student attitude toward the subject matter. The instructional, implementation, and student characteristic features were found to be highly colinear. Furthermore, the homogeneity of the effect sizes indicated no further analysis of the impact of features on the outcomes was warranted. The differential effects of the features, therefore, were not explored because the effect sizes were similar. Future research should be designed to investigate cooperative learning, dynamic simulations, locus of instructional control, and prior knowledge in ways in which these features are not highly related to subject matter. Most studies failed to provide information concerning the reliability and validity of the measures used to assess outcomes. Studies in which threats to internal and external validity were detected tended to be outliers. Furthermore, the nature of the threat to validity was directly related to the direction in which the effect size differed from the homogeneous group