A Causal Comparative Analysis of a Computer Adaptive Mathematics Program Using Multilevel Propensity Score Matching

The purpose of this study was to examine the effectiveness of a computer adaptive mathematics program, i-Ready® Mathematics Instruction in Central Florida School District (CFSD) using a variety of quantitative analyses. In particular, students in the fifth grade at CFSD were examined in relation to the gains made on their end-of-year statewide mathematics assessment, known as the Florida Standards Assessment (FSA Math). Per i-Ready®, students who use i-Ready® Mathematics Instruction for a minimum of 45 minutes per subject per week for at least 25 weeks show significantly greater gains than those who do not (Curriculum Associates, 2017b). Therefore, two propensity score matching methods (one considering the multilevel structure of the data) were examined and compared to split students into the following two comparison groups: those who used the program with fidelity and those who did not. Next, a series of multilevel models, using both propensity score matching methods, and dependent t-tests were examined to evaluate the effectiveness of i-Ready® Mathematics Instruction. In conclusion, the use of the multilevel propensity score matching technique yielded a better fit for explaining the variation in relation to gains students in the fifth grade made on the FSA Math assessment. Additionally, i-Ready® Mathematics Instruction did positively impact fifth grade students in Central Florida School District who used the program with fidelity with relation to their FSA Math score gains, compared to those who did not use the program with fidelity