Level up: A systematic review of the nexus between game-based learning and developmental math education

Developmental math currently poses a hurdle that many community college students are unable to overcome. Eighty percent of developmental math students do not successfully complete any college-level course within three years (Bailey, Jeong, & Cho, as cited by Strother, Van Campen & Grunow, 2013, p. 1). Despite numerous reform attempts, there has been a lack of consistent improvement and a lack of understanding regarding which methods are the most successful and the reasons why. Faculty need a new pedagogy to connect with developmental math students and effectively engage them in order to improve student learning outcomes. Proponents regard game-based learning as a way to leverage the intense engagement of video games in an educational context to increase engagement, increase the amount of time on task, and improve overall student success metrics. The author has reviewed the studies and the evidence; weighed the value and legitimacy of the studies; synthesized the weight of the evidence of introducing a game-based learning pedagogy, and then examined the potential efficacy of game-based learning in the developmental math environment. The author has created an evidence-based implementation plan to incorporate game-based learning into developmental math education in a way that has the highest potential to improve student learning.ABSTRACT Title of Dissertation LEVEL UP: A SYSTEMATIC REVIEW OF THE NEXUS BETWEEN GAME-BASED LEARNING AND DEVELOPMENTAL MATH EDUCATION Monique Wilson, Doctor of Management, 2013 Dissertation Directed By Dr. Pat Keir, Graduate School, University of Maryland University College Dr. Ronald Head, Graduate School, University of Maryland University College Developmental math currently poses a hurdle that many community college students are unable to overcome. Eighty percent of developmental math students do not successfully complete any college-level course within three years (Bailey, Jeong, & Cho, as cited by Strother, Van Campen & Grunow, 2013, p. 1). Despite numerous reform attempts, there has been a lack of consistent improvement and a lack of understanding regarding which methods are the most successful and the reasons why. Faculty need a new pedagogy to connect with developmental math students and effectively engage them in order to improve student learning outcomes. Proponents regard game-based learning as a way to leverage the intense engagement of video games in an educational context to increase engagement, increase the amount of time on task, and improve overall student success metrics. The author has reviewed the studies and the evidence; weighed the value and legitimacy of the studies; synthesized the weight of the evidence of introducing a game-based learning pedagogy, and then examined the potential efficacy of game-based learning in the developmental math environment. The author has created an evidence-based implementation plan to incorporate game-based learning into developmental math education in a way that has the highest potential to improve student learning. Key words: Game-based learning, gaming, serious games, developmental math, developmental education reform, innovation LEVEL UP: A SYSTEMATIC REVIEW OF THE NEXUS BETWEEN GAME-BASED LEARNING AND DEVELOPMENTAL MATH EDUCATION By Monique Wilson Dissertation submitted to the Faculty of the Graduate School of the University of Maryland University College, in partial fulfillment of the requirements for the degree of Doctor of Management 2013 Advisory Committee Dr. Pat Keir, Chair Dr. Ron Head © Copyright by Monique Wilson 2013 LEVEL UP ii DEDICATION To the late James and Wynetta Caldwell, who never wavered in their support of me and gave me tangible examples of faith, love, and empowerment. To the late Mary C. Scales, who instilled in me a love for math and reminded me that “the blessed of us have got to help the rest of us.” LEVEL UP iii ACKNOWLEDGMENTS This dissertation would not have been completed without the help and support of family, friends, co-workers, cohort members, and members of the University of Maryland University College faculty. To my parents, Dr. Walter and Dr. Aileen Bumphus, and Bobbie Wiggins, thank you for your support, encouragement, guidance, and most of all, love. To my daughters, Alexia and Olivia – thank you for the years of sacrifice, your support, and your encouragement. To the Duke and Nelson families, thank you for the continual encouragement and the constant supply of love. To my siblings, thank you for all the support and love. To my best friend, Dr. Sybril Bennett, thank you for always believing in me. Your support encouraged me to start this journey and you were there with me every step of the way. To my friends and fellow educators, Alysia Puckett and Carolyn Jenkins, thanks for providing a sounding board for theories and thanks for your constant support and encouragement. To Michaela Ware, Kenya Hollingsworth, and Rhonda Ware thanks for being my cheerleaders. To my friend, Leonard Green, thank you for providing a critical eye to my work. To my Pastor, Vincent Windrow, thank you for such relevant and timely sermons and encouragement! To the Board Members of the Dr. Martin Luther King, Jr. Scholarship Fund of Rutherford County, thank you for your continual support and encouragement. Each one of you has been invaluable at different points of this journey. This journey to the finish line is one that I have shared with my University of Maryland University College cohort. Thanks to all of them, especially Dwayne Hart, Anne Killen, Holly Hart, and Sheila Quark-Bailey. Each of you has helped to sharpen my intellectual sword, and it has indeed been a pleasure to learn from each one of you. LEVEL UP iv There are several university faculty members who have taught and coached me since I started in the Doctor of Management Community College Policy and Administration program in 2010. I thank University of Maryland University College faculty members Dr. Pat Keir, Dr. Ron Head, Dr. Charlene Nunley, and Dr. Mary Ellen Duncan for providing guidance and leadership during the program. I want to acknowledge Dr. Jeff Clark, a faculty member at Middle Tennessee State University who was the first faculty member to encourage me to pursue a career in higher education. LEVEL UP v Table of Contents List of Figures .......................................................................................................................... viii List of Tables ............................................................................................................................. ix Chapter One: Introduction .......................................................................................................... 1 Topic ...................................................................................................................................... 1 Background Information ......................................................................................................... 1 Problem Statement .................................................................................................................. 5 Significance of the Problem .................................................................................................... 8 Theoretical Base ................................................................................................................... 13 Change theory ................................................................................................................... 13 Adoption and scaling of technical innovations in higher education theory ...................... 14 Flow theory ....................................................................................................................... 14 Definitions ............................................................................................................................ 15 Research Questions ............................................................................................................... 16 Purpose of the Research ........................................................................................................ 17 Chapter Two: Literature Review .............................................................................................. 18 Overview ............................................................................................................................... 18 Understanding current math remedial education efficacy ................................................. 18 Effective math remedial education reform efforts ............................................................ 20 Game-Based Learning: Understanding the Potential Impact ................................................ 23 The Efficacy of Existing Game-Based Learning Efforts ...................................................... 30 Engagement/motivation .................................................................................................... 36 Increased time on task ....................................................................................................... 38 Improved student achievement ......................................................................................... 39 Longer retention period ..................................................................................................... 41 Problem solving ................................................................................................................ 41 Greater levels of decision quality ...................................................................................... 43 Neutral impact – subject matter educational objectives .................................................... 43 Management Theories Related to Game-based Learning for Developmental Math ............ 44 Change theory ................................................................................................................... 44 Chapter Three: Conceptual Model ........................................................................................... 47 Introduction ........................................................................................................................... 47 Current State ......................................................................................................................... 49 Game-Based Learning .......................................................................................................... 50 Relevant Factors for Effective Game-based Learning .......................................................... 51 Assumptions .......................................................................................................................... 52 Future State ........................................................................................................................... 52 LEVEL UP vi Chapter Four: Research Methodology ...................................................................................... 54 Introduction ........................................................................................................................... 54 Introduction of the Research Methodology .......................................................................... 56 Research Design Outline ...................................................................................................... 58 Background research ......................................................................................................... 58 Keywords and search criteria ............................................................................................ 58 Actual Research and Retrieval .............................................................................................. 61 Evaluation of the Literature .................................................................................................. 66 Classification of evidence ................................................................................................. 66 Critical review of studies .................................................................................................. 67 Use of Subject Matter Experts .............................................................................................. 80 Expert research/identification ........................................................................................... 80 Invitation to experts ........................................................................................................... 80 Distribution of evaluation material ................................................................................... 81 Collection of feedback ...................................................................................................... 81 Analysis of expert feedback .............................................................................................. 82 Incorporation of revisions into the study .......................................................................... 82 Conclusion ............................................................................................................................ 83 Chapter Five: Analysis and Discussions .................................................................................. 87 Introduction ........................................................................................................................... 87 Analysis of Subject-Matter Experts Review ......................................................................... 88 Summary, analysis, and application of expert feedback ................................................... 93 Analysis of the Literature Review Findings ......................................................................... 95 Studies and summaries of studies ..................................................................................... 97 Expert opinion articles, webinars, conference presentations, or books ............................ 99 Best practices for implementing a new innovation ......................................................... 101 Findings and Conclusions ................................................................................................... 102 Does game-based learning improve the overall efficacy of math developmental education? ....................................................................................................................... 103 Does math game-based learning have more favorable learning outcomes than traditional math learning? ................................................................................................................. 103 Are there correlations between student demographics and student learning outcomes associated with math game-based learning? ................................................................... 104 What are the best practices for game-based learning for developmental math education? .............................................................................................................. 104 What elements need to be included for a math game to be considered best of class? .... 105 How can colleges bring to scale the use of game-based learning for developmental math? ...................................................................................................... 107 Limitations on Interpretation of Study ................................................................................ 108 LEVEL UP vii Summary ............................................................................................................................. 109 Chapter Six: Management Implications ................................................................................. 111 Introduction ......................................................................................................................... 111 Implementation Plan for Game-based Learning ................................................................. 113 Finding games ................................................................................................................. 114 Analysis of COTS games ................................................................................................ 118 Selection of a COTS game .............................................................................................. 122 Piloting: Developing the curriculum for the course ........................................................ 123 Building support .............................................................................................................. 124 Implementing on a wider scale ....................................................................................... 127 Ongoing evaluation ......................................................................................................... 128 Implications for Management Practice for Community College Leaders .......................... 129 Issues for Consideration ...................................................................................................... 130 Future Research Agendas ................................................................................................... 131 Conclusions ......................................................................................................................... 132 References .............................................................................................................................. 134 Appendices ............................................................................................................................. 143 LEVEL UP viii List of Figures Figure 1. Nine Learning Styles, adapted from Nolting (2013) ..................................................... 6 Figure 2. OECD - US College Completion Rates ........................................................................ 9 Figure 3. Conceptual Model ....................................................................................................... 48 Figure 4. Methodology – Evidence-based research (EBR) approach ........................................ 85 Figure 5. Elements of Game-based Learning ........................................................................... 105 Figure 6. Steps for Implementation .......................................................................................... 114 Figure 7. Barriers to Adoption versus Mutual Adoption .......................................................... 125 LEVEL UP ix List of Tables Table 1. Comparison of Traditional Training, Hands-on, and Game-based Learning ............... 25 Table 2. Summary of Outcome Studies of Game-Based Learning ........................................... 32 Table 3. Keywords Listing ......................................................................................................... 59 Table 4. Database Keyword Search Results ............................................................................... 63 Table 5. Literature Analysis ....................................................................................................... 68 Table 6. Search Terms for Math Games ................................................................................... 116 Table 7. Math Games ............................................................................................................... 117 Table 8. Game-Based Learning Evaluation Rubric .................................................................. 119 LEVEL UP 1 Chapter One: Introduction “Don’t try to fix the students, fix ourselves first. The good teacher makes the poor student good and the good student superior. When our students fail, we, as teachers, too, have failed.” – Marva Collins (Lederer, 2011, p. 31) Topic Level Up: A Systematic Review of the Nexus Between Game-Based Learning and Developmental Math Education Background Information The majority of entering community college students are unable to demonstrate proficiency at college-level math (Strother, Van Campen, & Grunow, 2013, p. 1). Many innovation efforts have been attempted; unfortunately, there has not been a significant increase in the success rates for developmental math education. Bailey, Jeong, and Cho (as cited by Strother, Van Campen & Grunow, 2013) reported that 80% of developmental math students do not pass the assigned courses within three years (p. 1). Only half of all developmental students will successfully complete their assigned developmental courses, and “between 60 percent to 70 percent of students who take a remedial course never earn a postsecondary degree or credential” (Rutschow & Schneider, 2011, p. 53). Despite the fact that developmental math is a required component for many community college students today, the failure rate is very high and the current pedagogy and processes do not appear to be working. For community colleges to drastically improve the success rate of developmental math education, a radical transformation will need to take place. Rutschow and Schneider (2011) insisted that, “the research demonstrates that minor modifications in developmental education programs are unlikely to produce dramatic improvements. Given that current programs succeed LEVEL UP 2 at moving less than half of their students into college-level courses, more substantive and transformative changes are needed” (p. ix). It is time for community colleges to consider a new pedagogy that has the potential to improve the success rates for developmental math education. Game-based learning has qualities and aspects that have the potential to support its use to enrich developmental math instruction at community colleges. A recent statistic provided by the Entertainment Software