Evaluation System Design and Academic Performance Analysis Using Clustering and Simulation

The starting point of this study was to understand the possible causes of evaluation system change in a military academic environment. With that motivation the objectives of this study were defined as examining student profiles in a military academy, establishing the nature of academic performance, comparing student groups that were identified by course scores, analyzing student performance changes over time and developing a manageable evaluation system and curriculum by comparing different scenarios. An analysis was performed on the literature on academic performance prediction, cluster analysis methodologies and their development, and specifically summarized cluster analytic academic performance studies where these two fields intersected. The study started with data collection, database creation and preparation for clustering and simulation studies. A two step clustering methodology was used for grouping courses and cadets. The validated cluster distributions were used as inputs into simulation study. The simulation study started with modeling cadet movements among clusters over stages. The distribution of clusters was found for each course and the scores were transferred into grades using information gathered from historic data. A new evaluation system design procedure was summarized starting with benchmark examples. Then a simulation was used for the evaluation of new system design settings. Assumptions of the simulation study were evaluated. Parameter settings and decision variables were defined and simulation experiments were conducted and results were interpreted by output analysis. The study concludes with a summary of possible outcomes for alternative system designs. During this study it was observed that academic performance is affected by many cognitive noncognitive and demographic variables. Complex human behavior and its interactions with educational environment make it unpredictable. Though, any curriculum or evaluation system development study should focus on these differences and the difficulties due to variation among students