Comparative analysis of two methods of scoring word problems in mathematics: Implications on test validity and reliability

This study compares two methods of scoring word problems in mathematics (\u27without criteria\u27 and \u27with criteria\u27) in terms of reliability and validity of a test consisting of 12 word problems appropriate for high school sophomores enrolled in the external units of the Mindanao State University. This test was administered to 96 third year high school students belonging to the better sections of the four schools randomly chosen from the Mindanao State University - external units, including MSU-Science High School. Of these test papers, twenty (grouped into two of ten papers each) were randomly selected and six photocopies were made of each test papers for four teachers to rate. Given that each item is worth a maximum of ten points, phase 1 of this study required the raters to mark them using \u27without criteria\u27 which corresponds to their usual way of marking. After a week, phase 2 directed the same raters to mark each problem of the remaining photocopies using \u27with criteria\u27 or based on the criteria: representation (3 points), equation (4 points), computation (2 points), and answer (1 point).Reliability includes three coefficients: 1) inter-rater reliability coefficient 2) stability coefficient and 3) internal consistency coefficient. The relationship between the students\u27 second year mathematics grades and the scores obtained by each rater from \u27without criteria\u27 or \u27with criteria\u27 method of scoring explains the validity of the test. With the aid of Pearson product-moment method, split-half technique (corrected by Spearman-Brown prophecy formula), T- (and Z-) tests for both the dependent and independent samples, and analysis of variance, this study arrived at the following conclusions: \u27with criteria\u27 method of scoring word problems in mathematics produced better objectivity and higher reliability, preserved the homogeneity of the students involved in the study, slightly increased internal consistency of word problems in mathematics test, slightly increased the validity of the test, and partial knowledge of the students is given greater credit when \u27with criteria\u27 method was used