The identification of criteria to be utilised in mathematical diagnostic tests

School-related mistakes and low pass rates have led this researcher to perceive that some students are not adequately prepared for Mathematics 1. To address the problem of under-preparedness overseas universities use placement or diagnostic tests. Diagnostic testing identifies areas of weakness and provides information to guide the development of appropriate remedial support. This researcher embarked on a study to identify the sub-domains (criteria) that should be included in a diagnostic Mathematics test battery at the NMMU. An analysis of first-year curricula was undertaken to determine the required Mathematical pre-knowledge and skills entry-level students should have. Thereafter, the required pre-knowledge and skills were reflected against the standard grade school syllabi. From this it was determined that the school learners should acquire the necessary pre-knowledge and skills for university success as part of the school syllabus. However, in reality this is not the case as the researcher and other Mathematics lecturers identified a number of basic errors that incoming students make. This suggests that they have not developed all the required knowledge and skills. Furthermore, their performance in the matriculation examinations does not provide an adequate measure of the requisite Mathematical pre-knowledge and skills necessary for success at university-level Mathematics. No suitable existing diagnostic Mathematics test could be found. By utilizing both an action research as well as a test development methodology, the researcher thus proceeded to delineate the sub-domains that should be included in a diagnostic Mathematics test battery. Thereafter, test specifications were developed for two pilot tests and items were developed or sourced. The constructed response item-type was chosen for the pilot tests as it was argued that this item-type was more useful to use in a diagnostic test than a multiple-choice item format, for example. The pilot test battery, which consisted of a pilot Arithmetic and Algebra and Calculus tests, was administered to a sample of first-year students at the NMMU in 2004 and their performance in Mathematics at the end of the first year was tracked. Tests were scored holistically and analytically to provide a rich source of information. Thereafter, the test results were analysed to obtain evidence on the content validity of the pilot tests, including the item difficulty values and the item-total correlations; to determine the predictive validity of performance on the pilot tests with respect to final first-year Mathematics marks; and their reliability was determined using the Cronbach’s Alpha statistic. These findings suggest that appropriate sub-domains (criteria) were delineated and the items appropriately covered these sub-domains (i.e. the content validity of the pilot tests is acceptable). Furthermore, the predictive validity of the pilot ix tests was found to be acceptable in that significant correlations were found between the pilot tests and performance in first-year Mathematics. Finally, the pilot tests were found to be reliable. Based on the results, suggestions are made regarding how to refine the diagnostic test battery and the research related to it. The final diagnostic Mathematics test battery holds much potential to be able to assist in the early identification of at-risk students who can be timeously placed in developmentally appropriate Mathematics modules or provided with appropriate remedial interventio