Validation of educational assessments

The thesis focuses on validation as a process whereby the quality of educational assessments can be evaluated. On the basis of Messick’s (1989) validity concept, which takes consequences as well as inferences of assessment results into account, a model for validation is described. An argument made is that assessment purposes, epistemology, and curricular goals are necessary starting points for validation. Fur-thermore, it is argued that several of the criteria for good assessment proposed as alternatives to validity and reliability can be used together with Messick’s frame-work for validity and make it useful in practice. Four empirical studies, each pre-sented in a separate paper, supply examples of the application of this framework. In paper I it is claimed that reliability is a problem inherent in all educational assess-ments, and it is given a meaningful interpretation through the concept of classifica-tion accuracy. The article examines how the classification accuracy of a Swedish national test in mathematics is affected by changes in the test that are motivated by other validity concerns. The study is based on the results of 1,201 students partici-pating in a Swedish national test in mathematics. The results indicate that there can be a significant trade-off between reliability and other aspects of validity. Paper II presents a study of attainment differences in mathematics related to the grouping of students into different study programmes in the Swedish upper-secondary school. For a sample of 403 students, results from two consecutive national tests in mathe-matics were compared, one at the end of compulsory school and the other at the end of the first mathematics course in upper-secondary school. The results indicate that attainment is, on average, positively affected if students are grouped together with higher-achieving peers. This effect appears to be strongest for low-achievers. Paper III addresses questions concerning how perceptions of competence are formed in school mathematics. The study is based on a questionnaire distributed to a sample of Swedish upper secondary school students (n = 550). The results indicate that an inner feeling of understanding is most highly valued by the students as an indicator of competence, more than external cues like test results and teacher feed-back. Male and female students seem to value cues to perceived competence in similar ways. In paper IV an interview study with six mathematics teachers in upper-secondary school is presented, aiming at exploring the variation in what teachers at their particular school say about ability grouping. In summary, what these teachers express concerning the advantages and disadvantages of ability grouping varies a great deal. There are, however, some things they more or less agree on. They have a basically positive view of the way in which ability grouping has been practised at their school. They identify problems with ability grouping for low-ability groups, primarily related to the mixing of students who have trouble learning with those who primarily have motivational problems. The list of advantages and possibilities is definitely longer for groups of high-achieving students compared to groups of low-achievers