On the use of student evaluation of teaching: a longitudinal analysis combining measurement issues and implications of the exercise

Multi item questionnaires are widely used to collect students’ evaluation of teaching at university. This article makes an attempt to analyse students’ evaluation on a broad perspective. Its main aim is to adjust the evaluations from a wide range of factors which jointly may influence the teaching process: academic year peculiarities, course characteristics, students’ characteristics and item dimensionality. By setting the analysis in a generalised mixed models framework a large flexibility is introduced in the measurement of the quality of university teaching in students’ perception. In that way we consider (1) the effects of potential confounding factors which are external to the process under evaluation; (2) the dependency structure across units in the same clusters; (3) the assessment of real improvement in lecturers’ performance over time and (4) the uncertainty related to the use of an overall indicator to assess the global level of quality of the teaching as it has been assessed by the students. The implications related to a misuse of the evaluation results in implementing university policies are discussed comparing point versus interval estimates and adjusted versus unadjusted indicators