The construction and validation of rating scales for oral tests in English as a Foreign Language. Unpublished

The present research investigates the principles upon which rating scales in oral testing are constructed and used, and the subsequent claims of reliability and validity made for them. The research addresses two main questions: (i) can rating scales be constructed on the basis of an analysis of a database of actual student speech; and (ii) are scales produced on the basis of student speech superior to those produced using a-priori methods? A corpus of spoken data was collected and analyzed. Discriminant analysis was used in order to isolate factors which could discriminate between students of different ability levels, and a Fluency and an Accuracy rating scale constructed. The Fluency rating scale was seen to be the most stable in the construction phase of the study. Forty seven students took three tasks. Video recordings were rated by five raters on the two rating scales and the English Language Testing Service rating scale