Automated scoring of practical tests in an introductory course in information technology

In an introductory course in information technology at the University of Otago the acquisition of practical skills is considered to be a prime objective. An effective way of assessing the achievement of this objective is by means of a `practical test', in which students are required to accomplish simple tasks in a controlled environment. The assessment of such work demands a high level of expertise, is very labour intensive and can suffer from marker inconsistency, particularly with large candidatures. This paper describes the results of a trial in which the efforts of one thousand students in a practical test of word processing were scored by means of a program written in MediaTalk. Details of the procedure are given, including sampling strategies for the purpose of validation and examples of problems that were encountered. It was concluded that the approach was useful, and once properly validated gave rise to considerable savings in the time and effort.Unpublished[1] P.F. Summons, J. Coldwell, F. Henskens, and C.Bruff, Automated Evaluation of Spreadsheet Concepts - Excel in Action, Proceedings of 2nd Annual NSW Symposium on Information Technology and Information Systems SITIS’97, University of New South Wales, Sydney,Feb, 1997 [2] Jack J. Baroudi and Wanda J. Orlikowski, The Problem of Statistical Power in MIS Research, MIS Quarterly, March, 1989, 87-106 [3] Ranjan K. Som, Practical Sampling Techniques, (New York:Marcel Dekker, Inc. 1996