A NEW ERA IN U.S. NAVY TESTING: MULTIMEDIA NAVY ENLISTED EXAM

In September 2004, the U.S. Navy entered a new era of testing when Sailors in the Aerographer's Mate rating used a computer to take their enlisted advancement exam. The shift from a paper and pencil format to an interactive, multimedia exam provided the means to move from traditional assessment of subject matter knowledge to a more performance-based assessment. This paper reviews the processes from the exam’s inception to its implementation. These processes include the selection of authoring tools, evaluation of delivery options, design and beta testing of prototypes, establishing criteria for selecting the first rating for multimedia exams, hardware solutions and procurement decisions, multimedia design and development, and formative evaluation. Because Sailors Navy-wide take their advancement exams on the same day, unique administrative procedures, shipping, and security requirements for equipment, exams, and results had to be addressed to ensure exam timeliness and fairness to candidates. Test day evaluation data and candidate feedback data demonstrate the effectiveness and feasibility of this new approach. A discussion of lessons learned during the evolution to computerized assessment includes not only the successes but also the problems encountered and their solutions