Reliability and Validity of Assessing Student Performance of Psychomotor Skills in Entry Level Physical Therapy Curricula

Physical therapists rely on a large repertoire of psychomotor skills for patient evaluation and treatment. Reliable and accurate assessments of student performance of psychomotor skills are required to provide feedback to students and faculty, to establish student competence, and to enable faculty members to evaluate their instructional methods. In this study a scoring system was designed to objectively assess student performance of five selected ankle joint mobilization procedures. A detailed task analysis was performed on each joint mobilization procedure, which included dividing each procedure into 13 subtasks. Criteria were written for successful completion of each task and subtask. A scoring system was developed, which enabled raters to score each subtask on a scale of 0 to 2 points. From the task analyses a unique score sheet was devised for each joint mobilization. Content validity of this testing tool was established via expert opinion of four orthopaedic physical therapy specialists, who were not involved as raters for the study or in teaching the course. Sixty-two entry level master of physical therapy students, enrolled in a clinical orthopaedics course, served as subjects. Four of the course instructors were trained in the use of the score sheets and served as raters. Each student performed 3 of the 5 selected joint mobilization procedures. All four raters simultaneously scored each student’s performance of the joint mobilization procedures. All testing was videotaped, and 20 students’ performances were scored again at a later time from the videotapes by all four raters. Interrater reliability was calculated using interclass correlation coefficients (ICC 3,1) and percent agreement statistics for both students’ performances during live performance scoring and from video tape. Intrarater reliability was evaluated using interclass correlation coefficients (ICC 3,1) for total task scores. Intrarater reliability of subtask scores was assessed using percent agreement. Results indicated fair to moderate interrater and intrarater reliability as determined by ICCs. Interrater and intrarater reliability of total task scores, as evaluated by percent agreement statistics, was moderate to good. Subtask score reliability was highly variable. Further research into assessment of student performance of psychomotor skills is needed. Key Words: Assessment, Joint mobilization, Psychomotor skills, Student performance, Task analyse