Effect of formative evaluation using direct observation of procedural skills in assessment of postgraduate students of obstetrics and gynecology: Prospective study

Introduction: Direct Observation of Procedural Skills (DOPS)\ud is a way of evaluating procedural skills through observation in\ud the workplace. The purpose of this study was to assess the role of\ud DOPS in teaching and assessment of postgraduate students and to\ud know the effect of repeated DOPS on improvement of the skills\ud and confidence of the students.\ud Methods: This prospective study was conducted in the Obstetrics\ud and Gynecology department of a rural tertiary centre in Northern\ud India in two phases. All postgraduate students (n=20) were\ud randomly divided year-wise into two groups, using lottery system\ud such that both groups had equal number of first, second and\ud third year students for didactic lecture/simulation teaching using\ud mannequins, with hands-on exposure on Active management of\ud third stage of labor (AMTSL) and Postpartum hemorrhage (PPH)\ud management. In the first phase, Group 1 received simulation\ud teaching using mannequins with hands on for 4 sittings, and Group\ud 2 received didactic lecture on AMTSL. Following the students’\ud performance of AMTSL steps on delivering women, their\ud competencies were assessed using DOPS structured checklist of\ud ten points on 6 sessions. The students’ performance in six DOPS\ud was compared between the two groups. In the 2nd phase, after\ud flipping of the groups, group one received didactic lecture, and\ud group 2 simulation teaching on PPH management with hands on\ud for 4 sittings, followed by comparison of six DOPS performance\ud in both groups. The data were analyzed by Wilcoxon rank-sum\ud (Mann–Whitney) test using SPSS software, version 20.\ud Results: In both phases, significant difference was observed\ud between the two groups on first DOPS comparison (1st phase:\ud p=0.000; 2nd phase: p=0.002), with simulation group performing\ud better. Comparison of sixth DOPS in the two groups revealed no\ud difference in both phases, but significant difference on first and\ud sixth DOPS comparison in each group (p=0.000).\ud Conclusion: Repeated DOPS results in improved skills and\ud confidence of students in managing real life obstetric emergencies\ud irrespective of the teaching modality