Anticonvulsant teratogenesis 4: Inter-rater agreement in assessing minor physical features related to anticonvulsant therapy

Background: We report on inter-rater agreement in the assessment of newborn infants with respect to a range of minor physical features in a cohort study of the fetal effects of maternal anticonvulsant use during pregnancy. Methods: Infants from three groups (exposed to anticonvulsants, seizure history but no medication exposure, and unexposed controls) were examined by both a pediatrician/teratologist, who was blinded with respect to the mother's exposure status, and an unblinded research assistant. Agreement on assessments for selected anomalies associated with anticonvulsant therapy was measured by kappa-statistics, as well as by more sensitive log-linear modeling techniques, which allow examination of possible covariate effects on the strength of agreement. Although the physician and research assistant agreed on a high proportion of cases (80-90%), kappa values were modest (0.2-0.5), partly because of the low prevalence of the anomalies considered. To explore how agreement varies within subgroups, we used recently developed methods for studying agreement based on log-linear models. Results: Log-linear modeling indicated that there was substantial variation in pattern of agreement between different individual research assistants but that other factors (e.g., exposure category, sex, and birthweight) did not appear to be related to agreement. Our results suggest that research assistants with more experience showed the highest degree of agreement with the physicians. Conclusions: Our results have implications for both clinical practice and epidemiologic research and underline the importance of thorough training of staff in the definitions to be used and also the need for multiple independent assessments of these subtle anomalies. (C) 2000 Wiley-Liss, Inc