Children's oral health in Italy: training and clinical calibration of examiners for the National Pathfinder about caries disease

PURPOSE: To evaluate the dental status of 12-year-old children in 2004, an epidemiological survey called "National pathfinder about children's oral health in Italy" was planned. The present paper describes the clinical calibration method. MATERIALS AND METHODS: Thirty children (20 caries-affected and 10 caries-free) were selected. Seven examiners received 35 hours of training and calibration in making clinical measurements by a single trainer. The trainer's diagnosis was the benchmark validity reference. Increase in sensitivity was tested using paired Student's t-test. Inter-examiner reliability for DMFS was evaluated using Pearson's correlation coefficients and ANOVA. Percent agreement and Kappa statistics were calculated for dichotomous judgments (tooth-by-tooth analysis and caries prevalence). RESULTS: The raters' sensitivity increased from calibration I to calibration II (p < 0.05). For DMFS a good agreement between raters was found without significant differences among them at both calibration exercises. The Pearson's correlation coefficients between all pairs of raters were significant (p < 0.001) after both calibration exercises. Tooth-by-tooth reliability was high: the strength of agreement was substantial or almost perfect in calibration I (Kappa statistics from 0.81 to 0.95) and almost perfect in calibration II (Kappa statistics from 0.92 to 0.97). Regarding caries prevalence, sensitivity, specificity, percent agreement and Kappa statistics results were high but without significant differences between individual pairs of observations in the two calibration sessions (p > 0.05). CONCLUSIONS: The results suggest that sensitivity measurements should be used for agreement assessments in addition to the other World Health Organization-recommended reliability measurements. Kappa statistics are the eligible measurement of reliability for dichotomous judgments.