Training speech pathology students in the perceptual evaluation of speech in patients with cleft palate : reliability results and the students' perspective

Introduction: Listener training is necessary to achieve agreement between perceptual ratings. However, evidence is limited regarding the effect of a training in the perceptual evaluation of cleft palate speech on the reliability of students in speech pathology. The aim of this study was to evaluate the short-term and longer-term effect of a training using the Belgian Dutch outcome tool for cleft palate speech on reliability and the students' experiences.Methods: A 2-h training was provided during a specialist course for 31 first year master students in speech pathology. Information regarding listener characteristics, speech documentation and speech analysis (speech parameters and rating scales) was discussed. This last part focused on the Belgian Dutch outcome tool for speech in patients with cleft palate, including reference samples and consensus listening exercises. Pre training (T1) and post-training reliability (immediately following the training (T2) and one month later (T3)) were collected for the speech variables speech understandability, speech acceptability, hypernasality, hyponasality, nasal emission and nasal turbulence that were rated by means of ordinal scales. Inter-rater reliability was determined with interclass correlation coefficients whereas percentages of agreement were calculated for the intra-rater reliability.Results: Qualitative inter-rater ICC categories improved for hyponasality and speech acceptability at T2 and T3, whereas the inter-rater reliability of the other variables remained stable regarding their interpreted categories. All ICC values quantitatively improved at T2, except hypernasality and nasal emission. At T3, three of those variables showed a small drop one month later. Regarding intra-rater reliability, a significant decrease for hypernasality and nasal turbulence was found at T2, whereas a significant increase was observed for nasal turbulence and speech acceptability at T3. All variables, except hypernasality, had improved intra-rater reliability at T3 in comparison with T1. Students experienced fewer difficulties with most variables after training. Evaluation of nasal emission and nasal turbulence remained however difficult and required, in the students' opinion, more attention.Discussion & conclusion: Overall a positive training effect was noted, especially regarding the inter-rater reliability and the students' confidence in perceptual ratings. However, it remains a challenge to achieve a positive training effect for all target variables that is maintained over-time