Classifying Turn-Level Uncertainty Using Word-Level Prosody

Spoken dialogue researchers often use supervised machine learning to classify turn-level user affect from a set of tur n-level features. The utility of sub-turn features has been less exp lored, due to the complications introduced by associating a variable number of sub-turn units with a single turn-level classifica tion. We present and evaluate several voting methods for using word- level pitch and energy features to classify turn-level user un- certainty in spoken dialogue data. Our results show that when linguistic knowledge regarding prosody and word position is in- troduced into a word-level voting model, classification acc uracy is significantly improved compared to the use of both turn-le vel and uninformed word-level models. Index Terms: emotion recognition, speech dialogue systems