Out-of-Task Utterance Detection Based on Bag-of-Words Using Automatic Speech Recognition Results

APSIPA ASC 2011: 2011 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, October 18-21, 2011, Xi'an, China.Example-based question answering (QA) is an effective approach for real-world spoken dialogue systems. A limitation of an example-based QA is that a system cannot appropriately respond to a user’s question, if a similar questionanswer pair does not exist in the question and answer database (QADB). For a robust spoken dialogue system, it is important to classify if a user’s utterance is in the task or out of the task. In this paper, we describe our approach for out-of-task utterance (OOT) detection. Using the Support Vector Machines (SVM), the detection model is trained with the bag of words from the 10-best automatic speech recognition (ASR) results. The number of words in a question, the number of unknown words, and the maximum similarity score against QADB are also used as features for the OOT detection. We apply our detection model to the Takemaru-kun dialogue system. We evaluate our detection model using adult's utterances of two years and child’s utterances of one year spoken to Takemaru-kun. Our proposed method decreases the Equal Error Rate (EER) using speech recognition results by 4.4% (from 21.3% to 16.9%) in adult’s speech and by 3.6% (from 31.8% to 28.2%) in child’s speech, compared with the baseline method