Learning the Correspondence between Continuous Speeches and Motions

Roy [1] developed a computational model of early lexical learning to address three questions: First, how do infants discover linguistic units? Second, how do they learn perceptually-grounded semantic categories? And third, how do they learn to associate linguistic units with appropriate semantic categories? His model coupled speech recordings with static images of objects, and acquired a lexicon of shape names. Kaplan et al. [2] presented a model for teaching names of actions to an enhanced version of AIBO. The AIBO had built-in speech recognition facilities and behaviors. In this study, we try to build a system that learns the correspondence between continuous speeches and continuous motions without a built-in speech recognizer nor built-in behaviors. We teach RobotPHONE (Fig. 1) to respond to voices properly by taking its hands. For example, one says ‘bye-bye ’ to the RobotPHONE holding its hand and waving. Fig. 2 shows the overview of the proposed system. From continuous input, the system must segment speech and discover acoustic units which correspond to words. The segmentation is done based on recurrent patterns which was found by Incremental Reference Interval-Free Continuous DP (IRIFCDP) [5], [6], and we accelerate the IRIFCDP usin