Autonomous learning of action-word semantics in a humanoid robot

For creation of an artificial agent that is capable of using language naturally, models that only manipulate symbols or classify speech are ineffective. The semantic information which language conveys must be grounded in the agent’s complete sensorimotor experience. Typically, patterns from visual, auditory, and proprioceptive data streams which share the same conceptual cause are fused together in an associative memory at the core of the language model. Coupling of motor and auditory modalities, which is crucial for a large part of semantic understanding, presents a particularly difficult challenge. Words and actions both need models capable of capturing spatial and temporal structure, and training algorithms that can learn in a self-organizing, incremental fashion. Presented is a method for online learning of word and action lexicons based on the hidden Markov model. The model is then evaluated through action-word learning experiments implemented on a humanoid robot