Simulating imitative learning in a humanoid robot for the purpose of language acquisition

Humans are born with an innate mechanism to recognize faces. Infants within weeks after birth are able to mimic facial gestures—an early but significant milestone towards adulthood. These skills are vital for interacting with their environment in order to develop their language from babbling to first word forms. Adapting the abilities from this period of infancy towards humanoid robots can produce a testable model of language acquisition. Learning a language requires more than just aural observation. By developing the visual component, the robot can observe features such as lips, teeth, and tongue in order to learn correspondence between speech and motor functions. This thesis demonstrates progress toward the goal of developing a live lip-reading system. The first part describes experimentation with standard computer vision methods for human facial recognition, with emphasis on the mouth and jaw regions of the face. The next part shows methods to segment the mouth and discover salient patterns of lip configuration. The final challenge for this thesis is to implement this system live on the iCub, the humanoid robot of the Language Acquisition and Robotics Group, and have it respond in a way that demonstrates its interpretation of the lip movements. Although the face and mouth can be consistently located within a given image frame, the challenge encountered was in segmenting the mouth into relevant features. Distinct attributes such as teeth and tongue were barely present or not visible in the image. This limited the analysis to mainly lip configurations. However, the analysis could still pinpoint distinct moments where teeth or tongue were showing.Ope