Multimodal semantic learning from child-directed input

Comunicació presentada a: 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies celebrat del 12 al 17 de juny de 2016 a San Diego, California.Children learn the meaning of words by being exposed to perceptually rich situations (linguistic discourse, visual scenes, etc). Current computational learning models typically simulate these rich situations through impoverished symbolic approximations. In this work, we present a distributed word learning model that operates on child-directed speech paired with realistic visual scenes. The model integrates linguistic and extra-linguistic information (visual and social cues), handles referential uncertainty, and correctly learns to associate words with objects, even in cases of limited linguistic exposure.We acknowledge the European Network on Integrating Vision and Language for a Short-Term Scientific Mission grant, awarded to Raquel Fernandez to visit the University of Trento