Robotics: Environmental awareness through cognitive sensor fusion

ii In order for autonomous mobile robots to survive in the real world they have to be aware of the environment. The self-assembling micro robots developed in the European project Replicator are destined for such a task. By using several modalities, these robots must be able to detect and recognize interesting objects in the environment. This thesis presents a biologically inspired cognitive sensor fusion architecture to create environmental awareness in these micro robots. This architecture consists of a bi-modal attention module and multi-modal sensor fusion. A state of the art visual saliency detection system has been optimized and combined with biologically based sensor fusion methods to obtain a visual-acoustic at-tention module. For multi-modal sensor fusion a new type of ARTMAP (self-organizing as-sociative memory) called the Multi-directional ARTMAP (MdARTMAP) has been designed. With this MdARTMAP a module for unsupervised on-going learning of sound was devel-oped by clustering states of an echo state network, which processes cochlear filtered audio. Also unsupervised visual object recognition was obtained with this MdARTMAP by cluster