Internal control for autonomous open-ended acquisition of new behaviors

Mikhailova I. Internal control for autonomous open-ended acquisition of new behaviors. Bielefeld (Germany): Bielefeld University; 2009.With increasing demand for the autonomy and adaptivity of robots the focus of research on intelligent systems progressively moved from the direct solving of specific task to more sophisticated approach of learning task up to the highest level where the systems decide themselves what to learn when. Inspired by the example of child development modern research targets a similar process in artificial systems. The expected benefits of artificial development are higher degree of adaptivity to unforeseen situations, no necessity to redesign the system every time the task of the robot changes, and a break through the limits of the complexity of hand-designed behaviors. The aim of this work is the design of an initial system that can bootstrap a developmental process. We investigate which minimal internal control structures are needed in order to interact with the environment and acquire new behaviors in a task-unspecific and open-ended manner, i.e. without cancelling the development after the learning of several behaviors. Hence, we carefully analyze the elements of the initial bootstrapping: value system, abstraction system, and innate behaviors with respect to open-endness and sufficient grounding. We formulate the constraints and requirements for the design of these parts. We then investigate which possibilities do exist for the value system to influence the acquisition of the internal representations of the system-environment interaction. This analysis reveals a taxonomy of different ways to memorize the experience for the purpose of behavior generation. We discuss how the learning in one type supports the learning in other types thus creating a basis for an open-ended process. Inspired by brain research we propose an architecture that uses different abstraction types in parallel for the homeostatic control in a value system. We validate our general ideas about the system design by means of real-world experiments, which focus on the robot's behavior during learning-related interaction with a human: learning of object recognition, acquisition of gestures, and learning of speech labels. We implement a reactive layer as innate behavior for general interaction with the environment and extend it by increasingly complex types of abstraction layers. We demonstrate several methods for the stabilization of coupled dynamics of changing needs and rewards, changing internal representations and changing behavior: differentiation between long-term and short-term statistics, learning of different structures on different time-scales, and loose coupling of parallel control loops. The tested extension of the reactive layer shows a principle way for the qualitative progress from reactive to expectation-driven behavior. Exactly this task-unspecific development was the aim of this work. Summarizing, our work proposes a design of the initial bootstrapping system for the task-unspecific open-ended development. The design is validated by multiple implementations of real-world online learning from interaction between the robot and its environment. Several design features inspired by the brain research - homeostatic control, parallel representation structures, loose coupling between parallel control layers - contribute to the overall stability of the proposed architecture and make it attractive for further research