Machine learning of motor skills for robotics

UnrestrictedAutonomous robots have been a long standing vision of robotics, artificial intelligence, and cognitive sciences. Early approaches in the 1980s showed that approaches purely based on reasoning and human insights would not be applicable to all perceptuomotor tasks of future robots. Instead, new hope was put in the growing wake of machine learning. However, to date, learning techniques have yet to fulfill this promise as only few task-independent methods scale into manipulator or humanoid robotics. In order to overcome these difficulties, we investigate steps towards a structured, general approach to motor skill learning based on a theoretically well-founded novel framework for task control and appropriate learning algorithms.; As theoretical basis, we introduce a point-wise optimal control framework that allows the derivation of various well-known robot control laws and which has been applied successfully to task space tracking control for several different metrics on the anthropomorphic SARCOS Master Arm.; To overcome the requirement of accurate models, learning controllers for task space control offers an alternative. However, when learning to control a redundant system, we face the general problem of the non-convexity of the solution space. This problem can be resolved using the point-wise cost function of the analytical framework to ensure global consistency. An immediate reinforcement learning algorithm is derived from the expectation-maximization perspective which leads to a reward-weighted regression technique. We demonstrate the feasibility of the resulting framework in redundant endeffector tracking for simulated robot arms.; While learning to execute tasks in task space is an essential component of motor skill learning, learning the actual task is more important. We focus on learning of elemental tasks represented by parameterized motor primitives. While imitation learning of motor primitives is a well-understood, the self-improvement by interaction of the system with the environment remains a challenging problem, tackled in the fourth chapter of this thesis. For pursuing this goal, we highlight the difficulties with current reinforcement learning methods, and outline both established and novel algorithms for the gradient-based improvement of parameterized policies. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm