Data-driven acquisition of closed-loop robotic skills

2019-03-13As robots enter our daily lives they will have to perform a high variety of complex tasks, make sense of their high-dimensional sensory inputs, and cope with highly unstructured, uncertain and dynamic real-world environments. In order to address these challenges, the robots have to be able to not only improve their existing skills but also acquire new skills when presented with novel tasks. In this thesis, we develop several data-driven robot learning approaches and demonstrate their ability to learn challenging tasks using various sources of training signals and sensory information. ❧ First, this work presents a method for decomposing the training of high-dimensional visuo-motor policies into efficient learning of local components for tasks with non-linear and discontinuous dynamics. Next, we develop an algorithm for combining model-based optimization with model-free learning that removes the bias of the simplified models while still maintaining high sample efficiency for learning tasks with complex dynamics. In the second part of the thesis, a concept of self-supervision is used to employ external sources of information, such as pre-trained reward estimators and reward signals from different domains, to better automate the policy training. In particular, we present a self-supervised framework for learning grasp correction policies using tactile data and reinforcement learning, and show how robots can learn from third-person human video demonstrations in a self-supervised manner by building a common representation model of human videos and robot camera images. In the last part of the thesis, we first concentrate on imitation learning as a way to accelerate the acquisition of robotic skills, and present a method for training multi-modal stochastic policies from unstructured and unlabelled demonstrations that employs recent developments in generative adversarial learning. Finally, we show how to reduce the amount of needed real-world trials by moving a large portion of policy training into simulated environments and using only small amounts of real-world experience to adjust the distribution of simulated scenarios for a better transferability to the real world