Machine Behavior Development and Analysis using Reinforcement Learning

We are approaching a future where robots and humans will co-exist and co-adapt. To understand how can a robot co-adapt with humans, we need to understand and develop efficient algorithms suitable for our interactive purposes. Not only it can help us to advance the field of robotics but also it can help us to understand ourselves. A subject Machine Behavior, proposed by Iyad Rahwan in a recent Science article, studies algorithms and the social environments in which algorithms operate. What this paper's view tells us is that, when we would like to study any artificial robot we create, like natural science, a two-step method based on logical positivism should be applied. That is, we need to, on one hand, provide a complicated theory based on logical deduction, and on another hand, empirically setup experiments to conduct. Reinforcement learning (RL) is a computational model that helps us to build a theory to explain the interactive process. Integrated with neural networks and statistics, the current RL is able to obtain a reliable learning representation and adapt over interactive processes. It might be one of the first times that we are able to use a theoretical framework to capture uncertainty and adapt automatically during interactions between humans and robots. Though some limitations are observed in different studies, many positive aspects have also been revealed. Additionally, considering the potentials of these methods people observed from related fields e.g. image recognition, physical human-robot interaction and manipulation, we hope this framework will bring more insights to the field of robotics. The main challenge in applying Deep RL to the field of social robotics is the volume of data. In traditional robotics problems such as body control, simultaneous localization and mapping and grasping, deep reinforcement learning often takes place only in a non-human environment. In such an environment, the robot can learn infinitely in the environment to optimize its strategies. However, applications in social robotics tend to be in a complex environment of human-robot interaction. Social robots require human involvement every time they learn in such an environment, which leads to very expensive data collection. In this thesis, we will discuss several ways to deal with this challenge, mainly in terms of two aspects, namely, evaluation of learning algorithms and the development of learning methods for human-robot co-adaptation