Learning system adaptive legged robotic locomotion policies

The ability to form support contacts at discontinuous locations makes legged robots suitable for locomotion over highly unstructured terrains. While recent years have witnessed significant robotic developments, delivering extremely dynamic and robust hardware solutions, the control intelligence for legged robots to perform agile and sophisticated maneuvers remains an active area of research. This thesis, therefore, focuses on the control of legged systems, particularly, quadrupedal robots. The research presented in this thesis is driven by the motivation that a controller governing the behavior of a system should thoroughly utilize its potential while also adapting to variations in system dynamics through emergence of behavior that still achieves the control objective. Sampling-based search methods allow explorations along vast regions of operation, thereby, enabling discovery of near-optimal solutions. Similarly, data-driven reinforcement learning (RL) strategies allow development of controllers which exploit system dynamics to achieve the control objective described by a high-level reward function. The problem of legged robot locomotion can thus be approached using reinforcement learning to obtain robust and dynamic control solutions. Additionally, the control policy describing the behavior of the system can be parameterized as a deep neural network to perform complex non-linear mapping of the robot state information to desired control action. This approach of utilizing a deep neural network is then referred to as deep reinforcement learning. In this dissertation I focus on employing deep RL strategies for quadrupedal locomotion. I present that encouraging the RL control policy to even implicitly model system dynamics allows for emergence of adaptive control behaviors. I use this observation throughout this thesis to develop system adaptive control strategies leading up to an ambitious goal of obtaining an RL locomotion policy capable of zero-shot transfer to quadrupeds of varying kinematic and dynamic properties. Although the main contributions relate to RL, I also recognize the benefits and drawbacks of different control approaches, and thus explore modular control architectures which utilize both model-based and data-driven model-free strategies for robot locomotion. In this regard, I present training and control architectures which exhibit dynamic locomotion behavior over terrains with varying steps and inclines, both in lab experiments and field trials. In combination, the works presented in this thesis investigate, and, in my firm belief, further the state of legged robotic control advocating for development of artificial control intelligence which matches the level of complexity demonstrated by massively evolved biological counterparts.