Learning to Exploit Dynamics for Robot Motor Coordination

Humans exploit dynamics—gravity, inertia, joint coupling, elasticity, and so on—as a regular part of skillful, coordinated movements. Such movements comprise everyday activities, like reaching and walking, as well as highly practiced maneuvers as used in athletics and the performing arts. Robots, especially industrial manipulators, instead use control schemes that ordinarily cancel the complex, nonlinear dynamics that humans use to their advantage. Alternative schemes from the machine learning and intelligent control communities offer a number of potential benefits, such as improved efficiency, online skill acquisition, and tracking of nonstationary environments. However, the success of such methods depends a great deal on structure in the form of simplifying assumptions, prior knowledge, solution constraints and other heuristics that bias learning. My premise for this research is that crude kinematic information can supply the initial knowledge needed for learning complex robot motor skills—especially skills that exploit dynamics as humans do. This information is readily available from various sources such as a coach or human instructor, from theoretical analysis of a robot mechanism, or from conventional techniques for manipulator control. In this dissertation I investigate how each type of kinematic information can facilitate the learning of efficient “dynamic” skills. This research is multidisciplinary with contributions along several dimensions. With regard to biological motor control, I demonstrate that motor synergies, i.e, functional units that exploit dynamics, evolve when trial-and-error learning is applied to a particular model of motor skill acquisition. To analyze the effects of velocity on dynamic skills and motor learning, I derive an extension to the notion of dynamic manipulability that roboticists use to quantify a robot’s capabilities before specification of a task. And along the machine learning dimension, I develop a supervised actor-critic architecture for learning a standard of correctness from a conventional controller while improving upon it through trial-and-error learning. Examples with both simulated and real manipulators demonstrate the benefits that this research holds for the development of skillful, coordinated robots