Supervising the Reinforcement Learning of a joint control policy

Along this paper, we propose to model the learning process of the controller policy of a humanoid joint using an already known model of cognition. A policy gradient reinforcement learning method is used as optimization alternative to compute the function that commands the behavior of a robot’s motor. The objective is to model the trial-by-trial evolution of the learning and be able to observe its behavior, setting a starting point for the stability analysis of the learning algorithm. In order to achieve it the problem has been split in two input/output models related by a basis function. Dynamical system theory is used as alternative to create a cognition model, instead of the traditional computational hypothesis where cognitive adaptive agents are represented as a collection of logical instructions; this work focuses on the agent environment interaction and the bodily and sensorimotor mechanisms underlying cognitive processes. A general model is presented and a linear implementation of practical use formulated. As a demonstration, the PID control policy of a simulated humanoid’s joint is learned and their evolution modeled. The analysis of the results is developed from control system theory and machine learning points of view