Deep reinforcement learning-based safe interaction for industrial human-robot collaboration using intrinsic reward function

Aiming at human-robot collaboration in manufacturing, the operators safety is the primary issue during the manufacturing operations. This paper presents a deep reinforcement learning approach to realize the real-time collision-free motion planning of an industrial robot for human-robot collaboration. Firstly, the safe human robot collaboration manufacturing problem is formulated into a Markov decision process, and the mathematical expression of the reward function design problem is given. The goal is that the robot can autonomously learn a policy to reduce the accumulated risk and assure the task completion time during human-robot collaboration. To transform our optimization object into a reward function to guide the robot to learn the expected behaviour, a reward function optimizing approach based on the deterministic policy gradient is proposed to learn a parameterized intrinsic reward function. The reward function for the agent to learn the policy is the sum of the intrinsic reward function and the extrinsic reward function. Then, a deep reinforcement learning algorithm intrinsic reward-deep deterministic policy gradient (IRDDPG), which is the combination of the DDPG algorithm and the reward function optimizing approach, is proposed to learn the expected collision avoidance policy. Finally, the proposed algorithm is tested in a simulation environment, and the results show that the industrial robot can learn the expected policy to achieve the safety assurance for industrial human-robot collaboration without missing the original target. Moreover, the reward function optimizing approach can help make up for the designed reward function and improve policy performance.Funding Agencies|National Natural Science Foundation of ChinaNational Natural Science Foundation of China (NSFC) [51775399, 51675389]; Fundamental Research Funds for the Central UniversitiesFundamental Research Funds for the Central Universities [WUT: 2020III047]; International Science and Technology Innovation Cooperation Project of Sichuan Province [20GJHZ0039]The accepted version of this article is available on the author's personal web page