Learning to Ascend Stairs and Ramps:Deep Reinforcement Learning for a Physics-Based Human Musculoskeletal Model

This paper proposes to use deep reinforcement learning to teach a physics-based humanmusculoskeletal model to ascend stairs and ramps. The deep reinforcement learning architectureemploys the proximal policy optimization algorithm combined with imitation learning and is trainedwith experimental data of a public dataset. The human model is developed in the open-sourcesimulation software OpenSim, together with two objects (i.e., the stairs and ramp) and the elasticfoundation contact dynamics. The model can learn to ascend stairs and ramps with muscle forcescomparable to healthy subjects and with a forward dynamics comparable to the experimental trainingdata, achieving an average correlation of 0.82 during stair ascent and of 0.58 during ramp ascentacross both the knee and ankle joints.Keywords: deep reinforcement learning; computer simulatio