Recovering Optimal Cost Functions for Natural Walking: From Musculoskeletal Simulation to Exoskeleton Control

Human movement studies have contributed to our understanding of how the central nervous system's (CNS) interactions with our body results in rich and complex motor behaviours, such as human gait. Such understanding is particularly important for human-centered engineering such as lower-limb exoskeletons. Assuming the emerged natural gait patterns are the result of some optimization done by CNS, researchers modelled the walking simulation problem as an optimization problem that recast the walking task into a cost function. However, accurately capturing the CNS goal within the cost function is challenging. Cost functions in existing studies were often assumed a priori which either did not lead to natural gait behaviour, or were manually tuned based on the researcher's knowledge which is time consuming. Some studies attempted to tune the cost function algorithmically using inverse optimal control (IOC), but suffered from expensive computation. These limitations hinder the use of IOC for personalized cost function tuning and, by extension, exoskeleton controller design. To address this issue, computationally efficient tuning methods of the cost function were designed and validated in two optimization frameworks: deep reinforcement learning (DRL) and predictive simulation. For DRL, a novel learning method, which generates a control policy with close-to-natural walking behaviour, was developed. The proposed neuromechanically-inspired cost function contributed to the effective learning of the realistic gait by the DRL agent. The nature-inspired curriculum learning scheme led to efficient convergence to natural and bilateral symmetric gait by adaptively tuning the cost function weights while maintaining the agent's walking capability. To further improve the cost function tuning efficiency, an efficient IOC algorithm named Adaptive Reference IOC (AR-IOC) was proposed that used direct collocation for solving optimal gait trajectories and gradient-based weight optimization. We showcased the efficiency of the proposed algorithm in tuning cost functions and matching gait trajectories using both synthetic data and experimental data which outperformed the Genetic Algorithm by more than 80\% in computational time. With the AR-IOC, the correlation between the walking tasks and the cost function weights were studied which revealed a change in cost function compositions with respect to walking speed. With the efficient AR-IOC algorithm, we explored the potential of using predictive simulation to generate physics-informed reference trajectories for lower-limb exoskeleton tracking controllers. First, an accurate human-exoskeleton system was developed. By combining the optimal human cost function obtained using AR-IOC and the exoskeleton cost function, we obtained the optimal gait trajectories for the human-exoskeleton system which were different from the unassisted natural walking trajectories. These optimal trajectories were then tested in real exoskeleton systems using a time-dependent proportional-derivative (PD) controller and their performances in reducing muscle activities were compared to the unassisted natural walking trajectories. The likely limitations of the controller design were also discussed. With the proposed simulation frameworks and the efficient cost function tuning methods, this thesis serves as a catalyst for enabling personalized rehabilitation design based on detailed musculoskeletal simulation. The presented framework, that covers from data collection and post-processing, to simulation and experiments, serves as a guidance and reference to future developments in this field, such as extending the musculoskeletal simulation to impaired subjects with different locomotion tasks, and different control systems for lower-limb rehabilitation