Safe Hierarchical Reinforcement Learning for Fixed-Wing Flight Control

As a branch of machine learning, reinforcement learning (RL) has gained interest among researchers in the field of flight control. Flat reinforcement learning (FRL) methods often require many samples to achieve the desired training result, which is a time-consuming process. One learning method that has the potential to improve the learning efficiency is hierarchical reinforcement learning (HRL); this learning method divides the control tasks or control inputs into several hierarchies and the hierarchies are solved by lower-level agents or lower-level actions. Besides, it is often unsafe to apply flat reinforcement learning models to the learning of flight control. This is because the exploration process of the RLA can lead the aircraft into unsafe states. The contribution of this research is mitigating the curse of dimensionality and improving the safety of an existing hierarchical reinforcement learning method on fixed-wing flight control. The curse of dimensionality is mitigated by embedding the deep deterministic policy gradient algorithm (DDPG) in the existing HRL based on the dynamic decomposition framework. The safety of reinforcement learning is improved by an auxiliary safety modification layer (SML) with prior knowledge of the system dynamics that corrects the action generated by the agent. In this research, both the HRL and safe reinforcement learning (SRL) methods are applied on fixed-wing flight control and further effects on sample efficiency and learning performance are investigated with a fixed-wing flight model. It has shown that the HRL based on dynamic decomposition method has higher sample efficiency than FRL method and the SRL by SML method can effectively ensure the safety of the HRL learning progress.Aerospace Engineerin