Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty

Recently, the advances in reinforcement learning have enabled an artificial intelligent agent to solve many challenging problems (e.g. AlphaGo) at unprecedented levels. However, the robustness of reinforcement learning in safety critical operation remains unclear. In this work, the applicability of reinforcement learning in Air Traffic Control was explored. We focus on building an algorithm to automate flight conflict resolution which is the ultimate goal of air traffic control. For that purpose, a simulator, that provides a learning environment for reinforcement learning, was developed to simulate a variety of air traffic scenarios. We propose a variant of the reinforcement learning approach to resolve conflict in airspace and investigate the performance of the method in achieving that. A reinforcement learning model, specifically a deep deterministic policy gradient, was adopted to learn the conflict resolution with continuous action spaces. Experimental results demonstrate that our proposed method is effective in resolving the conflict between two aircraft even in the presence of uncertainty. The accuracy of our model is 87% at different uncertainty levels. Our findings suggest that reinforcement learning is a promising approach to conflict resolution.Civil Aviation Authority of Singapore (CAAS)Accepted versionThis research has been partially supported under Air Traffic Management Research Institute (NTU-CAAS) Grant No. M4062429.05