Aircraft-to-aircraft separation based on reinforcement learning

Air traffic has been increasing and with it the workload of air traffic controllers. Despite the pandemic, the latest figures show a rapid recovery and forecast exponential growth. This indicates the need to modernise air traffic control and the technology used, which is already being developed and implemented by organisations like SESAR, like applying AI to air traffic control (DART). A support tool with automatic conflict avoidance would be a great step to address the problem of possible overcapacity of air traffic controllers. This document describes two possible implementations of a conflict avoidance tool. The approach is to use Deep Reinforcement Learning to select actions that avoid conflict and help the air traffic controllers to take faster and better decisions. The basis for both approaches is a simple 2D airspace simulator and the same policy applied to all the aircraft. The first proposal is a stand-alone DQN algorithm (DRL) that has a 7.06% improvement in the number of simultaneous conflicts compared to the original environment without applying a policy. The second approach is a DQN algorithm that incorporates transfer learning of the rules of the air, and it is called by the acronym DRLT. It resulted in a degradation compared to the original environment, with a 6% increase in unremembered conflicts. Nevertheless, Deep Reinforcement Learning has shown a decrease in decision time and the idea of reusing the same strategy for all aircraft has solved the problem of unpredictability issue that some reinforcement learning solutions had. The proposal could be a good start for a self-separation tool for unmanned aircraft but still needs future improvements in results. It is not suitable for air traffic controllers or piloted vehicles due to the increased workload it would suppose