Towards real-time routing optimization with deep reinforcement learning: open challenges

The digital transformation is pushing the existing network technologies towards new horizons, enabling new applications (e.g., vehicular networks). As a result, the networking community has seen a noticeable increase in the requirements of emerging network applications. One main open challenge is the need to accommodate control systems to highly dynamic network scenarios. Nowadays, existing network optimization technologies do not meet the needed requirements to effectively operate in real time. Some of them are based on hand-crafted heuristics with limited performance and adaptability, while some technologies use optimizers which are often too time-consuming. Recent advances in Deep Reinforcement Learning (DRL) have shown a dramatic improvement in decision-making and automated control problems. Consequently, DRL represents a promising technique to efficiently solve a variety of relevant network optimization problems, such as online routing. In this paper, we explore the use of state-of-the-art DRL technologies for real-time routing optimization and outline some relevant open challenges to achieve production-ready DRL-based solutions.This work has received funding from the European Union’s Horizon 2020 research and innovation programme within the framework of the NGI-POINTER Project funded under grant agreement No 871528. This paper reflects only the author’s view; the European Commission is not responsible for any use that may be made of the information it contains. This work was also supported by the Spanish MINECO under contract TEC2017-90034-C2-1-R (ALLIANCE), the Catalan Institution for Research and Advanced Studies (ICREA) and the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund.Peer ReviewedPostprint (author's final draft