Efficient Provision of Service Function Chains in Overlay Networks using Reinforcement Learning

IEEE Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) technologies facilitate deploying Service Function Chains (SFCs) at clouds in efficiency and flexibility. However, it is still challenging to efficiently chain Virtualized Network Functions (VNFs) in overlay networks without knowledge of underlying network configurations. Although there are many deterministic approaches for VNF placement and chaining, they have high complexity and depend on state information of substrate networks. Fortunately, Reinforcement Learning (RL) brings opportunities to alleviate this challenge as it can learn to make suitable decisions without prior knowledge. Therefore, in this paper, we propose an RL approach for efficient SFC provision in overlay networks, where the same VNFs provided by multiple vendors are with different performance. Specifically, we first formulate the problem into an Integer Linear Programming (ILP) model for benchmarking. Then, we present the online SFC path selection into a Markov Decision Process (MDP) and propose a corresponding policy-gradient-based solution. Finally, we evaluate our proposed approach with extensive simulations with randomly generated SFC requests and a real-world video streaming dataset, and implement an emulation system for feasibility verification. Related results demonstrate that performance of our approach is close to the ILP-based method and better than deep Q-learning, random, and load-least-greedy methods