Model-Based Reinforcement Learning Framework of Online Network Resource Allocation

Online Network Resource Allocation (ONRA) for service provisioning is a fundamental problem in communication networks. As a sequential decision-making under uncertainty problem, it is promising to approach ONRA via Reinforcement Learning (RL). But, RL solutions suffer from the sample complexity issue; i.e., a large number of interactions with the environment are needed to find an efficient policy. This is a barrier to utilize RL for ONRA as on one hand, it is not practical to train the RL agent offline due to lack of information about future requests, and on the other hand, online training in the real network leads to significant performance loss because of the sub-optimal policy during the prolonged learning time. The performance degradation is even higher in non-stationary ONRA where the agent should continually adapt the policy with the changes in service requests. To address this issue, we develop a general resource allocation framework, RADAR, using model-based RL for a class of ONRA problems with known immediate rewards. RADAR improves sample efficiency via exploring the state space in background and exploiting the policy in decisiontime using synthetic samples by the model of the environment, which is trained by real interactions. Applying RADAR on the multi-domain service federation problem to maximize profit via selecting proper domains for service requests deployment, shows its continual learning capability and up to 44% performance improvement w.r.t. the standard model-free RL solution. © 2022 IEEE.This work has been partially funded by the EC H2020 5Growth Project (grant no. 856709), and Generalitat de Catalunya grant 2017 SGR 1195. © 2022, IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other work