Computation Offloading in Heterogeneous Vehicular Edge Networks: On-line and Off-policy Bandit Solutions

In this paper, we consider the computation offloading problem from mobile users in a heterogeneous vehicular edge computing scenario and focus on the network and base station selection problem, where the different networks have different traffic loads. In a fast-varying vehicular environment, computation offloading of users is strongly affected by the latency due to the congestion at the edge computing servers. However, as a result of the dynamicity of such an environment and information shortage, predicting this congestion is an involved task. To address this challenge, we propose an on-line learning algorithm and an off-policy learning algorithm based on multi-armed bandit theory. To dynamically select the least congested network in a piece-wise stationary environment, these algorithms predict the latency of the offloaded tasks using the offloading history. In addition, to minimize the task loss we develop a method for base station selection and a relaying mechanism. Through intensive numerical analysis, we demonstrate that the proposed learning-based solutions adapt to the traffic changes of the network by selecting the least congested network, thereby reducing the latency of offloaded tasks. Moreover, we demonstrate that the proposed joint base station selection and the relaying mechanism minimize the task loss in a vehicular environment