Application of Reinforcement Learning in 5G Millimeter-Wave Networks

The increasingly growing number of mobile communications users and smart devices have attracted researchers and industry pioneers to the largely under-utilized spectrum in the millimeter-wave (mmWave) frequency bands for the 5th generation of wireless networks. This could provide hundreds of times more capacity as compared to 4G cellular networks. The main reason for ignoring the mmWave spectrum until now, has been its vulnerability to signal blockages and possible disconnection or interruption in service. Considering that today’s mobile users expect high reliability and throughput connections, the mmWave signal sensitivity to blockages must be addressed. This research proposes to predict base stations that can service a user without disconnections, given the user’s path or destination in the network. In modern networks, reinforcement learning has been effectively utilized to obtain optimal decisions (or actions being taken) in small state-action spaces. Deep reinforcement learning has been able find optimal policies in larger network spaces. In this work, similar techniques are employed to find ways to serve the user without service disconnection or interruption. First, using dynamic programming for a fixed user path, the exact optimal serving base stations are listed. Then, using Q-learning, the network will learn to predict the optimal user path and serving base stations listed, given a fixed destination for the user. Lastly, deep Q-learning is used to approximate optimal user paths and base station lists along that path, similar to the Q-learning results, which can also be applied to networks with more sophisticated state spaces