Spectral- and energy-efficient antenna tilting in a HetNet using reinforcement learning

In cellular networks, balancing the throughput among users is important to achieve a uniform Quality-of-Service (QoS). This can be accomplished using a variety of cross-layer techniques. In this paper, the authors investigate how the down-tilt of base-station (BS) antennas can be adjusted to maximize the user throughput fairness in a heterogeneous network, considering the impact of both a dynamic user distribution and capacity saturation of different transmission techniques. Finding the optimal down-tilt in a multi-cell interference-limited network is a complex problem, where stochastic channel effects and irregular antenna patterns has yielded no explicit solutions and is computationally expensive. The investigation first demonstrates that a fixed tilt strategy yields good performances for homogeneous networks, but the introduction of HetNet elements adds a high level of sensitivity to the tilt dependent performance. This means that a HetNet must have network-wide knowledge of where BSs, access-points and users are. The paper also demonstrates that transmission techniques that can achieve a higher level of capacity saturation increases the optimal down-tilt angle. A distributed reinforcement learning algorithm is proposed, where BSs do not need knowledge of location data. The algorithm can achieve convergence to a near-optimal solution rapidly (6-15 iterations) and improve the throughput fairness by 45-56% and the energy efficiency by 21-47%, as compared to fixed strategies. Furthermore, the paper shows that a tradeoff between the optimal solution convergence rate and asymptotic performance exists for the self-learning algorithm