Performance Analysis and Learning Algorithms in Advanced Wireless Networks

Over the past decade, wireless data traffic has experienced an exponential growth, especially with multimedia traffic becoming the dominant traffic, and such growth is expected to continue in the near future. This unprecedented growth has led to an increasing demand for high-rate wireless communications.Key solutions for addressing such demand include extreme network densification with more small-cells, the utilization of high frequency bands, such as the millimeter wave (mmWave) bands and terahertz (THz) bands, where more bandwidth is available, and unmanned aerial vehicle (UAV)-enabled cellular networks. With this motivation, different types of advanced wireless networks are considered in this thesis. In particular, mmWave cellular networks, networks with hybrid THz, mmWave and microwave transmissions, and UAV-enabled networks are studied, and performance metrics such as the signal-to-interference-plus-noise ratio (SINR) coverage, energy coverage, and area spectral efficiency are analyzed. In addition, UAV path planning in cellular networks are investigated, and deep reinforcement learning (DRL) based algorithms are proposed to find collision-free UAV trajectory to accomplish different missions. In the first part of this thesis, mmWave cellular networks are considered. First, K-tier heterogeneous mmWave cellular networks with user-centric small-cell deployments are studied. Particularly, a heterogeneous network model with user equipments (UEs) being distributed according to Poisson cluster processes (PCPs) is considered. Distinguishing features of mmWave communications including directional beamforming and a detailed path loss model are taken into account. General expressions for the association probabilities of different tier base stations (BSs) are determined. Using tools from stochastic geometry, the Laplace transform of the interference is characterized and general expressions for the SINR coverage probability and area spectral efficiency are derived. Second, a distributed multi-agent learning-based algorithm for beamforming in mmWave multiple input multiple output (MIMO) networks is proposed to maximize the sum-rate of all UEs. Following the analysis of mmWave cellular networks, a three-tier heterogeneous network is considered, where access points (APs), small-cell BSs (SBSs) and macrocell BSs (MBSs) transmit in THz, mmWave, microwave frequency bands, respectively. By using tools from stochastic geometry, the complementary cumulative distribution function (CCDF) of the received signal power, the Laplace transform of the aggregate interference, and the SINR coverage probability are determined. Next, system-level performance of UAV-enabled cellular networks is studied. More specifically, in the first part, UAV-assisted mmWave cellular networks are addressed, in which the UE locations are modeled using PCPs. In the downlink phase, simultaneous wireless information and power transfer (SWIPT) technique is considered. The association probability, energy coverages and a successful transmission probability to jointly determine the energy and SINR coverages are derived. In the uplink phase, a scenario that each UAV receives information from its own cluster member UEs is taken into account. The Laplace transform of the interference components and the uplink SINR coverage are characterized. In the second part, cellular-connected UAV networks is investigated, in which the UAVs are aerial UEs served by the ground base stations (GBSs). 3D antenna radiation combing the vertical and horizontal patterns is taken into account. In the final part of this thesis, deep reinforcement learning based algorithms are proposed for UAV path planning in cellular networks. Particularly, in the first part, multi-UAV non-cooperative scenarios is considered, where multiple UAVs need to fly from initial locations to destinations, while satisfying collision avoidance, wireless connectivity and kinematic constraints. The goal is to find trajectories for the cellular-connected UAVs to minimize their mission completion time. The multi-UAV trajectory optimization problem is formulated as a sequential decision making problem, and a decentralized DRL approach is proposed to solve the problem. Moreover, multiple UAV trajectory design in cellular networks with a dynamic jammer is studied, and a learning-based algorithm is proposed. Subsequently, a UAV trajectory optimization problem is considered to maximize the collected data from multiple Internet of things (IoT) nodes under realistic constraints. The problem is translated into a Markov decision process (MDP) and dueling double deep Q-network (D3QN) is proposed to learn the decision making policy