Deep Learning for Wireless Multi-User Multi-Antenna Communications

Multi-user multiple-input multiple-output (MU-MIMO) can significantly improve the system capacity, spectrum efficiency, and link reliability by multiplexing different user terminals' (UT) transmissions in the spatial domain. However, demultiplexing the transmitted signal at the receiver side, known as MU-MIMO detection, can become a signal processing challenge when the user load in the spatial domain is high. In the past two decades, enormous research effects have been paid towards achieving a good performance-complexity trade-off. Recently, deep learning technologies have been introduced into this domain, where neural networks are employed to replace partially or fully the conventional function inside the MU-MIMO detection. Deep learning-based MIMO (deep-MIMO) detection is appealing in the sense that it has the potential to offer: 1) better support to parallel computing; 2) low online computation-complexities which are comparable with that of linear MIMO receivers; and 3) native optimization for specific wireless channels. In the literature, most of the existing deep-MIMO detection approaches belong to the family of model-driven detection, where their performance depends heavily on their conventional models. Moreover, they are mostly coherent detection, which either assume perfect channel knowledge or require channel estimation. In this case, the channel estimation overhead becomes a big problem in delay-sensitive communications. Motivated by the above observations, major contributions of this thesis are presented:First, a number of data-driven deep-MIMO detection approaches are developed. The development starts from a simple detection network, termed DDNet. It has been shown that DDNet can achieve near-optimum performance in simple MU-MIMO networks. However, our theoretical work reveals that DDNet is challenged by the signal processing scalability problem with respect to the number of active users. In order to scale up the data-driven deep-MIMO detection, a modular neural network approach, termed ModNet, is developed. It is shown that ModNet offers scalable detection performances for both under- and fully- loaded MIMO systems, while other approaches can perform well mainly for under-loaded MIMO systems. Moreover, ModNet demonstrates remarkable performance in over-loaded MIMO systems with a performance gap less than 1.5 dB to the optimum MLSD at BER of 10^{-2}. Even so, we show that existing deep-MIMO detection approaches suffer from channel over-training. To solve this problem, an orthogonal stochastic gradient descent (O-SGD) algorithm is developed, which enables single ModNet to efficiently work under multiple channel models.Second, an end-to-end learning approach for MU-SIMO joint transmitter and non-coherent receiver design, termed JTRD-Net, where transmitter side consists of a group of parallel linear layers for multiuser waveform design. The non-coherent receiver is modeled as a feed-forward deep neural network (DNN) so as to provide multiuser detection (MUD) capabilities. The entire JTRD-Net can be trained from end to end through deep learning. After training, JTRD-Net can work efficiently in a non-coherent manner. Simulation results show that JTRD-Net outperforms the existing pilot-based solutions and non-coherent detection approaches for at least 2 dB in complex Gaussian channels and 4.5 dB in Kronecker MIMO channels thanks to the coding gain offered by joint transmitter and receiver design