Multi-antenna signal processing strategies and performance analysis for wireless communication systems

Multiple-input-multiple-output (MIMO) technology is well known as a promising solution to provide high data rate and link reliability for wireless communication systems. Over the past decades, it has been extensively investigated by the research community. However, there are still many important open problems. The goals of this thesis are to design practical signal processing strategies and study the corresponding analytical performance for some important MIMO communication scenarios. Specifically, we will consider the following scenarios: MIMO systems with cochannel interference: We first investigate the analytical performance of MIMO systems employing joint linear transmit-receive processing in the presence of unequal power cochannel interference and noise. Based on new closed-form exact and asymptotic expressions which we derive for the marginal ordered eigenvalue distributions of a certain class of finite-dimensional complex random matrices, new exact expressions are derived for the symbol error rate (SER) of each eigen-subchannel, the global SER, and the outage probability. In addition, we propose a simple adaptive transmission method to maximize the spectral efficiency whilst meeting a target bit error rate (BER) constraint. Tractable expression of error free rate for the proposed scheme is derived using the obtained results of random matrix theory. Downlink multiuser MIMO systems (or MIMO broadcast channel): We investigate the joint design of low complexity transceivers and multiuser scheduling algorithms to achieve the capacity of MIMO broadcast channel. We propose two low complexity eigenmode-based transmission techniques for the MIMO broadcast channel, employing greedy semi-orthogonal user selection (SUS).We first employ new analytical methods to prove that as the number of users K grows large, the average sum rates of both techniques converge to the average sum capacity of the MIMO broadcast channel. Our results also provide key insights into the benefit of multiple receive antennas and the effect of the SUS algorithm. We also show a very interesting result that the semi-orthogonality constraint imposed by SUS, whilst facilitating a very low complexity user selection procedure, asymptotically does not reduce the multi-user diversity gain in either first (log K) or second-order (log log K) terms. Relaying for MIMO communication: We endeavor to design relaying protocol with low implementation complexity for MIMO wireless communications. In contrast to the conventional approaches where all of the relays in the network participate, we propose an opportunistic relaying protocol in which only a few very important relays (VIRs) are selected to share the total power. We also propose two low complexity relay selection algorithms to determine the VIRs, and obtain the corresponding network capacity scaling laws achieved. Our results show that, compared to the conventional approach, the proposed opportunistic relaying protocol can achieve much higher capacity when the number of relay nodes is reasonably large. MIMO system with antenna mutual coupling: We deal with the optimal transmitter design for MIMO wireless systems with antenna mutual coupling. Motivated by practical considerations, the transmitter optimization is based on maximizing the capacity subject to two separate power constraints; namely, a constraint on the available power at the transmitter, and a constraint on the allowable radiated power. We present a solution to the dual-constraint problem based on dynamic water-filling and mode dropping. By effectively accounting for the antenna mutual coupling, our proposed optimal transmission approach is shown to yield significant improvements in capacity in comparison to conventional techniques which ignore this effect, and are designed based on available power constraint only