Spectrally efficient multicarrier communication systems: signal detection, mathematical modelling and optimisation

This thesis considers theoretical, analytical and engineering design issues relating to non-orthogonal Spectrally Efficient Frequency Division Multiplexing (SEFDM) communication systems that exhibit significant spectral merits when compared to Orthogonal FDM (OFDM) schemes. Alas, the practical implementation of such systems raises significant challenges, with the receivers being the bottleneck. This research explores detection of SEFDM signals. The mathematical foundations of such signals lead to proposals of different orthonormalisation techniques as required at the receivers of non-orthogonal FDM systems. To address SEFDM detection, two approaches are considered: either attempt to solve the problem optimally by taking advantage of special cases properties or to apply sub-optimal techniques that offer reduced complexities at the expense of error rates degradation. Initially, the application of sub-optimal linear detection techniques, such as Zero Forcing (ZF) and Minimum Mean Squared Error (MMSE), is examined analytically and by detailed modelling. To improve error performance a heuristic algorithm, based on a local search around an MMSE estimate, is designed by combining MMSE with Maximum Likelihood (ML) detection. Yet, this new method appears to be efficient for BPSK signals only. Hence, various variants of the sphere decoder (SD) are investigated. A Tikhonov regularised SD variant achieves an optimal solution for the detection of medium size signals in low noise regimes. Detailed modelling shows the SD detector to be well suited to the SEFDM detection, however, with complexity increasing with system interference and noise. A new design of a detector that offers a good compromise between computational complexity and error rate performance is proposed and tested through modelling and simulation. Standard reformulation techniques are used to relax the original optimal detection problem to a convex Semi-Definite Program (SDP) that can be solved in polynomial time. Although SDP performs better than other linear relaxations, such as ZF and MMSE, its deviation from optimality also increases with the deterioration of the system inherent interference. To improve its performance a heuristic algorithm based on a local search around the SDP estimate is further proposed. Finally, a modified SD is designed to implement faster than the local search SDP concept. The new method/algorithm, termed the pruned or constrained SD, achieves the detection of realistic SEFDM signals in noisy environments