Advanced direction-of-arrival estimation and beamforming techniques for multiple antenna systems

In this thesis, we develop advanced techniques and concepts for direction-of-arrival (DOA) estimation and beamforming. We study narrowband high-resolution search-free DOA estimation methods that can be applied in the case of arbitrary array geometries. We derive an asymptotic first-order performance analysis of the popular manifold separation (MS) and interpolated root-MUSIC techniques, which takes into account the finite sample effect as well as manifold approximation errors. Moreover, we propose two rooting-based DOA estimators for arbitrary arrays. It is demonstrated by means of computer simulations that the proposed estimators provide attractive tradeoffs between DOA estimation performance and computational complexity. We also develop a novel array geometry design for azimuthal DOA estimation. The proposed array design stems from the design of minimum redundancy arrays (MRAs), but the sensors are not required to lie on a uniform grid. The proposed array design facilitates a novel subspace-based DOA estimation technique, which allows estimating the DOAs of more uncorrelated sources than there are sensors, using only second-order statistics of the received data. Furthermore, we study robust adaptive beamformers for narrowband and broadband signals. In the narrowband case, we show that the popular beamformer based on one-dimensional (1D) covariance fitting leads to inherently non-optimum results in the presence of interferers. To mitigate the detrimental effect of interferers, we extend the 1D covariance fitting approach to multi-dimensional (MD) covariance fitting, modeling the source steering vectors by means of uncertainty sets. The proposed MD covariance fitting approach leads to a non-convex optimization problem. We develop a convex approximation of this problem, which can be solved, for example, by means of the logarithmic barrier method. The complexity required to compute the barrier function and its first- and second-order derivatives is derived. Simulation results show that the proposed beamformer based on MD covariance fitting achieves improved performance as compared to the state-of-the-art narrowband beamformers in scenarios with large sample support. In the broadband case, we develop two finite impulse response (FIR) beamformers based on worst-case output power minimization, which use different constraints to maintain the desired signal. These constraints strictly limit the sensitivity to signal steering vector estimation errors. Additionally, these constraints lead to an incentive for a low sensitivity if the true signal steering vectors lie within the presumed uncertainty sets. This incentive becomes stronger with increasing signal powers. We study the relation between the proposed FIR beamformers and the norm-bounded broadband minimum variance distortionless response (MVDR) beamformer. Furthermore, we develop the discrete Fourier transform (DFT) beamformer counterparts of the proposed FIR beamformers. The proposed FIR beamformers control the frequency response towards the desired signal only for a finite set of frequencies. Based on the theory of positive trigonometric polynomials, we also develop a modified version of the first proposed FIR beamformer, which avoids the frequency discretization and associated errors. Our simulation results verify that the proposed beamformers are attractive alternatives to the current state-of-the-art broadband beamformers. In particular, the proposed FIR beamformers provide a significantly improved capability to suppress interferers and noise as compared to previous FIR beamformers based on worst-case output power minimization