Adapting: From Wiener To Widrow

It is shown that the commonly used adaptive algorithms are closely related to each other and can be derived from one another. The Wiener and LMS/Newton statistical approach to the optimum design of filters is presented first as a starting point. Next, several families of fast block and fast sequential algorithms are introduced where each family is derived from the preceding one. These are: the optimum block adaptive algorithms with individual adaptation of parameters (OBAI), the optimum block adaptive algorithms, the sequential optimum adaptive algorithms, the individual (IA) and the homogeneous (HA) algorithm, and the Widrow-Hoff least mean square algorithm (LMS). Progressing from one algorithm to the next, the gradual transition in the trade-off between the computational complexity, the length of the processed data record, and adaptation performance, such as speed and accuracy, is demonstrated. Comparative discussions supported with computer results are given