The lms, pnlms, and exponentiated gradient algorithms

Sparse impulse responses are encountered in many applica-tions (network and acoustic echo cancellation, feedback can-cellation in hearing aids, etc). Recently, a class of exponen-tiated gradient (EG) algorithms has been proposed. One of the algorithms belonging to this class, the so-called EG ± al-gorithm, converges and tracks much better than the classi-cal stochastic gradient, or LMS, algorithm for sparse impulse responses. In this paper, we show how to derive the differ-ent algorithms. We analyze the EG ± algorithm and explain when to expect it to behave like the LMS algorithm. It is also shown that the proportionate normalized LMS (PNLMS) al-gorithm proposed by Duttweiler in the context of network echo cancellation is an approximation of the EG±. 1