Optimal design of magnitude responses of rational infinite impulse response filters

This correspondence considers a design of magnitude responses of optimal rational infinite impulse response (IIR) filters. The design problem is formulated as an optimization problem in which a total weighted absolute error in the passband and stopband of the filters (the error function reflects a ripple square magnitude) is minimized subject to the specification on this weighted absolute error function defined in the corresponding passband and stopband, as well as the stability condition. Since the cost function is nonsmooth and nonconvex, while the constraints are continuous, this kind of optimization problem is a nonsmooth nonconvex continuous functional constrained problem. To address this issue, our previous proposed constraint transcription method is applied to transform the continuous functional constraints to equality constraints. Then the nonsmooth problem is approximated by a sequence of smooth problems and solved via a hybrid global optimization method. The solutions obtained from these smooth problems converge to the global optimal solution of the original optimization problem. Hence, small transition bandwidth filters can be obtained