Noisy Data and Impulse Response Estimation

Abstract—This paper investigates the impulse response estima-tion of linear time-invariant (LTI) systems when only noisy finite-length input-output data of the system is available. The competing parametric candidates are the least square impulse response esti-mates of possibly different lengths. It is known that the presence of noise prohibits using model sets with large number of parame-ters as the resulting parameter estimation error can be quite large. Model selection methods acknowledge this problem, hence, they provide metrics to compare estimates in different model classes. Such metrics typically involve a combination of the available least-square output error, which decreases as the number of parame-ters increases, and a function that penalizes the size of the model. In this paper, we approach the model class selection problem from a different perspective that is closely related to the involved de-noising problem. The method primarily focuses on estimating the parameter error in a given model class of finite order using the available least-square output error. We show that such an estimate, which is provided in terms of upper and lower bounds with certain level of confidence, contains the appropriate tradeoffs between the bias and variance of the estimation error. Consequently, these mea-sures can be used as the basis for model comparison and model se-lection. Furthermore, we demonstrate how this approach reduces to the celebrated AIC method for a specific confidence level. The performance of the method as the noise variance and/or the data length varies is explored, and consistency of the approach as the data length grows is analyzed. Index Terms—Least square estimate, LTI system modeling, noisy data. I