Non-asymptotic confidence regions for errors-in-variables systems

© 2020 Masoud Moravej KhorasaniSystem identification deals with the problem of building mathematical models of dynamical systems based on observed data. As data has become the main source of information in many settings nowadays and models of dynamical systems are used in most fields of science and engineering, system identification is of great practical relevance. However, the obtained model is of little use without a statement about the uncertainty assigned with the model. Finite-sample system identification (FSID) methods provide guaranteed confidence regions for the unknown model parameter of dynamical systems under mild statistical assumptions} for a finite number of data points. In this thesis, two FSID methods, the Leave-out Sign-dominant Correlation Region (LSCR) and Sign-Perturbed Sums (SPS) methods are extended to Errors-In-Variables (EIV) systems. In EIV systems the measurements of both the input and the output are corrupted by noise. The LSCR and SPS methods require a sequence which is independent corresponding to the true parameter but correlated when the model parameter is different from the true system parameter to construct the confidence region. In standard systems where the noise-free input is available, the data generating system is noise-invertible in the sense that the noise signals can be recovered from the measured signals given the true system. The recovered noise signal can play the role of the required sequence in the LSCR and SPS methods. However, EIV systems are not noise-invertible. Hence, standard FSID methods are not applicable. In this thesis, building on the LSCR and SPS methods three different approaches to the construction of the confidence regions for EIV systems have been developed. In the first approaches, all exiting signals are assumed to be i.i.d. Gaussian processes. An appropriate correlation sequence required for both LSCR and SPS is computed by a Kalman filter and accordingly, a state-space form of the EIV system where both input and output are regarded as outputs is utilized. The constructed confidence regions include the true parameter with a user-chosen probability, and parameter values different from the true ones will be left out of the confidence region as the number of data points increases. The theoretical results are illustrated in simulation examples. In the second approach, the input and the input noise are i.i.d. Gaussian process while there are no restrictive assumptions on the output noise. By utilizing an alternative regression model and swapping the role of the input and the prediction error in the LSCR and SPS methods, confidence regions are constructed which include the true model parameter with a guaranteed user-chosen probability. It is shown that the confidence regions are asymptotically included in an epsilon-neighborhood of the true parameter. The shape and size of the confidence regions are investigated, and an ellipsoidal approximation which can be computed at low computational cost is proposed for the SPS confidence region. The methods and their properties are illustrated in numerical experiments. Finally, the importance of constructing a confidence region for a module in a network of dynamical systems is the motivation behind the third approach. In dynamic networks, all measurements are usually contaminated by noise, and an EIV model is a natural representation of a module in the network. In a dynamic network, the input of a module is not usually an independent sequence since it is typically the output of another dynamical system. The third approach in this thesis constructs confidence regions for EIV systems without making any assumption on the true input signal. The approach is used to construct a confidence region for a single module in a simple cascade network by incorporating additional data and taking advantage of the cascade structure. This is done without estimating other modules in the network. The method is illustrated in numerical experiments