Measurement System Characterization in the Presence of Measurement Errors

In the calibration of a measurement system, data are collected in order to estimate a mathematical model between one or more factors of interest and a response. Ordinary least squares is a method employed to estimate the regression coefficients in the model. The method assumes that the factors are known without error; yet, it is implicitly known that the factors contain some uncertainty. In the literature, this uncertainty is known as measurement error. The measurement error affects both the estimates of the model coefficients and the prediction, or residual, errors. There are some methods, such as orthogonal least squares, that are employed in situations where measurement errors exist, but these methods do not directly incorporate the magnitude of the measurement errors. This research proposes a new method, known as modified least squares, that combines the principles of least squares with knowledge about the measurement errors. This knowledge is expressed in terms of the variance ratio - the ratio of response error variance to measurement error variance. The variance ratio takes on values between 0 and 1, and for calibration applications, the ratio is typically less than 0.0625. In addition to modified least squares, a new definition of residual errors based on the variance ratio is proposed. Through several simulation studies, it is observed that the new estimator can yield different estimates of the regression coefficients and improve the residual error over ordinary least squares. As a result, modified least squares is shown to be an alternative estimation method in the presence of measurement errors