Data management of river water quality data: a semi-automatic procedure for data validation

Monitoring networks typically generate large amounts of data. Before the data can be added to the database, they have to be validated. In this paper, a semi-automatic procedure is presented to validate river water quality data. On the basis of historical data, additive models are established to predict new observations and to construct prediction intervals ( PI's). A new observation is accepted if it is located in the interval. The coverage of the prediction intervals and its power to detect anomalous data are assessed in a simulation study. The method is illustrated on two case studies in which the method detected abnormal nitrate concentrations in the water body provoked by a dry summer which was followed by an extreme winter period. The case studies also show that similar to classical multivariate outlier detection tools, the semi-automatic procedure allows the detection of suspicious observations lying at the edges as well as observations lying at the center of the univariate distribution of the observations, but, without having to impose linear relationships typically associated with these classical methods.Monitoring networks typically generate large amounts of data. Before the data can be added to the database, they have to be validated. In this paper, a semi-automatic procedure is presented to validate river water quality data. On the basis of historical data, additive models are established to predict new observations and to construct prediction intervals ( PI's). A new observation is accepted if it is located in the interval. The coverage of the prediction intervals and its power to detect anomalous data are assessed in a simulation study. The method is illustrated on two case studies in which the method detected abnormal nitrate concentrations in the water body provoked by a dry summer which was followed by an extreme winter period. The case studies also show that similar to classical multivariate outlier detection tools, the semi-automatic procedure allows the detection of suspicious observations lying at the edges as well as observations lying at the center of the univariate distribution of the observations, but, without having to impose linear relationships typically associated with these classical methods.A