Data assimilation: a fully nonlinear approach to ensemble formation using indistinguishable states

Operational forecasting with simulation models involves the melding of observations and model dynamics to determine a set of initial conditions for each forecast. The Kalman filter (KF) provides the optimal closed-form solution to a general linear stochastic (perfect model) case, while the target of the problem has not even been defined in the case of imperfect models. Data assimilation in a nonlinear, perfect-model scenario is considered. It is shown that a new fully nonlinear approach based upon the indistinguishable states (IS) systematically outperforms the ensemble Kalman filter (EnKF). The IS provides an ensemble of initial conditions, consistent with (i) the model dynamics, (ii) the observational noise model, and (iii) the particular observations over a window. It is argued that this is the relevant limit to consider in data assimilation, when the desire is to place high probability density in the vicinity of the target state. The advantages of the IS approach come in part from its ability to provide attractor-balanced ensembles near any attracting manifold the system may evolve on. The use of an EnKF provides a computationally cheaper alternative that place points in the general vicinity of the target. A low (i.e., 2) dimensional example is used to provide easily visualized evidence for these claims, which are then tested in a higher (i.e., 12) dimensional system. Inasmuch as the IS approach is shown to outperform the EnKF systematically in these perfect-model experiments, it provides an interesting alternative approach when informative ensembles are desired