Predicting conditional probability densities of stationary stochastic time series

Feedforward neural networks applied to time series prediction are usually trained to predict the next time step x(t + 1) as a function of m previous values, x(t) := (x(t), x(t + 1),…, x(t − m + 1)), which, if a sum-of-squares error function is chosen, results in predicting the conditional mean 〈y|x(t)〉. However, further information about the distribution is lost, which is a serious drawback especially in the case of multimodality, where the conditional mean alone turns out to be a rather insufficient or even misleading quantity. The only satisfactory approach in the general case is therefore to predict the whole conditional probability density for the time series, P(x(t + 1)|x(t), x(t − m 1),…, x(t − + 1)). We deduce here a two-hidden-layer universal approximator network for modelling this function, and develop a training algorithm from maximum likelihood. The method is tested on three time series of different nature, which will demonstrate how state-space dependent variances and multimodal transitions can be learned. We will finally note comparisons with other recent neural network approaches to this problem, and will state results on a benchmark problem