Numerical approximations for nonlinear stochastic systems with delays, Stochastics: An

We extend the numerical methods of [10], known as the Markov chain approximation methods, to controlled general nonlinear delayed reflected diffusion models. The path and the control can both be delayed. For the no-delay case, the method covers virtually all models of current interest. The method is robust, the approximations have physical interpretations as control problems closely related to the original one, and there are many effective methods for getting the approximations, and for solving the Bell-man equation for low-dimensional problems. These advantages carry over to the delay problem. It is shown how to adapt the methods for get-ting the approximations, and the convergence proofs are outlined for the discounted cost function. Extensions to all of the cost functions of cur-rent interest as well as to models with Poisson jump terms are possible. The paper is particularly concerned with representations of the state and algorithms that minimize the memory requirements. Key words: Optimal stochastic control, numerical methods, delay stochastic equa-tions, numerical methods for delayed controlled diffusions, Markov chain approxima-tion method